Why is the Dev site indexing and not my actual Domain
-
hi guys I had 2 word press sites built but since they went live a couple of things aren't working.
1.when I do a keyword search one of the sites comes back with the actual URL and when I do another keyword search for the same site the DEV site comes back indexed and not the actual URL.
2. The other site originally started indexing with the DEV site and not the URL and the developer tried to fix it and now it doesn't index at all. Its been long enough time for it to index.
Both URL's are live when put into a browser.
Any advice would be great
Thanks
Jamie
-
Thanks David.
-
Thanks David PMing you now.
-
Looks like your dev didn't block the dev site while it was being created. That sucks.
This is easy to fix. Make sure you have an updated sitemap file with recent change dates, and all the final URLs added. Also, go through Google's webmaster tools and fetch each new URL that has been added. If you still have a dev directory or folder on the server, block this with robots.txt.
Sitemap example:
<url><loc>https://DOMAIN.com/subpage/</loc>
<lastmod>2016-09-26T04:14:14Z</lastmod></url>
<priority>1.0</priority> (or whatever priority you want to set for the URLs)
<changefreq>Daily</changefreq>
-
Hi Jamie,
It sounds like the dev site has been removed from index, which is good, but I would check two things straight away on the proper site:
- Make sure your site doesn't have a meta robots "noindex" tags on each page
- Make sure your site isn't blocking crawlers (including search engines) in robots.txt
If this is a WordPress site, check Settings > Reading and make sure that the box that says 'Discourage search engines from indexing this site' is unchecked.
If you aren't sure how to do any of this, send me a PM with the domain names and I'd be happy to take a quick look.
Cheers,
David
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Removing a site from Google index with no index met tags
Hi there! I wanted to remove a duplicated site from the google index. I've read that you can do this by removing the URL from Google Search console and, although I can't find it in Google Search console, Google keeps on showing the site on SERPs. So I wanted to add a "no index" meta tag to the code of the site however I've only found out how to do this for individual pages, can you do the same for a entire site? How can I do it? Thank you for your help in advance! L
Technical SEO | | Chris_Wright1 -
Google Indexing of Site Map
We recently launched a new site - on June 4th we submitted our site map to google and almost instantly had all 25,000 URL's crawled (yay!). On June 18th, we made some updates to the title & description tags for the majority of pages on our site and added new content to our home page so we submitted a new sitemap. So far the results have been underwhelming and google has indexed a very low number of the updated pages. As a result, only a handful of the new titles and descriptions are showing up on the SERP pages. Any ideas as to why this might be? What are the tricks to having google re-index all of the URLs in a sitemap?
Technical SEO | | Emily_A0 -
Moving multiple domains into one domain
Hi, We're currently moving a group of websites (approximately 12) under one domain so we've moved from www.example.de , www.example.co.uk , www.example.com to www.example.com/de www.example.com/uk and so on. However I have read an article online today saying that this can lead to crawling complications. Has anyone done something similar and if there were any issues how did you overcome them? Many thanks
Technical SEO | | Creditsafe0 -
Dev Site Was Indexed By Google
Two of our dev sites(subdomains) were indexed by Google. They have since been made private once we found the problem. Should we take another step to remove the subdomain through robots.txt or just let it ride out? From what I understand, to remove the subdomain from Google we would verify the subdomain on GWT, then give the subdomain it's own robots.txt and disallow everything. Any advice is welcome, I just wanted to discuss this before making a decision.
Technical SEO | | ntsupply0 -
Redirect non www. domain to WWW. domain for established website?
Hey guys, The website in question has been online for more than 5 years but there are still 2 versions of the website. Both versions are indexed by Google and of course, this will result in duplicate content. Is it necessary to redirect the non-www domain to the www. domain. What are the cons and advantages? Will the www. links replace the non-www links when it comes to keyword rankings? Thanks.
Technical SEO | | BruLee0 -
Different domains
Firstly apologies for the very brief question as I am mainly looking for your thoughts as opposed to specific help about a specific problem. I am working on a site which has two sepreate domains and within one domain, two sub domains. The two different sites both havea high page rank, PR6 each, one is the corporate site and the other is the company blog. There are also two sub domains within the corporate site, again both domains have high pr and tons of content. My question is would it be better to consolidate all the assets under one domain or is it better to keep the sites sepreate, from an seo perspective that is.
Technical SEO | | LiquidTech0 -
We're working on a site that is a beer company. Because it is required to have an age verification page, how should we best redirect the bots (useragents) to the actual homepage (thus skipping ahead of the age verification without allowing all browsers)?
This question is about useragents and alcohol sites that have an age verification screen upon landing on the site.
Technical SEO | | OveritMedia0 -
Site not being Indexed that fast anymore, Is something wrong with this Robots.txt
My wordpress site's robots.txt used to be this: User-agent: * Disallow: Sitemap: http://www.domainame.com/sitemap.xml.gz I also have all in one SEO installed and other than posts, tags are also index,follow on my site. My new posts used to appear on google in seconds after publishing. I changed the robots.txt to following and now post indexing takes hours. Is there something wrong with this robots.txt? User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /wp-login.php Disallow: /wp-login.php Disallow: /trackback Disallow: /feed Disallow: /comments Disallow: /author Disallow: /category Disallow: */trackback Disallow: */feed Disallow: */comments Disallow: /login/ Disallow: /wget/ Disallow: /httpd/ Disallow: /*.php$ Disallow: /? Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.wmv$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Disallow: /? Disallow: /*?Allow: /wp-content/uploads User-agent: TechnoratiBot/8.1 Disallow: ia_archiverUser-agent: ia_archiver Disallow: / disable duggmirror User-agent: duggmirror Disallow: / allow google image bot to search all imagesUser-agent: Googlebot-Image Disallow: /wp-includes/ Allow: /* # allow adsense bot on entire siteUser-agent: Mediapartners-Google* Disallow: Allow: /* Sitemap: http://www.domainname.com/sitemap.xml.gz
Technical SEO | | ideas1230