Site being indexed by Google before it has launched
-
We are currently coming towards the end of migrating one of our retail sites over to magento. To our horror, we find out today that some pages are already being indexed by Google, and we have started receiving orders through new site. Do you have any suggestions for what may have caused this? Or similarly, what the best solution would be to de-index ourselves? We most recently excluded anything with a certain parameter from robots.txt - could this being implemented incorrectly have caused this issue?
Thanks
-
Google has ignored a robots.txt for my site because it thought we were "hiding" important content.
Depending on your site structure, putting a "no index" tag on your page is also a good idea. Doing that as pages are built can prevent it (better) from being indexed in the first place. Then you remove the tag when you are ready to go live. And, submit your updated sitemap via GWT when you finish your move so Google will know what pages to index.
-
I would be very careful on how you manage this right now. It all depends on the old and new urls. If this is something in Dev staging, like http://dev.examplesite.com, then it is alright to do everything possible to clean them up. But if the new urls are going to be the new final urls, then you have to be careful.
The best thing you could do right now would be to password protect the new site. That way there isn't a chance of anything getting to it.
All it takes is Google finding one link to the site then it will crawl from there.
-
Exclude the whole new site using robots.txt, and remove the current new site-indexed URL's within Webmaster Tools: Optimization > Remove URL's.
When the new site is ready to launch, undo these removed URL's using the "Reinclude" button, also found within the Remove URL's section of Webmaster Tools. Do this prior to 301'ing the old site pages to the new, and you will probably wish to announce the new site launch via social media as well to encourage indexation. +1'ing a page is considered the fastest way to get it indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Sites website https://www.opcfitness.com/ title NOT GOOD FOR SEO
We set up a website https://www.opcfitness.com/home on google sites. but google sites page title not good for SEO. How to fix it?
Technical SEO | | ahislop5740 -
Google is still indexing the old domain a year after 301 redirects are put in place
Hi there, You might have experienced this before but for me this is the first. A client of mine moved from domain A (www.domainA.com) to domain B (www.domainB.com). 301 redirects are all in place for over a year. But the old domain is still showing in Google when you search for "site:domainA.com" The HTTP Header check shows this result for the URL https://www.domainA.com/company/cookie-policy.aspx HTTP/1.1 301 Moved Permanently =>
Technical SEO | | iQi
Cache-Control => private
Content-Length => 174
Content-Type => text/html; charset=utf-8
Location => https://www.domain_B_.com/legal/cookie-policy
Server => Microsoft-IIS/10.0
X-AspNetMvc-Version => 5.2
X-AspNet-Version => 4.0.30319
X-Powered-By => ASP.NET
Date => Fri, 15 Mar 2019 12:01:33 GMT
Connection => close Does the redirect look wrong? The change of address request was made on Google Console when the website was moved over a year ago. Edit: Checked the domainA.com on bing and it seems that its not indexed, and replaced with domainB.com, which is the right. Just Google is indexing the old domain! Please let me know your thoughts on why this is happening. Best,0 -
Google is indexing our old domain
We changed our primary domain from vivitecsolutions.com to vivitec.net. Google is indexing our new domain, but still has our old domain indexed too. The problem is that the old site is timing out because of the https: Thought on how to make the old indexing go away or properly forward the https?
Technical SEO | | AdsposureDev0 -
Is my website indexed correctly in Google - www.couponshop.co.uk
Our website www.couponshop.co.uk has just had a relaunch after a change of direction.
Technical SEO | | LaurenGT
A lot of the pages were redirected. When I checked the indexing of the website on Google, I put site:couponshop.co.uk and only two pages come up, but when I put site:www.couponshop.co.uk they all show up.
Is this correct or are we doing something wrong?0 -
Why is Google not indexing my site?
I'm a bit confused as to why my site just isn't indexing on Google. Even if I type in my brand name, my social channels rank and there's no evidence of my website. I've followed all of the advice I've read and gone into webmaster tools and got the Wordpress yoast plug-in but nothing seems to be making a difference!One thing I've noticed, in Google Webmaster Tools it says "Couldn’t communicate with the DNS server." in site errors. I've called GoDaddy and they said that everything is fine. A bit frustrating. Trying to work out what my next steps should be but feeling a bit lost to be honest! Any help GREATLY appreciated!
Technical SEO | | j1066s0 -
Does Google differentiate between a site with spammy link building practices from a victim of a negative SEO attack?
I've be tasked with figuring out how to recover our rankings as we are likely being hurt by an algorithmic penalty. I have no idea if this was the workings of a previously hired SEO or the result of negative SEO, **how does Google differentiate between a site with bad/spammy link building practices from a victim of a negative SEO attack? **
Technical SEO | | Syed_Raza0 -
How do we ensure our new dynamic site gets indexed?
Just wondering if you can point me in the right direction. We're building a 'dynamically generated' website, so basically, pages don’t technically exist until the visitor types in the URL (or clicks an on page link), the pages are then created on the fly for the visitor. The major concern I’ve got is that Google won’t be able to index the site, as the pages don't exist until they're 'visited', and to top it off, they're rendered in JSPX, which makes things tricky to ensure the bots can view the content We’re going to build/submit a sitemap.xml to signpost the site for Googlebot but are there any other options/resources/best practices Mozzers could recommend for ensuring our new dynamic website gets indexed?
Technical SEO | | Hutch_e0 -
Google: site gone from SERPs, back in 1 day, then gone again?
Last December we fumbled our 404 error page with a misconfigured server and broken links on the page. Needless to say, our site dropped into an abyss - for 4 months. Yesterday we appeared again in our regular placement (actually stronger placement). Our site has been around since 09-Jan-1999 and has been a highly regarded site with good link structure and solid content for engineers. Then today we're gone again. Yesterday morning we had 125,000 pages indexed with Google which grew to 187,000 by late afternoon. Then this morning we're nowhere to be found and only 21,800 pages are now indexed. We've been working with Bruce Clay Inc through an SEO site audit doing several updates to improve through good seo practices. We haven't made any changes to the site since last Friday, April 13th (maybe the unlucky number has something to do with it....) Any ideas, insight, suggestions???? Thanks!
Technical SEO | | Prospector-Plastics1