Development site accidentally got indexed and now appears in SERPs. How to fix?
-
I work at a design firm, and we just redesigned a website for a client. When it came time for the coding, we initially built a development site to work out all the kinks before going live. Then we relaunched the actual site about a week ago.
Here's the problem: Somehow, the developer who coded the site for us (a freelancer) allowed the development site to be indexed by Google.
Now, when you enter the client's name into Google, the development site appears higher in the results pages than the real site! In fact, the real site isn't even in the top 50 search results. The client is understandably angry about this for multiple reasons.
We quickly added a robots.txt file to the development site and a 301 redirect to the real site. However, that did seemed to have no effect on the problem.
Any ideas on how to fix this mess? Thank you in advance!
-
For the future, I'd suggest using something like Code Monitor (https://polepositionweb.com/roi/codemonitor/index.php) to monitor for any changes to your live or staging robots.txt. I've been in situations where too many people had access to the site that didn't know what they were doing, and getting an alert within 24 hours that someone changed something in robots.txt really saved us.
-
This exact same thing happened to me. I wasn't aware of the remove URL tool that others have suggested, so if it helps I'll tell you what I did. I simply added a rel=canonical tag from the development site to the new one. Then, once the new one had regained its SERP I deleted the development site. That way, I was sure that the client's site would have good positions.
-
Both nailed it spot on. Thats the way to go.
-
Use Google Webmaster tools. Under Site Configuration > Crawler Access, there is a "Remove URL" tab that will allow you to request URLs be removed from Google's index.
-
Hi Matt
Through Google Webmaster Tools, you can request that the whole website (the development site) be removed from Google's index. Could speed up the removal process by doing so.
Regards
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moving site from html to Wordpress site: Should I port all old pages and redirect?
Any help would be appreciated. I am porting an old legacy .html site, which has about 500,000 visitors/month and over 10,000 pages to a new custom Wordpress site with a responsive design (long overdue, of course) that has been written and only needs a few finishing touches, and which includes many database features to generate new pages that did not previously exist. My questions are: Should I bother to port over older pages that are "thin" and have no incoming links, such that reworking them would take time away from the need to port quickly? I will be restructuring the legacy URLs to be lean and clean, so 301 redirects will be necessary. I know that there will be link juice loss, but how long does it usually take for the redirects to "take hold?" I will be moving to https at the same time to avoid yet another porting issue. Many thanks for any advice and opinions as I embark on this massive data entry project.
Technical SEO | | gheh20130 -
Can anyone tell me why some of the top referrers to my site are porn site?
We noticed today that 4 of the top referring sites are actually porn sites. Does anyone know what that is all about? Thanks!
Technical SEO | | thinkcreativegroup1 -
Representing categories on my site
My site serves a consumer-focused industry that has about 15-20 well recognized categories, which act as a pretty obvious way to segment our content. Each category supports it's own page (with some useful content) and a series of articles relevant to that category. In short, the categories are pretty focal to what we do. I am moving from DNN to WordPress as my CMS/blog. I am taking the opportunity to review and fix SEO-related issues as I migrate. One such area is my URL structure. On my existing site (on DNN), I have the following types of pages for each topic: / <topic>- this is essentially the landing page for the topic and links to articles</topic> /<topic>/articles/ <article-name>- topics have 3-15 articles with this URL structure</article-name></topic> With WordPress, I am considering moving to articles being under the root. So, an article on (making this up) how to make a widget would be under /how-to-make-a-widget, instead of /<widgets>/article/how-to-make-a-widget I will be using WordPress categories to reflect the topics taxonomy, so I can flag my articles using standard WordPress concepts.</widgets> Anyway, I'm trying to get my head around whether it makes sense to "flatten" my URL structure such that the URLs for each article no longer include the topic (the article page will link to the topic page though). Thoughts?
Technical SEO | | MarkWill1 -
Wrong canonical URL was specified. How to refresh the index now?
Wrong canonical URL was applied to thousands of pages of a client website, pointing them all to a single non-existing URL. Now Google has de-indexed most of those pages. We have fixed the problem now, but do we get Search engines crawl those pages again and start showing in Search results? I understand that a slow recovery is possible if we don't do anything. Was wondering if we can fast track the recovery... Any pointers? Thanks
Technical SEO | | Krupesh0 -
Domain recommendations for non indexed sites
For complicated internal reasons (which I swear have validity) we are starting websites that will then disallow from Google for ~1 year. Is there any value in purchasing "premium" domains with SEO juice or will we lose the value in the time we are hiding the website from Google?
Technical SEO | | theLotter0 -
If you are organizing the site structure for an ecommerce site, how would you do it?
Should you use not use slashes and use all dashes or use just a few slashes and the rest with dashes? For example, domain.com/category/brand/product-color-etc OR domain.com/anythinghere-color-dimensions-etc Which structure would you rather go for and why?
Technical SEO | | Zookeeper0 -
Www. version of my site shows nothing in Open Site Explorer
When I first setup my site the domain was learnbonds.com. I moved hosts a couple of months ago and as part of the process I asked them to make the site show as www.learnbonds.com which they did. Now however when I goto www.learnbonds.com in open site explorer it says there is no data. When I enter learnbonds.com into open site explorer it gives me data but says that the site has been redirected to the www. version which shows no data. Also in google webmaster when I try to set the preferred domain as the www. version it gives me the following message: Part of the process of setting a preferred domain is to verify that you own http://www.learnbonds.com/. Please verify http://www.learnbonds.com/. I am concerned that this is hurting my SEO and would appreciate any advice you can give. Thanks Dave
Technical SEO | | fxtrader19790 -
How to find all the links to my site
hi i have been trying to find all the links that i have to my site http://www.clairehegarty.co.uk but i am not having any luck. I have used the open explorer but it is not showing all the links but when i go to my google webmaster page it shows me more pages than it does on the semoz tool. can anyone help me sort this out and find out exactly what links are going into my site many thanks
Technical SEO | | ClaireH-1848860