Post Site Migration - thousands of indexed pages, 4 months after
-
Hi all,
Believe me. I think I've already tried and googled for every possible question that I have. This one is very frustrating – I have the following old domain – fancydiamonds dot net.
We built a new site – Leibish dot com and done everything by the book:
- Individual 301 redirects for all the pages.
- Change of address via the GWT.
- Trying to maintain and improve the old optimization and hierarchy.
4 months after the site migration – we still have to gain back more than 50% of our original organic traffic (17,000 vs. 35,500-50,000
The thing that strikes me the most that you can still find 2400 indexed pages on Google (they all have 301 redirects).
And more than this – if you'll search for the old domain name on Google – fancydiamonds dot net you'll find the old domain!
Something is not right here, but I have no explanation why these pages still exist.
Any help will be highly appreciated. Thanks!
-
Thanks Dana. Honestly, we have a lot of experience dealing with site migrations - I read dozens of posts and we've implemented our own step-by=step guidelines for successful site migration.
As you can see, sometimes even when you do everything by the book you can encounter some unexpected issues.
-
Yes I have. I could see the 301 redirects correctly and without any further issues.
-
Yes, it sounds like perhaps there is a technical issue here. I like Keri's suggestion below. Also, have you grepped your server logs to see if Googlebot is having issues?
It can taken Google a long long time to take down search results to old pages that either don't exist any more or that 301 to a new page. You may have to resort to using the removal tool. I realize that for 2,000 URLs doing these one at a time is inconvenient, but it may just be what you have to do.
I have some old notes on domain migration that I'll try to dig up, but unfortunately I don't think there's much there that's helpful after the fact. But I'll see what I can find.
-
the URL remover tool would be one of my last options, since I too would be afraid of any authority vanishing with the old link.
Google must have some reason to continue to index the pages and I wouldn't want them removed until I'm positive I gained back all the authority I could, from these old pages.
-
Are you certain the 301 redirects are active and working?
-
Can you add canonical tags to the 301'ed pages?
-
Make sure that none of the URLs in the 301 URL chain are disallowed by a robots.txt file. If they were in the redirect chain, Google would not be able to properly crawl the new page and properly index.
That last point may be what's preventing a portion of the old URLs from dropping, if they are being blocked in the robots.txt file.
-
-
What happens when you go into GWT and fetch fancydiamonds.net as googlebot? Is there some reason that perhaps googlebot isn't seeing the redirects correctly?
-
Hi David,
see my answer to RaymondPP.
Also, what do you mean by saying "you are linking out to your other site"?
Did you see anything?
-
There is a perfect correlation between the organic drop and the revenue – It has decreased dramatically. Of course I checked for Analytics issue but all the other traffic sources have stayed the same. We have big PPC campaigns and the traffic data is correct.
About management of expectations – usually we say that we expect 3-4 months of traffic droppings, but this had taken us a bit by surprise.
-
Thanks for the answer.
That's always a possibility - the problem is that these url's have not too few links (the old homepage is still indexed!).
If I'll use the url remover won't this result in losing all the link juice for those url's?
-
I agree with both of the previous suggestions and thought I would add a comment and a question too.
Seeing a decline of 50% or even more in traffic after a site migration is not uncommon. Hopefully your clients went into the migration with eyes open, knowing that they could see significantly lower traffic for anywhere from 6 weeks to a year, and maybe never fully recover. This sometimes happens. That's why the planning process is so important (and management of expectations).
That being said, when you installed Google Analytics on the new site, did anything change in your GA tracking code? Sometimes this happens and can lead to old analytics reports and new analytics reports not being an "apples to apples" comparison. It's just a thought. It could be that the traffic isn't actually 50% lower, but has changed much less than that.
Has revenue (or whatever your conversion goal is) dropped, increased or stayed the same?
-
I'm not understanding why your traffic is lower. If you have 301 redirects in place, even if your old pages show up and someone clicks the link, it will take them to the new site.
Another option you could pursue is a 410 (gone) for your old pages. This states to Google that the page has been removed and should no longer be indexed or linked.
But beware, you are linking out to your other site.
The 410 error is primarily intended to assist the task of web maintenance by notifying the client system that the resource is intentionally unavailable and that the Web server wants remote links to the URL to be removed. Such an event is common for URLs which are effectively dead i.e. were deliberately time-limited or simply orphaned. The Web server has complete discretion as to how long it provides the 410 error before switching to another error such as 404.
-
Hi skifr - Have you tried using the URL remover tool in GWT? And if you really want those pages out of the search engine, how about a noindex tag on the old domain?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What are the SEO considerations when migrating a whole site from http to https
Hi Mozzers, I'm in the process of migrating a whole site, which has excellent rankings built through ongoing SEO over the years, from http to https. What is the safest way of doing this, while maintaining rankings? I'm assuming 301 redirect of every page from http to https? Thanks!
Technical SEO | | A_Q1 -
Site indexed by Google, but (almost) never gets impressions
Hi there, I have a question that I wasn't able to give it a reasonable answer yet, so I'm going to trust on all of you. Basically a site has all its pages indexed by Google (I verified with site:sitename.com) and it also has great and unique content. All on-page grades are A with absolutely no negative factors at all. However its pages do not get impressions almost at all. Of course I didn't expect it to be on page 1 since it has been launched on Dec, 1st, but it looks like Google is ignoring (or giving it bad scores) for some reason. Only things that can contribute to that could be: domain privacy on the domain, redirect from the www to the subdomain we use (we did this because it will be a multi-language site, so we'll assign to each country a subdomain), recency (it has been put online on Dec 1st and the domain is just a couple of months old). Or maybe because we blocked crawlers for a few days before the launch? Exactly a few days before Dec 1st. What do you think? What could be the reason for that? Thanks guys!
Technical SEO | | ruggero0 -
What would cause a sudden drop in indexed sitemap pages?
I have made no changes to my site for awhile and on 7/14 I had a 20% drop in indexed pages from the sitemap. However my total indexed pages has stayed the same. What would cause that?
Technical SEO | | EcommerceSite0 -
Drastic increase of indexed pages correlated to rankings loss?
Our ecommerce website has had a drastic increase in indexed pages, and equal loss of Google organic traffic. After 10/1 the number of indexed pages jumped from 240k to 5.7 million by the end of the year, according to GWT. Coincidentally, the sitemap tops at 14,192 pages, with 13,324 indexed. Organic traffic on some top keyphrases began declining by half after 10/26 and ranking (previously placing in the top 5 spots) has dropped to the fifth page of results. This website does produce session id's (/c=) so we been blocking /c=/ in the robots.txt file. We also have a rel=canonical on all pages pointing at the correct url. With all of this in place, traffic hasn't recovered. Is there a correlation between this spike of indexed pages and the lost keyword ranking? Any advice to investigate and correct this further would be greatly appreciated. Thanks.
Technical SEO | | marketing_zoovy.com0 -
How to avoid duplicate content penalty when our content is posted on other sites too ?
For recruitment company sites, their job ads are posted muliple times on thier own sites and even on other sites too. These are the same ads (job description is same) posted on diff. sites. How do we avoid duplicate content penalty in this case?
Technical SEO | | Personnel_Concept0 -
Google Has Indexed Most of My Site, why won't Bing?
We've got 600K+ pages indexed by Google and have submitted our same sitemap.xml's to Bing, but have only seen 100-200 pages get indexed by Bing. Is this fairly typical? Is there anything further we can do to increase indexation on Bing?
Technical SEO | | jamesti0 -
Site not being Indexed that fast anymore, Is something wrong with this Robots.txt
My wordpress site's robots.txt used to be this: User-agent: * Disallow: Sitemap: http://www.domainame.com/sitemap.xml.gz I also have all in one SEO installed and other than posts, tags are also index,follow on my site. My new posts used to appear on google in seconds after publishing. I changed the robots.txt to following and now post indexing takes hours. Is there something wrong with this robots.txt? User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /wp-login.php Disallow: /wp-login.php Disallow: /trackback Disallow: /feed Disallow: /comments Disallow: /author Disallow: /category Disallow: */trackback Disallow: */feed Disallow: */comments Disallow: /login/ Disallow: /wget/ Disallow: /httpd/ Disallow: /*.php$ Disallow: /? Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.wmv$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Disallow: /? Disallow: /*?Allow: /wp-content/uploads User-agent: TechnoratiBot/8.1 Disallow: ia_archiverUser-agent: ia_archiver Disallow: / disable duggmirror User-agent: duggmirror Disallow: / allow google image bot to search all imagesUser-agent: Googlebot-Image Disallow: /wp-includes/ Allow: /* # allow adsense bot on entire siteUser-agent: Mediapartners-Google* Disallow: Allow: /* Sitemap: http://www.domainname.com/sitemap.xml.gz
Technical SEO | | ideas1230 -
What pages of my site does Google rank as the most important?
If I type site:youtube.com into Google, are the results listed by what Google considers to be the most important pages of the site? If I change my sitemap should this order change? Thanks!
Technical SEO | | Seaward-Group0