Sitemaps during a migration - which is the best way of dealing with them?
-
Many SEOs I know simply upload the new sitemap once the new site is launched - some keep the old site's URLs on the new sitemap (for a while) to facilitate the migration - others upload both the old and the new website together, to support the migration. Which is the best way to proceed? Thanks, Luke
-
Very much appreciated CleverPhD!
-
Found this while looking for a answer for another question could not find this the other day- right from the mouth of Google to not include pages that do not exist in XML sitemaps.
http://googlewebmastercentral.blogspot.com/2014/10/best-practices-for-xml-sitemaps-rssatom.html
URLs
URLs in XML sitemaps and RSS/Atom feeds should adhere to the following guidelines:
- Only include URLs that can be fetched by Googlebot. A common mistake is including URLs disallowed by robots.txt — which cannot be fetched by Googlebot, or including URLs of pages that don't exist.
-
Mate nailed it completely!
-
I would say make sure that your new sitemap has all the latest URLs. The reason people say that you should have old URLs in the sitemap is so that Google can quickly crawl the old URLs to find the 301s to the new URLs.
I am not convinced that this helps. Why?
Google already has all your old URLs in its systems. You would be shocked how far back Google has data on your site with old URLs. I have a site that is over 10 years old and I still see URL structures referenced in Google from 7 years ago that have a 301 in place. Why is this?
Google will assume that, "Well, I know that this URL is a 301 or 404, but I am going to crawl it every once in a while just to make sure the webmaster did not do this by mistake." You can notice this in Search Console error or link reports when you setup 301s or 404s, they may stay in there for months and even come back once they fall out of the error list. I had an occurrence where I had some old URLs showing up in the SERPs and various Search Console reports for a site for 2 years following proper 301 setups. Why was this happening?
This is a large site and we still had some old content still linking to the old URLs. The solution was to delete the links in that old content and setup a canonical to self on all the pages to help give a definitive directive to Google. Google then finally replaced the old URLs with the new URLs in the SERPs and in the Search Console reports. The point here being that previously our site was giving signals (links) that told Google that some of the old URLs were still valid and Google was giving us the benefit of the doubt.
If you want to have the new URLs seen by Google, show them in your sitemap. Google already has all the old URLs and will check them and find the 301s and fix everything. I would also recommend the canonical to self on the new pages. Don't give any signals to Google that your old URLs are still valid by linking to them in any way, especially your sitemap. I would even go so far as to reach out to any important sites that link to old URLs to ask for an updated link to your site.
As I mentioned above, I do not think there is an "advantage" of getting the new URLs indexed quicker by putting old URLs in the sitemap that 301 to the new URLs. Just watch your Google Search Console crawl stats. Once you do a major overhaul, you will see Google really crawl your site like crazy and they will update things pretty quick. Putting the old URLs in the sitemap is a conflicting signal in that process and has the potential to slow Google down IMHO.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best way to handle deletion of a forum subdomain?
Hello All Our site www.xxxx.com has long had a forum subdomain forum.xxxx.com. We have decided to sunset the forum. We find that the 'Ask a Question' function on product pages and our social media presence are more effective ways of answering customers' product & project technical Qs. Simply shutting down the forum server is going to return thousands of 404s for forum.xxxx.com, which I can't imagine would be helpful for the SEO of www.xxxx.com even though my understanding is that subdomains are sort of handled differently than the main site. We really tremendously on natural search traffic for www.xxxx.com, so I am loathe to make any moves that would hurt us. I was thinking we should just keep the forum server up but return 410s for everything on it, including the roughly ~3,000 indexed pages until they are removed from the index, then shut it down. The IT team also gave the option of simply pointing the URL to our main URL, which sorta scares me because it would then 200 and return the same experience hitting it from forum.xxxx.com as www.xxxx.com, which sounds like a very bad idea. (Yes, we do have canonicals on www.xxxx.com). In your opinion, what is the best way to handle this matter? Thank You
Intermediate & Advanced SEO | | jamestown0 -
Best way to get love from expired domain
Hi so i have bought a couple of expired domains (which when they were live had good PA , link profile etc) my question is is it better to just 301 the domains to my main site or to set up a "landing page" for the expired domain and then put a number of links on this page to my main site which would give me the most benefit google-wise ? obviously 301ing would be the easiest !
Intermediate & Advanced SEO | | odysseytravels0 -
Should I migrate .co.uk to .com?
I have previously searched the forum and could not find a definitive answer on this subject so would appreciate any guidance. I have just joined a new company, we have a .co.uk site which gets lots of traffic. We have a .com site which is targeting USA and .com/de/ targeting Germany. 'hreflang' is configured on the .com (between the USA and German sites) but not on .co.uk. This means that in the eyes of search engines (and Moz Pro) the 2 domains are competitors (and the .co.uk has much more presence than the .com in the USA). I know how to fix this and I am in the process of doing so. My question is whether it would make sense to migrate the .co.uk site to .com As previously mentioned the .co.uk site already does very well both in the UK and around the world (as our product is well known in our niche). As .co.uk can only primarily be targeted to UK would our global reach increase enough to justify migrating it to .com? We have dealers/distributors in maybe 30 countries and are continuing to expand, we will at point point add additional languages so my suggestion is that we migrate now as the authority of the .co.uk will help the emerging markets as well as increase our visibility in markets that are not currently primary targets. We are also in the process of hiring new staff specifically to focus on Content Marketing. So again this suggests having the 1 domain will make sense in the long run (as any value gained from content marketing success will be seen by all country/language focussed sites). I am also planning to rebuild the sites in the next few months as the current ones are not fit for purpose so the migration would coincide with this (I know this is not ideal). Apologies for the lengthy question, I hope the additional background information will help in providing some feedback to help me make the decision. David
Intermediate & Advanced SEO | | JamesCrossland0 -
Best Way to Create SEO Content for Multiple, International Websites
I have a client that has multiple websites for providing to other countries. For instance, they have a .com website for the US (abccompany.com), a .co.uk website for the UK (abccompany.co.uk), a .de website for Germany (abccompany.de), and so on. The have websites for the Netherlands, France, and even China. These all act as separate websites. They have their own addresses, their own content (some duplicated but translated), their own pricing, their own Domain Authority, backlinks, etc. Right now, I write content for the US site. The goal is to write content for long and medium tail keywords. However, the UK site is interested in having myself write content for them as well. The issue I'm having is how can I differentiate the content? And what is the best way to target content for each country? Does it make sense to write separate content for each website to target results in that country? The .com site will still show up in UK web results still fairly high. Does it make sense to just duplicate the content but in a different language or for the specific audience in that country? I guess the biggest question I'm asking is, what is the best way of creating content for multiples countries' search results? I don't want the different websites to compete with each other in a sense nor do I want to spend extra time trying to rank content for multiple sites when I could just focus on trying to rank one for all countries. Any help is appreciated!
Intermediate & Advanced SEO | | cody1090 -
Invest in a Image Sitemap - Yes or No?
Hey Mozers, 2 part question I'm reaching out to see if you all think Image Sitemaps are totally worth it for a big company. I can totally understand its value for a smaller mom & pop company. With a larger company they would have way more products so is it worth it having an image site map? I cant find examples of image sitemaps online. Would you be able to provide a website that is doing it? I can only find video sitemaps.
Intermediate & Advanced SEO | | rpaiva0 -
Dealing with 404s during site migration
Hi everyone - What is the best way to deal with 404s on an old site when you're migrating to a new website? Thanks, Luke
Intermediate & Advanced SEO | | McTaggart0 -
Worth it to redirect .swf during a website migration?
Recently one of our clients switched web providers. We've been implementing 301 redirects to the new URL structure for the most important pages. We noticed upon doing a site search, Google has indexed a .swf URL on page 3 of search results. Please see attached image. 1. Should I implement a redirect for this? There are no links to this filetype.
Intermediate & Advanced SEO | | EEE3
2. There are three others (.swf) that are indexed I would think the best thing to do is to have the new website return a 404 for this page, as it's currently doing, and it will eventually drop out of the index. Any other suggestions? Thanks! g2tur4r0 -
Best practice?
Hi there, I have recently written an article which I have posted on an online newspaper website. I want to use this article and put it on my blog also, the reason the article will be placed on my blog is to drive users from my email marketing activities. Would it simply be best practice to disallow Google from crawling this page? or put a rel canonical on the article placed on my blog pointing to the article placed on the online newspaper website? Thanks for any suggestions
Intermediate & Advanced SEO | | Paul780