Sitemaps during a migration - which is the best way of dealing with them?
-
Many SEOs I know simply upload the new sitemap once the new site is launched - some keep the old site's URLs on the new sitemap (for a while) to facilitate the migration - others upload both the old and the new website together, to support the migration. Which is the best way to proceed? Thanks, Luke
-
Very much appreciated CleverPhD!
-
Found this while looking for a answer for another question could not find this the other day- right from the mouth of Google to not include pages that do not exist in XML sitemaps.
http://googlewebmastercentral.blogspot.com/2014/10/best-practices-for-xml-sitemaps-rssatom.html
URLs
URLs in XML sitemaps and RSS/Atom feeds should adhere to the following guidelines:
- Only include URLs that can be fetched by Googlebot. A common mistake is including URLs disallowed by robots.txt — which cannot be fetched by Googlebot, or including URLs of pages that don't exist.
-
Mate nailed it completely!
-
I would say make sure that your new sitemap has all the latest URLs. The reason people say that you should have old URLs in the sitemap is so that Google can quickly crawl the old URLs to find the 301s to the new URLs.
I am not convinced that this helps. Why?
Google already has all your old URLs in its systems. You would be shocked how far back Google has data on your site with old URLs. I have a site that is over 10 years old and I still see URL structures referenced in Google from 7 years ago that have a 301 in place. Why is this?
Google will assume that, "Well, I know that this URL is a 301 or 404, but I am going to crawl it every once in a while just to make sure the webmaster did not do this by mistake." You can notice this in Search Console error or link reports when you setup 301s or 404s, they may stay in there for months and even come back once they fall out of the error list. I had an occurrence where I had some old URLs showing up in the SERPs and various Search Console reports for a site for 2 years following proper 301 setups. Why was this happening?
This is a large site and we still had some old content still linking to the old URLs. The solution was to delete the links in that old content and setup a canonical to self on all the pages to help give a definitive directive to Google. Google then finally replaced the old URLs with the new URLs in the SERPs and in the Search Console reports. The point here being that previously our site was giving signals (links) that told Google that some of the old URLs were still valid and Google was giving us the benefit of the doubt.
If you want to have the new URLs seen by Google, show them in your sitemap. Google already has all the old URLs and will check them and find the 301s and fix everything. I would also recommend the canonical to self on the new pages. Don't give any signals to Google that your old URLs are still valid by linking to them in any way, especially your sitemap. I would even go so far as to reach out to any important sites that link to old URLs to ask for an updated link to your site.
As I mentioned above, I do not think there is an "advantage" of getting the new URLs indexed quicker by putting old URLs in the sitemap that 301 to the new URLs. Just watch your Google Search Console crawl stats. Once you do a major overhaul, you will see Google really crawl your site like crazy and they will update things pretty quick. Putting the old URLs in the sitemap is a conflicting signal in that process and has the potential to slow Google down IMHO.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What to do when migrating to HTTPS?
Hi guys, I am planning on migrating an entire http domain to https. For it to be successful and making sure that Google indexes these new https URLs and not losing any SEO juice, do I simply make sure all legacy URLs get 301 redirected to those new https URLs? Finally what is a server side 301 redirect? What are all the kinds of 301 redirects if there are multiple kinds? Thanks!
Intermediate & Advanced SEO | | Ideas-Money-Art1 -
What is optimal sitemap for large website
My website is having more than 3500 posts. Please let me know what sitemap plugin I need to use for the website and what is the best practice for it?
Intermediate & Advanced SEO | | Michael.Leonard0 -
2 eCommerce stores that are identical 1 for US 1 for CA, what's the best way to SEO?
Hello everyone! I have an SEO question that I cannot solve given the parameters of the project, and I was wondering if someone could provide me with the next best alternative to my situation. Thank you in advance. The problem: Two eCommerce stores are completely identical (structure, products, descriptions, content) but they are on separate domains for currency and targeting purposes. www.website-can.com is for Canada and www.website-usa.com is for US. Due to exchange rate issues, we are unable to combine the 2 domains into 1 store and optimize. What's been done? I have optimized the Canadian store with unique meta titles and descriptions for every page and every product. However I have left the US store untouched. I would like to gain more visibility for the US Store but it is very difficult to create unique content considering the products are identical. I have evaluated using canonicals but that would ask Google to only look at either the Canadian or US store, , correct me if i'm wrong. I am looking for the next best solution given the challenges and I was wondering if someone could provide me with some ideas.
Intermediate & Advanced SEO | | Snaptech_Marketing0 -
Best practice to prevent pages from being indexed?
Generally speaking, is it better to use robots.txt or rel=noindex to prevent duplicate pages from being indexed?
Intermediate & Advanced SEO | | TheaterMania0 -
Best Way to Optimize 38 Local Directory Listing In Major Directories
Hi Folks, I am trying to figure out the best way to get our company's 38 U.S. locations in the major local directories. To start, I'd like to get us listed in the major ones: Google, Yahoo, Bing, and Yelp. I do have the resources myself here on staff to do everything manually. So, I don't necessarily need a service like Yext (but would also like any opinions on that offering if anyone can offer it). But, from what I know in the past, every time you try to claim a local listing within each platform, you have to confirm your existence there somehow - whether it be by a mailed postcard or some sort of automated call they give you. Considering that we want to manage all social and local platforms here at corporate, how can we do this? I am not physically at these locations, but I'm sure this is possible to manage everything through one account. The addresses will be local, but the phone numbers on each local profile will route to our customer service here at corporate because the local locations are mostly administrative. In other words, businesses is booked through corporate and carried out at local destinations. Thoughts/Comments?
Intermediate & Advanced SEO | | CSawatzky
I want to do what's best for SEO and also dont' want to harm anything or our link equity. Thanks,
Pedram0 -
Best way to move from mixed case url to all lowercase?
We are currently in the process of moving our site from a mixed case structure i.e -> <sitename>/franchise/childrens-child-care/party/Bricks-4-Kidz/company-information.cfm</sitename> to all lowercase i.e -> <sitename>/franchise/childrens-child-care/party/bricks-4-kidz/company-information.cfm.</sitename> In order to maintain as much link juice as possible, should we be using 301 redirects to point from the old to the new? or would it be more advantageous to wait for the next crawl and the link juice would also be somewhat maintained even though the all the upper case letters have been converted to lowercase?
Intermediate & Advanced SEO | | franchisesolutions0 -
Best tool to calculate link distribution?
What is the best tool to calculate the total link distribution throughout a site? I know opensiteexplorer.com's "top pages" breaks down the numbers for you? Are there any others?
Intermediate & Advanced SEO | | nicole.healthline0 -
Can a XML sitemap index point to other sitemaps indexes?
We have a massive site that is having some issue being fully crawled due to some of our site architecture and linking. Is it possible to have a XML sitemap index point to other sitemap indexes rather than standalone XML sitemaps? Has anyone done this successfully? Based upon the description here: http://sitemaps.org/protocol.php#index it seems like it should be possible. Thanks in advance for your help!
Intermediate & Advanced SEO | | CareerBliss0