Why might Google be crawling via old sitemap, when the new one has been submitted and verified?
-
We have recently relaunched Scoutzie.com and re-submitted our new sitemap to Google. When I look on Webmaster tools, our new sitemap has been submitted just fine, but at the same time, Google is finding a lot of 404s when crawling the site. My understanding, it is still using crawling the old links, which do not exists. How can I tell Google to refresh it's index and to stop looking at all the old links?
-
Yes it should. However, as Alan mentioned below, if you still have links pointing to the 404 pages, Google will always attempt to crawl them, and will keep you informed that you have errors.
If you do have external links to those 404 pages, you can 301 redirect them to an appropriate page using .htaccess. This way you'll keep the link value and also get rid of the Webmaster Tools error.
If you don't have any links to them, then yes, Google will eventually stop trying to crawl them.
-
It's very likely that we do. Given that I cannot track down a 1000+ links that now 404, will they eventually fall out by themselves, or do I have to tell Google that everything that's 404'ed should be dropped from crawl index? Thanks!
-
What if I simply pushed the new sitemap over the old one? In other words, scoutzie.com/sitemap is the same link, except now it contains the new map. That should be okay, right?
-
you may still have links pointing to those 404 pages on your site or externally. If not then eventually they will fall out of the index
-
Hey scoutzie,
This is actually covered pretty well in Joe Robison's blog post on fixing Webmaster Tools crawl errors: http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools
I'll quote the related info:
"One frustrating thing that Google does is it will continually crawl old sitemaps that you have since deleted to check that the sitemap and URLs are in fact dead. If you have an old sitemap that you have removed from Webmaster Tools, and you don’t want being crawled, make sure you let that sitemap 404 and that you are not redirecting the sitemap to your current sitemap."
Hope this helps, good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's more valuable: new content or optimizing old content
We are a niche legacy print publication that's been around for close to 20 years. Recently, we combined several old sites in one new responsive site. We have over 7,000 articles -many of which are evergreen and can be repurposed when needed. Most of the old pieces although published, have not been optimized for SEO. However, as we create new pieces, we optimize them for search and social and they tend to get more organic traffic. Where we're torn is on how much we should balance our limited editorial resources between cleaning up and optimizing our extensive archive to improve our organic reach, vs. pumping out new original pieces each week. I realize that without a lot of data the answers will be varied - I guess I'm looking for a best practices approach for content publishers. If it helps at all, our main conversion goal is selling subscriptions to our print and digital publications. We know that organic traffic tends to be more engaged than our social referrals. Unfortunately, due to the nature of the magazine fulfilment business, it's tough to know which channels convert better. Thanks!
Moz Pro | | RicardoSalcedo0 -
Meta Tag Descriptions not being found in Moz Crawls
Hey guys, I have been managing a few websites and have input them into Moz for crawl reports, etc. For a while I have noticed that we were getting a gratuitous amount of errors when it came to the number of missing meta tags. It was numbering in the 200's. The sites were in place before I got here and a lot of the older posts no one had even attempted to include tags, links of the page or anything. As they are all Wordpress Sites and they all already had the Yoast/Wordpress SEO plug-in installed on them, I decided I would go through each post and media file one at a time and update their meta tags via the plug in. I personally did this so I know that I added and saved each one, however the Moz crawl reports continue to show that we are missing roughly 200 meta tags. I've seen a huge drop off in 404 errors and stuff since I went through and double checked everything on the sites, however the meta tag errors persist. Is this the case that Moz is not recognizing the tags when it crawls because I used the Yoast Plugin? Or would you say that the plugin is the issue and I should find another way to add meta tags to the pages and posts on the site? My main concern is that if Moz is having issues crawling the sites, is Google also seeing the same thing? The URLS include:
Moz Pro | | MOZ.info
sundancevacationsblog.com
sundancevacationsnews.com
sundancevacationscharities.com Any help would be appreciated!0 -
Accurate Google Keyword Ranking for a domain
Hi Folks, SEO N00b here, in a new job so please go easy on me if i ask stupid questions. I'm looking for some clarification on Google Keyword rankings. I am looking for accurate Google Keywords rankings as when I log into Google Webmaster Tools I am given values like (6-10) for a ranking? When I used SEOMoz (Research Tools -> Keyword & Ranking -> Rank Tracker) it gives me an even less accurate range by simply saying >#50?Why is SEOMoz giving a completely useless metric? Why is its actual rank not shown? I know rankings are becoming less useful as a metric with Google moving toward personalized search but I still have reports to deliver to clients so the conversation of what metrics are going in the report is to be had another day. Right now I just want to get accurate rankings, from what I've found online Google Webmaster tools is recommended as the de-facto ranking tool but it doesn't provide accurate results so how can that be (yes I know it averages out the ranking but 6-10 is a range of values not a single value that represents an average) Dave
Moz Pro | | icanseeu0 -
Rankings Check and Googles Quality Guidelines
Hi, I am trialing SEOmoz at present. What i dont understand is how come the rankings check, which presumably involves sending requests to Google, doesnt contravine the Google quality guidelines and cause a website to be penalised. Sorry if being a bit dim - but if anybody could explain it would be really helpful cheers stayin
Moz Pro | | stayin0 -
Crawl Diagnostics Update
I have corrected some errors in my SEOMoz Crawl Diagnostics, however the errors are still showing. It says a crawl has happen since. Any idea's why?
Moz Pro | | petewinter0 -
Rankings in Google UK
Hi there, Can someone explain the difference between the results in Google UK - the default is 'The Web', whereas I can opt to choose 'Pages from the UK'. My question is: If I'm tracking rankings for Google United Kingdom in SEOMoz, is it tracking the results from 'Pages from the UK' or the default setting of 'The Web'? Are there any statistics on how many people choose to see 'Pages from the UK'? Thanks
Moz Pro | | columbus0 -
Google Peru rank and SEO Moz
Hello, I wanna track the google Peru rank but the tool say me that im not in the top 50 and i'm sure that I'm, how to get a real ranking?
Moz Pro | | Kuna0 -
Page Rank is different from google search
I use the ranking tools to check each page of our site. I use the keyword flora grand and exact url http://www.florahospitality.com/dubai-flora-grand-hotel.aspx . It return page rank 1 in US, UAE, and UK region but when I try to search it in google it doesn't appear.
Moz Pro | | shebinhassan0