Why might Google be crawling via old sitemap, when the new one has been submitted and verified?
-
We have recently relaunched Scoutzie.com and re-submitted our new sitemap to Google. When I look on Webmaster tools, our new sitemap has been submitted just fine, but at the same time, Google is finding a lot of 404s when crawling the site. My understanding, it is still using crawling the old links, which do not exists. How can I tell Google to refresh it's index and to stop looking at all the old links?
-
Yes it should. However, as Alan mentioned below, if you still have links pointing to the 404 pages, Google will always attempt to crawl them, and will keep you informed that you have errors.
If you do have external links to those 404 pages, you can 301 redirect them to an appropriate page using .htaccess. This way you'll keep the link value and also get rid of the Webmaster Tools error.
If you don't have any links to them, then yes, Google will eventually stop trying to crawl them.
-
It's very likely that we do. Given that I cannot track down a 1000+ links that now 404, will they eventually fall out by themselves, or do I have to tell Google that everything that's 404'ed should be dropped from crawl index? Thanks!
-
What if I simply pushed the new sitemap over the old one? In other words, scoutzie.com/sitemap is the same link, except now it contains the new map. That should be okay, right?
-
you may still have links pointing to those 404 pages on your site or externally. If not then eventually they will fall out of the index
-
Hey scoutzie,
This is actually covered pretty well in Joe Robison's blog post on fixing Webmaster Tools crawl errors: http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools
I'll quote the related info:
"One frustrating thing that Google does is it will continually crawl old sitemaps that you have since deleted to check that the sitemap and URLs are in fact dead. If you have an old sitemap that you have removed from Webmaster Tools, and you don’t want being crawled, make sure you let that sitemap 404 and that you are not redirecting the sitemap to your current sitemap."
Hope this helps, good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unable to view crawl test
After doing a crawl test i get a download report. It then downloads in csv form and when I go to view it there is a curruption error or just a load of gibberish signs Can I not see the report onsite?
Moz Pro | | hantaah0 -
Tracking Rankings In Google
Is there a way to see historical rankings of your site in Google. I have a site that has been running for a few years now. I want to see how are Google Ranking has performed throughout the years. We recently redesigned and suffered in rankings but I only know(from Google Analytics) What our traffic used to be, not what our rankings were .
Moz Pro | | pinksgreens0 -
Unable to crawl pages
Hi, I am trying to set up a campaign for our website - www.salvationarmy.org.au however, I can't seem to get a scan of more than three pages. I have tried the following: www.salvationarmy.org.au (only 2 pages) www.salvationarmy.org.au/home (only 1 page) salvationarmy.org.au (only 3 pages) There is a geo IP redirect on www.salvationarmy.org.au but the second domain listed above should resolve the full site. I'm a newbie to SEOmoz so any help would be appreciated! Thanks, Mel
Moz Pro | | KingPings0 -
Campaigns - crawled
The new Pages Crawled: 2. I have many 404 and other errors, I wanted to start working on it tomorrow but the new crawl only crawled to pages and doesn't show any errors. Whats the problem and what can I do? Yoseph
Moz Pro | | Joseph-Green-SEO0 -
Integration of Google+, LinkedIn, YouTube?
I ran across a blog on Moz from last November that was introducing the social monitoring feature. It mentioned adding new social platforms that would eventually be included in SEOMoz. Any word on when/if that is happening? Excited to have SEO and Social under the same dashboard! Jake
Moz Pro | | AESEO2 -
Hey from a New guy to SEO
Hi everybody!!! Hope everybody is dandy. Have spent a long time looking into SEOmoz and decided to just take the plunge and see what happens. Im quite new to the world of SEO but I really want to get good at it. Hoping Moz and this community will aid with this! Currently in the process of reading the SEO beginners guide and the SEOmoz user guide! The two sites that I am currently working on are: 1. www.advanced-incar.co.uk 2. www.advanced-incarcamerasystems.com The first site seems to have crawled well and have entered my competitors and am getting some info back (looks like I have a lot of work to do judging by the errors D'oh) In the world of Google what is the best way to ensure you have authority on certain terms. For instance if I search advanced-incar we are top and we have all our sitelinks underneath etc etc, but when I search advanced incar we are second and as such we don't display sitelinks. Any ideas? The second site was built using 1&1 and gives us a good little site for what we need. I have found out through research the most popular keywords are in car camera & in car cameras. Currently Im using Adwords to get us sponsored hits which is getting us some nice traffic but costing a bit. Those are the two main keywords i want this site to really hit home with. Any advice? Also on this site, when I go to SEOmoz and select Link Analysis I get no info. It just shows 0 or 0.0 wheras my competitor sites all have values against them. Is something wrong with my site or what im doing? (Have added an example screenshot) I appreciate its a long ended question but would really appreciate some help. Hoping that the guides will help me understand all this linkjuice stuff etc as well! Thanks guys, really appreciative of any info you can provide. Tim CompetitiveDomainAnalysis-AdvancedInCarCameraSystems-SEOmozPRO.png
Moz Pro | | timborrill1 -
Crawler has stopped crawling my website
Hi all, I recently changed my site www.ipbskinning.com to ipbskinning.com. For some reason my weekly scan states that no pages have been crawled. I tried making a new campaign with the sub-domain ipbskinning.com but still the starter crawl returns nothing? Am I doing something wrong? :S Thanks all!
Moz Pro | | pezza34340 -
Unsubscribe to weekly crawl notifications never works
Hello! All of my campaigns have the box 'Weekly crawl completed for campaign ...' unticked under Campaign Settings, yet for all of them I still receive an email regularly with the subject 'New crawl completed for ...'. How do I stop this? Is there a bug here? Adam Bishop
Moz Pro | | arbishop0