Sitemap as Referrer in Crawl Error Report
-
I have just downloaded the SEOMoz crawl error report, and I have a number of pages listed which all show FALSE.
The only common denominator is the referrer - the sitemap.
I can't find anything wrong, should I be worried this is appearing in the error report?
-
Thanks Tom.
The site map is pointing to the correct pages, and when visiting the pages in the search engines no problems arises.
I don't understand why these pages are listed in the crawl error report when I can't see any obvious issue.
-
Hi Christina
If the referrer is the sitemap, it means that the SEOMoz crawler has been directed to that page because of the sitemap you have submitted.
If you're getting 404 errors or access errors for certain pages and they are only able to be accessed via the sitemap, then it's a good idea to remove those URLs from the sitemap altogether. It doesn't make sense to have URLs listed in your sitemap if those URLs don't exist or have restricted access.
A cleaner sitemap will ultimately help in the long run. Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If I'm using a compressed sitemap (sitemap.xml.gz) that's the URL that gets submitted to webmaster tools, correct?
I just want to verify that if a compressed sitemap file is being used, then the URL that gets submitted to Google, Bing, etc and the URL that's used in the robots.txt indicates that it's a compressed file. For example, "sitemap.xml.gz" -- thanks!
Technical SEO | | jgresalfi0 -
Duplicate content and 404 errors
I apologize in advance, but I am an SEO novice and my understanding of code is very limited. Moz has issued a lot (several hundred) of duplicate content and 404 error flags on the ecommerce site my company takes care of. For the duplicate content, some of the pages it says are duplicates don't even seem similar to me. additionally, a lot of them are static pages we embed images of size charts that we use as popups on item pages. it says these issues are high priority but how bad is this? Is this just an issue because if a page has similar content the engine spider won't know which one to index? also, what is the best way to handle these urls bringing back 404 errors? I should probably have a developer look at these issues but I wanted to ask the extremely knowledgeable Moz community before I do 🙂
Technical SEO | | AliMac260 -
Crawl issues
Hello there, I have found that when crawling my site I have errors regarding the meta description and it says it is missing from few pages. I checked these pages but there is a meta description. I also ran the same report with other tools and it comes up the same issues. What should I do?
Technical SEO | | PremioOscar0 -
How can I fix this home page crawl error ?
My website shows this crawl error => 612 : Home page banned by error response for robots.txt. I also did not get any page data in my account for this website ... I did get keyword rankings and traffic data, I am guessing from the analytics account. url = www.mississaugakids.com Not sure really what to do with this ! Any help is greatly appreciated.
Technical SEO | | jlane90 -
Client error 404
I have got a lot (100+) of 404´s. I got more the last time, so I rearranged the whole site. I even changed it from .php to .html. I have went to the web hotel to delete all of the .php files from the main server. Still, I got after yesterdays crawl 404´s on my (deleted) .php sites. There is also other links that has an error, but aren't there. Maybe those pages were there before the sites remodelling, but I don't think so because .html sites is also affected. How can this be happening?
Technical SEO | | mato0 -
Client error 404
I have an 404 error but what does that mean? I go to the site and click on the link to exampleX.com there is no problem. What can it be? The error message http://www.example.com/www.example.com/exampleX.html
Technical SEO | | mato0 -
Increase in authorization permission errors error after site switch
We launched our new site 2 days ago , since site was down for 12 hours for maintenance, we saw google webmaster tool shows this error . Since then google hasnt crawled, its been 36 hours. Do we need to do anyting? We have close to a million page google crawled before and I am wondering if this will effect anything.
Technical SEO | | tpt.com0 -
Wordpress SEO Errors - Any advice?
Hi all! My site is on the WP platform and I'm having a crawl error. Wondering if you guys could possibly help me figure out what's going on? I have a good number of 404 errors where the links seems to be appended and I can't figure out why. I've scoured my individual posts and cannot seem to find the broken link? The crawl error looks a bit like this: http://preciousthingsphotography.com/2007/12/10/chicago-family-photographer-welcome/http:%2F%2Fpreciousthingsphotography.com%2F2007%2F12%2F10%2Fchicago-family-photographer-welcome%2F You can see that my original link is somehow being doubled with the slashes being replaced? This is happening on all of my posts. Any ideas as to what could be going on? Thanks so much!
Technical SEO | | ptpgen0