Bogus Crawl Errors in Webmaster Tools?
-
I am suddenly seeing a ton of crawl errors in webmaster tools.
Almost all of them are URL links coming from scraper sites.that I do not own.
Do you see these in your Webmaster Tools account?
Do you mark them as "fixed" if they are on a scraper site? There are waaaay too many of these to make redirects.
Thanks!
-
Thanks, Marcus,
My numbers are rising rapidly right now... but hopefully the trend will reverse.
I'll let you know if I learn anything.
-
Hey, I know, it's kind of bonkers but I certainly think that assuming Google does not know what they are doing is a good place to start.
For us they just cleared up in time, obviously, this is webmaster tools so it was a good old bit of time (months rather than weeks) but it did sort itself out.
Take care!
Marcus -
Hello Marcus,
Thank you for sharing your experience and finding those posts. I appreciate it.
I think I am going to ignore these and assume that Google doesn't know what they are doing.
It surprises me that the URL errors on spammer sites are being presented to me as something that should be fixed.
Thanks again!
-
Hey EGOL
I have seen this in the past on my own site and on a few client sites in the past (which is not to say I have an answer here).
We were seeing completely random looking URLs that at first made me think the site had been somehow hacked or compromised but further investigation revealed that was not the case. We were just getting the strangest of links to pages that did not exist like xhyx.php?id=jamesbrown (that kind of thing).
We did nothing here and over time it seems to have resolved itself and these pages are not listed any longer. I tend to think of the webmaster tools data as diagnostics and it is telling me these pages don't exist so I can check for problems. Well, there is no problem, they don't exist and I am happy about that. Still, whether to mark them as fixed or not, I am unsure and would err towards not doing anything with them as they are not 'errors' as far as I am concerned. Likewise, I don't want to redirect them in most cases as I don't like the linking sites and have better things to do with my working day (I am not getting that time back - it's the digital equivalent or ironing clothes or some such laborious grind).
I had a look around again and whilst I can't find any specific answers regarding whether to mark them as fixed the following posts are of interest:
- http://productforums.google.com/forum/#!topic/webmasters/3GTOLCE-8pk
- https://productforums.google.com/forum/?hl=en#!category-topic/webmasters/webmaster-tools/rKI-38ohfbc
Particularly this quote from John Mueller at Google (webmaster tools guy I believe):
"In general, if a URL is really a 404, that's fine for us, and not something that would cause your site any problems in the long run. At any rate, you don't need to "fix" this problem (eg with a 301 redirect), if you're sure that the URL should really not exist. Having 404s listed in Webmaster Tools will generally not affect your site's crawling, indexing, or ranking; it's normal for websites to return 404 for URLs that don't exist."
So, my take is not to bother but would be interesting to ask the question in webmaster tools section of the Google product forums: https://productforums.google.com/forum/?hl=en#!categories/webmasters/webmaster-tools
Not an answer as such but hope that helps.
Marcus
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Missing xml tag error
Our xml sitemap is divided up in to many smaller xml sitemaps so we have fewer products per sitemap, in order to easily identify errors. A couple of weeks ago, we changed our xml sitemap by reordering some of the products. However, this has left some old xml sitemaps without any data, and they are no longer appearing in our xml sitemap. But, Google is still identifying these sitemaps since they once existed, and they are giving errors since they can't locate them. Should we 404 those xml sitemaps, or is there a better way to handle this?
Technical SEO | | ang0 -
Google index graph duration in Google Webmaster Tools
Hello guys, I wonder, my sites are currently being indexed every 7 days, exactly. At Index Status page in GWT. However, this new site gets updated almost everyday, how can I ask google to index faster and more frequently/almost daily? Is it about SItemap.xml frequency ? I changed it today to Daily. Thanks!
Technical SEO | | mdmoz0 -
Can increase in crawl errors in GWT) be caused by input fields and jquery?
Dear Mozzerz We took over www.urgiganten.dk not long ago and last week we opened up for indexation, after having taken the old website down for a couple of months. One week after opening for indexation we saw a huge increase in crawl errors.Google is discovering some weird links to e.g http://www.urgiganten.dk/30-garmin-urremme/ which returns a 404. In GWT we are told that we are linking to this url from http://www.urgiganten.dk/garmin-urremme. But nowhere on http://www.urgiganten.dk/garmin-urremme will you find this link. However you will find the following script in the source code, which is the only code part that contains "/30-garmin-urremme/":Can it be true that google take the id and adds it to our tld to form a url? We have seen quite a lot of these errors not only on Urgiganten.dk but also some of our other websites!
Technical SEO | | urgiganten0 -
My backlinks do not register on your software but are registered on my google webmaster tools
Why are my back links not being recognized by any software other than google webmaster?
Technical SEO | | SteveK640 -
Google Webmaster Tool - Crawl Stats Query ?
Dear All, I have been looking at GWT Crawl Stats and wondering how should I be interrupting the crawl stats chart. AllI I see is 3 charts telling me a high , low and average for the below but I am wondering is there anything I really need to be looking for ?. Pages crawled per day Kilobytes downloaded per day Time spent downloading a page (in milliseconds) thanks Sarah
Technical SEO | | SarahCollins0 -
Should we redirect 404 errrors seen in webmaster tools with ... (dot.dot,dot) ?
Lately I have seen lots of 404 errors showing in webmaster tools that are not really links. Many of them from shammy pages. (I did not put them there) One of the most common types is ones that show the link ending in ... ( dot, dot, dot) The appearance of the link is being sent from pages like this http://www.the-pick.com/00_fahrenheit,2.html For example a link like this would show up in webmaster tools as a 404 error. http://www.ehow.com/how_2352088_easily-... Are these worth redirecting? So far I have redirected some of them and found that is was not helpful and possibly harmful. Anyone else had the same experience? Also getting lots of partial urls showing up from pages that reference my site but the url is cut off and the link is not active. Does Google really count these as links? Is redirecting a link from a spammy page acknowledging acceptance and could it count against you?
Technical SEO | | KentH0 -
Error Reporting
http://pro.seomoz.org/campaigns/33868/issues/18 Rel Canonical Found about 16 hours ago <dl> <dt>Tag value</dt> <dd>http://www.geeks.com/</dd> <dt>Description</dt> <dd>Using rel=canonical suggests to search engines which URL should be seen as canonical.</dd> <dd>We do have rel canonical on some of the pages this report is recommending that we "fix" this issue.</dd> <dd> Rel Canonical Found about 16 hours ago <dl> <dt>Tag value</dt> <dd>http://www.geeks.com/products.asp?cat=MBB</dd> <dt>Description</dt> <dd>Using rel=canonical suggests to search engines which URL should be seen as canonical.</dd> </dl> <a class="more expanded">Minimize</a> </dd> </dl>
Technical SEO | | JustinGeeks0 -
How to increase the crawl rate?
hello, Our site was hosted in North America and Google was crawling it reasonably fast. Since our traffic is mostly from India we moved it to India, now the crawling is terribly slow from Google. Is there anyway to fix the crawl rate(we have increased the crawl rate in GWT)
Technical SEO | | greyniumseo0