Mobile Googlebot vs Desktop Googlebot - GWT reports - Crawl errors
-
Hi Everyone,
I have a very specific SEO question. I am doing a site audit and one of the crawl reports is showing tons of 404's for the "smartphone" bot and with very recent crawl dates. If our website is responsive, and we do not have a mobile version of the website I do not understand why the desktop report version has tons of 404's and yet the smartphone does not. I think I am not understanding something conceptually.
I think it has something to do with this little message in the Mobile crawl report.
"Errors that occurred only when your site was crawled by Googlebot (errors didn't appear for desktop)."
If I understand correctly, the "smartphone" report will only show URL's that are not on the desktop report. Is this correct?
-
Hey Carla,
I'm not entirely sure what you're saying with:
"one of the crawl reports is showing tons of 404's for the "smartphone" bot and with very recent crawl dates. If our website is responsive, and we do not have a mobile version of the website I do not understand why the desktop report version has tons of 404's and yet the smartphone does not. I think I am not understanding something conceptually."
You say that the smartphone bot is seeing tons of 404s and the desktop report is showing tons of 404s, but the smartphone does not. If you can clarify that, I can probably better answer your question.
However, the answer is likely that Google may decide not to crawl URLs that it has already identified as 404s in one context. That is to say if they identify URLs on the mobile device as 404s they will know not to crawl them if they encounter them on desktop and vice versa.
-Mike
-
Ok I forgot to add something to this as well. Why would the URLS show up on the smartphone report if they are not on the desktop report? Afterall a 404 from either device is still a 404?
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
GSC is reporting a lot of chopped URLs
Recently, in the last two weeks, I started seeing a lot of odd 404 errors in GSC for my site. Upon investigation, the URLs are for fairly new articles, and the URLs are chopped in various places. From missing a character at the end to missing about 10 characters at the end of the URL. (an old similar issue is that GSC reports duplicate contents on weird subdomains that we've never used like 'smtp' 'ww1' or even random ones like 'bobo'.) GSC doesn't report any 'linked from' for those odd URLs and I know for sure these links aren't on the site itself. They're definitely not errors in the CMS. The site is a long established site (started 1997-1998) and we've been subject to a lot of negative SEO. I recently had to disavow about 1000 .ru domain linking to us, with some domains containing over a million link each. Could these chopped links be a new tactic of negative SEO? How do I find these seemingly intentionally broken links to us?
Intermediate & Advanced SEO | | Lazeez2 -
Pagination vs. Scroll for Ecommerce
Hi I wanted to see what opinions were on having a product listings on paginated pages vs. loading as the user scrolls? We use pagination but I have heard scroll may be better for SEO? Thanks!
Intermediate & Advanced SEO | | BeckyKey0 -
Will a disclaimer affect Crawling?
Hello everyone! My German users will have to get a disclaimer according to German laws, now my question is the following: Will a disclaimer affect crawling? What's the best practice to have regarding this? Should I have special care in this? What's the best disclaimer technique? A Plain HTML page? Something overlapping the site? Thank you all!
Intermediate & Advanced SEO | | NelsonF0 -
GWT Message - CMS Update Available
Howdy Moz, Just received a message in Google Webmaster Tools about a CMS update: "Joomla Update Available As of the last crawl of your website, you appear to be running Joomla 1.5. One or more of the URLs found were: http://www.website/custom-url/article5034 Google recommends that you update to the latest release. Older or unpatched software may be vulnerable to hacking or malware that can hurt your users. To download the latest release, visit the Joomla download page. If you have already updated to the latest version of Joomla, please disregard this message. If you have any additional questions about why you are receiving this message, Google has provided more background information in a blog post about this subject." Read through the associated blog post. According to the post a generator meta tag is created in Joomla that notes the CMS version. Here's the oddity: The site was on Joomla 1.5 over 2 years ago. 1 Year ago it was updated to Joomla 2.5. About a week ago it was converted completely to Wordpress. According to GWT the last date the Google bot accessed the site was the day before (5/1/14) the email. I went through the code, css/html, and the database and found no reference of Joomla 1.5. Has anyone seen this message? If so, how did you rectify it? Were there any adverse effects on rankings?
Intermediate & Advanced SEO | | AaronHenry0 -
Desktop & Mobile Sitemaps Covering The Same Ground - Any Benefit To Having Both?
If my URL structure is the same for the desktop and mobile experience, is there any benefit to creating a mobile sitemap, considering that the sitemap for our desktop site covers the same URLs?
Intermediate & Advanced SEO | | edmundsseo0 -
GWT Crawl Error Report Not Updating?
GWT's crawl error report hasn't updated for me since April 25. Crawl stats are updating normally, as are robots.txt and sitemap accesses. Is anyone else experiencing this?
Intermediate & Advanced SEO | | tonyperez0 -
Sub Domains vs. Persistent URLs
I've always been under the assumption that when building a micro-site it was better to use a true path (e.g. yourcompany.com/microsite) URL as opposed to a sub domain (microsite.yourcompany.com) from an SEO perspective. Can you still generate significant SEO gains from a sub domain if you were forced to use it providing the primary (e.g. yourcompany.com) had a lot of link clout/authority? Meaning, if I had to go the sub domain route would it be the end of the world?
Intermediate & Advanced SEO | | VERBInteractive0 -
How to find what Googlebot actually sees on a page?
1. When I disable java-script in Firefox and load our home page, it is missing entire middle section. 2. Also, the global nav dropdown menu does not display at all. (with java-script disabled) I believe this is not good. 3. But when type in <website name="">in Google search and click on the cached version of home page > and then click on text only version, It displays the Global nav links fine.</website> 4. When I switch the user agent to Googlebot(using Firefox plugin "User Agent Swticher)), the home page and global nav displays fine. Should I be worried about#1 and #2 then? How to find what Googlebot actually sees on a page? (I have tried "Fetch as Googlebot" from GWT. It displays source code.) Thanks for the help! Supriya.
Intermediate & Advanced SEO | | Amjath0
