Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Query on google analytic benchmarking report
Hi All, First I select My Industry Vertical - ABCD then I select Region - US ( all region) then size by daily session - 10000-99999 finally reports appears "Blank" but when I decrease daily session from 10000-99999 to 1000-4999 or less then report is perfect. So what does it mean? My Avg daily session is 70k to 80k. So how to analysis benchmarking in this case? Thanks!
Reporting & Analytics | | pragnesh96390 -
Google Analytics - Dashboard Question
I'm looking to set up a dashboard widget in Google Analytics that does the following but can't : Shows traffic sources in a table as the dimension, and wanting to show goal completions to a specific product page and then only show what the average time spent (by source) on just those product pages. It looks like it's showing the whole session duration for the entire source, but I want to create a secondary filter that is only showing the time spent on those specific pages. Can anyone help - or is this possible? Thanks all!
Reporting & Analytics | | ReunionMarketing0 -
Google Cookies
Without Google Analytics, how would we be able to identify visitors who are from Google Organic? What's the cookie?
Reporting & Analytics | | AMHC0 -
Sudden dramatic drop in "some but not all" keyphrase rankings for client, 3 weeks ago
I noticed recently that 4 of my keyphrases which ranked on page one, and were climbing 3 weeks ago, have suddenly dropped to > 50. What could have happened? I did find an issue with my sitemap (yoast generated), but why would that suddenly change? And even more importantly, how do I fix it?
Reporting & Analytics | | chill9860 -
Testing IP Exclusion Filters in Google Analytics
(I hope I haven't missed something simple 😉 I have add IP exclusion filters in Google Analytics. What I haven't been able to determine is how to test to see if it is working. I'd imagine there would be a dip in traffic but if a site is low volume it might not even register. Thanks!
Reporting & Analytics | | AUDigitalMarketing0 -
Yikes..what is this? "We have detected that the root domain xxxxxx.com does not repsond to web requests"!!!
Can anyone tell me what this means? "We have detected that the root domain xxxxxx.com does not repsond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information"
Reporting & Analytics | | MannixSEO0 -
"not selected" is gone from Google Webmaster Tools Index Status Advanced
Just noticed today that the "not selected" has been removed from the Index status, Advanced section of Google Webmaster Tools. Anyone know why. I've used this metric to determine how or why Google was not selecting pages, particularly to gauge canonical's, 301's and duplicate content. It will be missed if gone for good.
Reporting & Analytics | | tdawson090 -
Google Analytics data - Canonical problems?
Hi everyone, We're trying to determine why Google Analytics is showing multiple versions of the same page as having "landing page traffic". For instance, these 2 pages are both shown as landing pages in GA: www.oursite.com/product_page
Reporting & Analytics | | darkgreenguy
www.oursite.com/product_page/ This occurs many times in Google analytics. Also, there are instances such as these: www.oursite.com/index.php/custom_product_url www.oursite.com/custom_product_url I can't find anything in Google Webmaster tools that would indicate a problem. However, this GA data is making me think there are duplicate content issues on the site... Thanks in advance for any help...0