Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics View Filters
Using the same GA property, I would like to set up three filtered views: 1. Tracking across one subdomain and one primary domain (example: shop.example.com & example.com) 2. Track only primary domain (example.com) 3. Track only subdomain (shop.example.com) Can this be achieved by using view filters? If so, how do they need to be set? Also, according to this article: https://moz.com/blog/cross-domain-subdomain-tracking-in-google-analytics, with cross domain tracking, I need to ignore self-referrals, which can only be done at the property level. If set up to ignore example.com referrals, will this cause problems with filter 2 and 3?
Reporting & Analytics | | Evan340 -
Google Tag Manager Tracking
We have Google Tag Manager set up to track our website https://kontenta.co.uk/ Although we have a blog - https://kontenta.co.uk/designroom/ I'm just wondering if it's better to track the sites separately? If so, how do I add the blog to tag manager? Does it need setting up as a filter to match the regular expression https://kontenta.co.uk/designroom/* We just need to figure out the best way to measure both the website and blog traffic and if users are coming from the blog to the site, and what interactions they are making. Thanks
Reporting & Analytics | | Jseddon920 -
Referral Traffic from Google
Hello, I have a question about my company's new website. I've worked in SEO and studied Google Analytics results for a few years now but have never really come across something like this. I started in this position in January of this year and when I started breaking down the traffic sources in Google Analytics, I noticed most of the traffic was coming from Google.com as a referral source. I had never seen Google.com as a referral source before so I looked into options for what it could be. It was not a paid ad and our organic traffic was coming through in Analytics, Before I could get any further, our new website was launched (we switched CRM's to WordPress) and the referral traffic from google went from 2,966 in January of 2015 to 22 in February 2015. for more comparison, in February of 2014, the referral traffic from Google was 2,496. I expected a drop when we switched CRM's but we correctly re-directed all pages and created a new sitemap and our organic traffic is up since the switch (not enough to cover drop in referral). I thought at first this had to do with our Google sellers account being de-activated when we made the switch, but I quickly fixed this over a month ago and no change. I'm wondering if anyone has ever seen Google.com come through as a referral source in Google Analytics and if they we're able to figure out what it actually was. This would be a great help! Thank you, Alex
Reporting & Analytics | | RASEO1 -
2 questions on avoiding issues with Google and while being right in it.
Hi SEOmoz community In fact I have two questions I would like to ask (with future SEO in mind). Do you consider a WordPress Multisite or various Single installs 'safer' for SEO? Theoretically, having various sites packed into one Multisite network seems like an ideal solution. However, is there a chance that once a site in the network encounters a little 'negative turbulence', that your other sites in the network might get impacted too due to the cross-referencing, linked account i.e. Webmaster Tools etc.? It would seem outrageous, but then again I wouldn't rule it out. Do I even have to go as far as setting up new Gmail, Google Analytics and Webmaster Tools accounts, so they (the sites) are technically not linked? You can see, I don't trust search engines one bit... Is there still a point posting articles once Google is having a hissy fit with your site? Basically I am currently going through a 'rankings and traffic drops storm'. It's not as bad as being de-indexed, but it's still having enough of an impact. In addition, Google does not seem to treat my new articles (unique content) with the same attention anymore i.e. does not seem to index them 'fully' or not at all (i.e. posting the headline in Google should return the article, but it doesn't). Is there even a point spending time now and posting new material or may it pick it up again once I am through this low phase? Does Google still index what it considers worth or is it a waste of time right now to keep posting, posting and posting more? Thanks for your help. I really appreciate it.
Reporting & Analytics | | Hermski0 -
Google Analytics Campaigns
I need the help of a smart Mozzer. In Google Analytics: Traffic Sources>Sources>Campaigns all the results shown are from RSS. Can anyone help me with why RSS results would be displayed in Campaigns?
Reporting & Analytics | | waynekolenchuk0 -
Google Analytics: Do url redirects show up
Good Morning from 6 degrees c mostly cloudy wetherby UK 🙂 A redirect has been set up with the effect that when you key in www.hyload.co.uk it forwards to www.ikogroup.co.uk My question is please...within Google analytics will referall traffic from www.hyload.co.uk show up in referral traffic or because its a redirect will it be counted as direct traffic. Thanks in advance, David
Reporting & Analytics | | Nightwing0 -
Cookie tracking in Google Analytics
Hi How do I remove the "/?__utma=...." at the end of my URLs?We have a site http://www.jetonline.co.za/, if you click on one of the menu navigation links i.e. "fashion". A long "/?__utma=...." url appears. I understand this is for tracking as we have separate domains for each page but is there a way to remove this dynamic url and keep it hidden from users?Thanks in advance
Reporting & Analytics | | NeilPursey0 -
Google Analytics | REAL TIME
So I noticed today that there is now Real Time Data: http://analytics.blogspot.com/2011/09/whats-happening-on-your-site-right-now.html and I cannot figure out how to access this.
Reporting & Analytics | | joseph.chambers1