Can you confirm legitimate Google Bot traffic?
-
We use Cloudflare as a firewall. I noticed a significant number of blocks of bot traffic. One of the things they do is try to block bad bot traffic. But it seems they are mistakenly blocking Google Bot traffic. If you use Cloudflare, you may want to look into this as well. Also, can you confirm if the following IPs are for legitimate Google Bots?
66.249.79.88
66.249.79.65
66.249.79.8066.249.79.76
Thanks,
-
Thank you very much for all of your feedback folks. I really appreciate it. Cloudflare has acknowledged that they are aware of the issue and working to correct it. It also affected Bing Bots. I ended up white listing the IP ranges, and our Google crawl has increased as a result.
-
Dammit, my mistake.
Was proabaly still asleep. I've checked with my log files and updated itThanks Alex!
-
10.0.0.0/8 is a private subnet and therefore unroutable, so I don't think you would have seen traffic from that subnet from Googlebot!
-
What Alex has pointed out is what you're needing.
Also, I'd like to remember you that Googlebot crawls from several IP ranges. Not only the 66.xxx.xxx.xx.
EDIT:
Alex pointed me out that I've made a mistake. So I've checked my data in server logs.
In some of the sites I'm working, most of the traffic comes from the range: [66.249.6x.xxx-66.249.7x.xxx]Always, when in doubt, with a reverse DNSlookup you'll find out whether it's an official GoogleBot or not.
Hope it helps.
Best luck.
Gaston -
Following the advice here: https://support.google.com/webmasters/answer/80553?hl=en I can confirm that they are indeed Googlebot.
If you use Fetch as Google in Search Console, are you seeing errors?
Cloudflare has an article here on the subject:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can we re rank our Penalyzed website in Google?
Hello This is Maqbul, from India. I have a jobs portal blog [ bharatrecruit.com]. It was getting around 50K to 100K Views a Day and made me $100 a day. But after a few months, my competitor made negative SEO with 12,000 Spammy backlinks. Suddenly my site was hit by Google and now it is getting 200 to 300 Pageviews a day. So the question is I did not disavow bad links for a long time like 3 to 4 months. Now I disavow all the bad links but the website is not ranking. Can we re-rank this site or create another website. Please reply must. None of the bloggers can answer this. Thanks, Regards Maqbul
Technical SEO | | vinaso960 -
Google Bot is seeing the desktop version in cache for Mobile website too.
Hi, I have an e-commerce website that is dynamic(not responsive) in nature. when i check the cached version of mobile website it shows the desktop version in cache. Will it create any problem . How can i tell google bot to see my mobile cached version instead of desktop one.
Technical SEO | | dhananjay.kumar10 -
Why can no tool crawl this site?
I am trying to perform a crawl analysis on a client's website at https://www.bravosolution.com I have tried to crawl it with IIS for SEO, Sreaming Frog and Xenu and not one of them makes it further than the home page of the site. There is nothing I can see in the robots.txt that is blocking these agents. As far as I can see, Google is able to crawl the site although they have noticed a significant drop in organic traffic. Any advise would be very welcome Regards Danny
Technical SEO | | richdan0 -
Can I have an H1 tag below an H2?
Quick question for you all - Is there an issue with me having an H1 tag physically below an H2 tag on a web page??
Technical SEO | | Pete40 -
What can i do to let google know we are a lifestyle magazine
Hi my site http://www.in2town.co.uk/ is a lifestyle magazine and before we changed templates and changed the joomla language from 1.0 to 1.5 our lifestyle magazine was getting around 10,000 visitors a day and we were number one in google for the search term lifestyle magazine. In the past few months we have been number five for the search term lifestyle magazine and now since yesterday we have dropped to ten. I have a serious problem that being a online lifestyle magazine i do not have the luxury to have lots of text telling people it is a lifestyle magazine and google. What i mean is, if this was a website about benidorm then you would have an introduction about benidorm so google would understand what the site is about. What i would like help on, is to get back to the top and know where i can put a nice introduction which does not look out of place and what i can do to get back to the top under the search term lifestyle magazine. any help would be great
Technical SEO | | ClaireH-1848860 -
How can I improve my google places ranking?
I am currently registered with google places for 'video conferencing in Melbourne australia' however I don't show up on page 1 of the places sesrch results for this search term. How can I improve it. I do note that my office address is in a residential area and not Melbourne CBD. Thanks Dan
Technical SEO | | dantmurphy0 -
Google Sandboxing
I have a new site with a new domain that ranked well the 1st week or so after it was indexed then it totally dropped off the SERP. My question is, does Google Sandboxing affect new sites on new domains that don't have any incoming links? The site dropped off before I began link building - from what I've read unnatural link build is often the cause. Can you still be sandboxed without any link building? If this is the case, are there things I can do to get out of the sandbox? Thanks folks, Jason
Technical SEO | | OptioPublishing0 -
Google Pake Rank of Zero?
Hi! We've been running our website www.opzeggen.nl for a while now, but we still have a google page rank of 0. Searching for a keyword like 'opzeggen' does give us good ranking however. Should I be worried about this? And if so, is there a way to fix this? Best Regards, Pieter
Technical SEO | | greenonline0