About Google Spider
-
Hello, people!
I have some questions regarding on Google spider.
Many people are saying that "Google spiders only have US IP address."
Is this really true?
But I also saw video from Google's offical blog and it said "Google spider come from all around the world."
At this point I am really confused.
Q1) I researched and it seems like Google spiders have only US IP address. THen what does exactly mean by "Google spider come from all around the world."?
Q2) If Google spider have only US IP address, what happen to site which use IP delivery?
Is this means that Google spider always redirect to us site since they only have US IP?
Can anyone help me to understand??
One more questions! When Google analyzing for cloaking issue, do you think Google analyze when spider crawls the site or after they crawled the site?
-
I think some of the confusion may be due to Google's primarily using IP addresses assigned to their headquarters in Mountain View, California. Google has many (around 20) data centers located outside the US. I recall reading an article whereby at times they used their Mountain View IPs from centers around the world. For security reasons they do not wish the location of all their data centers to be known.
I researched this topic before and I was unable to locate any official information from Google. It would only seem reasonable they crawl from all over the world. If they didn't, then a lot of sites which use geo-based targeting for site navigation would not have most of their content indexed. While it's true a sitemap could be used to overcome the issue, many sites do not use sitemaps and they still get indexed.
-
I do not believe this is true, Google has data centers all over the world in which they crawl from.
Google does not only have spiders crawling from US data centers.
I also have the feeling crawls are based on many factors such as link diversity per region, TLD of domain per region, PR user (still a crawling factor imo) and many more factors.
Overall do not stress Google can crawl from various regions all over the globe, I would be more worried about geo server location and TLD of your domain and also local links.
Kind Regards,
James Norquay.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
Hi Everyone! My website is not being crawled regularly by Google - there are weeks when it's regular but for the past month or so it does not get crawled for seven to eight days. There are some specific pages, that I want to get ranked but they of late are not being crawled AT ALL unless I use the 'Fetch As Google' tool! That's not normal, right? I have checked and re-checked the on-page metrics for these pages (and the website as a whole, backlinking is a regular and ongoing process as well! Sitemap is in place too! Resubmitted it once too! This issue is detrimental to website traffic and rankings! Would really appreciate insights from you guys! Thanks a lot!
Technical SEO | | farhanm1 -
Massive drop off in Google crawl stats
Hi Could i get a second opinion on the following please. ON a client site we seem to have had a massive drop off in google crawling in the past few weeks, this is linked with a drop in search impressions and a slight reduction in penalty. There are no warning messages in WMT to say the site is in trouble, and it shouldn't be, however cannot get to the bottom of what is going on. In Feb the Kilobytes downloaded per day was between 2200 and about 3800, all good there. However in the past couple of weeks it has peaked at 62 and most days are not even over 3! Something odd has taken place. For the same period, the Pages crawled per day has gone from 50 - 100 down to under 3. At the same time the site speed hasn't changed - it is slow and has always been slow (have advised the client to change this but you know how it is....) Unfortunately I am unable to give the site url out so i understand that may impact on any advice people could offer. Ive attached some screen shots from WMT below. Many thanks for any assistance. stats.png
Technical SEO | | daedriccarl0 -
Google not pulling correct Meta Description
For some reason Google is not pulling the meta description for one of our key pages. Instead it just takes the first sentence from the page which makes the description horrible. Why would it be doing this if the meta tag field is populated?
Technical SEO | | inhouseninja0 -
When do you use 'Fetch as a Google'' on Google Webmaster?
Hi, I was wondering when and how often do you use 'Fetch as a Google'' on Google Webmaster and do you submit individual pages or main URL only? I've googled it but i got confused more. I appreciate if you could help. Thanks
Technical SEO | | Rubix1 -
Google Alerts News Images
I have a Google Alert setup which is pulling information from a blog. I am receiving images as part of the alert. The issue that I am having is that the images have nothing to do with the blog post. Is there a way to control what images are received in the alert. From what I have gathered, if it grabs an image it should be part of the blog post.
Technical SEO | | ricknakao0 -
Severe Drops in Google UK rankings
Hi Guys, I'm Chris - SEO Noob & appointed website man @ www.customdesignedcable.co.uk I signed up to SEO moz at the end of February and i have slowly but surely been trying to implement the on-page optimisation suggestions with my selected keywords against the appropriate pages. Since i have done this, our pages are still ranking but no where near what they were at before. Out of 70 keywords, we've gone from at least 60+ pages on the first page down to 20+ and the number in the top three has dropped from 55 to 33. Obviously i'm no seo guru far far from it but all of the changes that have been suggested i have adhered to and the result is the exact opposite of what i was hoping for and expecting. I'm after some advice as to how to use seomoz to target my keywords to google primarily to get these page rankings back. Should i just upload backup copies of the site? How would you suggest that i move forward to try and achieve at least similar rankings to what i had previously? Any help is much appreciated as most of our enquiries come from the internet and it has almost completely dried up these last couple of months. Thanks Guys! Chris. P.S. Would redesigning the site look, layout and feel with the same content negatively affect seo? Also, as a completely separate question, how come sites with content that is a lot worse than ours rank higher for the same keywords as ours? Thanks Again
Technical SEO | | Chris_CDC0 -
Does Google index XML files?
Does Google or other search engines include XML files in their index? More specifically, I am wondering how Google knows the difference between an xml filetype and an RSS feed.
Technical SEO | | nicole.healthline0 -
"Site Suspended" in Google Adwords + Lost all rankings in Google => is this related?
Can anyone share thoughts on this: Does the S recently (mid april) we revamped our website (same content, new layout, strong brand), but a few days later our google rep contacted us to tell that she got a "red flag" for one of our SEA campaigns (we broke the bridge page policy, not on purpose to be clear), they were completely correct on this matter. We even got some extra time to correct this, normal policy is only 10 days. But, we were a little slow, so all our Adwords Campaigns are suspended and we get the message "Site suspended". We are working to have this fixed, our Google rep even granted some more time to fix this. Now, almost simultaneously, same time frame, all our new pages, that were already ranking well tx to proper 301 rules, suddenly fell out of the google SERPS, nothing to be found anymore up till now. Our website is live since 1996, no issues, up till now. There seems to be a strong correlation to what happened in our SEA and what happened in our SEO can anyone share some info?
Technical SEO | | TruvoDirectories0