What IP Address does Googlebot use to read your site when coming from an external backlink?
-
Hi All,
I'm trying to find more information on what IP address Googlebot would use when arriving to crawl your site from an external backlink.
I'm under the impression Googlebot uses international signals to determine the best IP address to use when crawling (US / non-US) and then carries on with that IP when it arrives to your website?
E.g. - Googlebot finds www.example.co.uk. Due to the ccTLD, it decides to crawl the site with a UK IP address rather than a US one. As it crawls this UK site, it finds a subdirectory backlink to your website and continues to crawl your website with the aforementioned UK IP address.
Is this a correct assumption, or does Googlebot look at altering the IP address as it enters a backlink / new domain?
Also, are ccTLDs the main signals to determine the possibility of Google switching to an international IP address to crawl, rather than the standard US one? Am I right in saying that hreflang tags don't apply here at all, as their purpose is to be used in SERPS and helping Google to determine which page to serve to users based on their IP etc.
If anyone has any insight this would be great.
-
There's a few things you need to marry up if you want to do this. You need the referring page or domain / hostname (to validate that the session came from a backlink you know about). Once you filter the data down like that, you just need to filter by user-agent ("googlebot" - or any user-agent string which contains "googlebot"). Then you just want to look at the IP address field in the tabular data and you have your answers!
Here's the problem, most IP-level data is contained within basic server-side analysis packages (like AWStats which is installed on most sites, within the cPanel) or alternatively you can go to the log files for much of the same data. Most referrer-level data (stuff that deals with attribution) is contained within Analytics suites like Adobe Omniture or Google Analytics.
In GA, you can't usually get to 'individual' IP-level data. There used to be a URL hack to force it to render, but it was killed off (and many people who used it were banned by Google). The reason for that is, Google don't want too much PID (Personally Identifiable Data) harvested by their tool. It creates too many legal issues for Google (and also, whomever is leveraging that data for potentially nefarious marketing purposes)
Since you won't get enough IP-level data from GA, you're going to have to go to log files and log analysis tools instead. Hopefully they will contain at least some referral level data... The issue is, getting all the pieces you want to align in a legally compliant way
Obviously you have your reasons for looking. I'd check if you can find anything on your CPanel in AWStats (if that's installed) or get the log files and analyse them with something like Screaming Frog Log File Analyser
I can't promise this will return the data you want, but it's probably your only hope
-
Hi,
First of all "Google crawls from many IPs and they have confirmed that they do periodically add new ones. And there are also various Googlebot useragents, not just the regular one. This is why Google doesn't publish a list of all the IPs, because there are so many of them and they can change" .
You can see full conversation here @ https://productforums.google.com/forum/#!msg/webmasters/4fKthSy7oFQ/GgslLXJnDQAJ
Second Today Google says "IP Addresses Don't Matter For Backlinks & Search Rankings"
https://www.seroundtable.com/google-ip-addresses-backlinks-rankings-26561.html
Hope this helps
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New site migration (multiple sites into one + new domain)
Hi, I have read so many very helpful guides and experiences from you guys that will greatly help me but I have a few questions please. Our company has 3 sites, the main site and 2 sites for different product ranges: BrandProductName.com (main site - DA = 22 raking well for product name) Productname2.com (DA = 10 ranking very well for product name and little competition) BrandProductName3.com (DA = 10 poor ranking) We wish to bring all the sites into one with categories for the 3 different product. The main site is an e-commerece site whereas the other 2 are not (currently). On top of this as the main domain has one of the product names in it they wish to change the domain to be just Brandname.com. So the plan is to combine site 2 and 3 into site 1 and change that domain name. As you can imagine this is going to be quite a job. I am fairly happy with the steps required (having read all the guides and migrated many sites in the past) but with the added domain name change this is a little daunting. So my questions are: Should I merge the 3 sites into 1 and then changed the domain at a later point? Should I change the domain of the main site first and then merge site 2 and 3 in later? Should I just do it all together? Or based on the data i have provided do you disagree with the plan, what would you recommend? We are not in a massive rush to complete all of this so we have the time to plan and execute this when we are fully ready. Any help / advise would be greatly appreciated. Thanks all
Intermediate & Advanced SEO | | csimmo0 -
301 redirecting a site that currently links to the target site
I have a personal blog that has a good amount of back links pointing at it from high quality relevant authoritative sites in my niche. I also run a company in the same niche. I link to a page on the company site from the personal blog article that has bunch of relevant links pointing at it (as it's highly relevant to the content on the personal blog). Overview: Relevant personal blog post has a bunch of relevant external links pointing at it (completely organic). Relevant personal blog post then links (externally) to relevant company site page and is helping that page rank. Question: If I do the work to 301 the personal blog to the company site, and then link internally from the blog page to the other relevant company page, will this kill that back link or will the internal link help as much as the current external link does currently? **For clarity: ** External sites => External blog => External link to company page VS External sites => External blog 301 => Blog page (now on company blog) => Internal link to target page I would love to hear from anyone that has performed this in the past 🙂
Intermediate & Advanced SEO | | Keyword_NotProvided0 -
Dfferent url of some other site is shown by Google in cace copy of our site's page
Hi, When i check cached copy of url of my site http://goo.gl/BZw2Zz , the url in cache copy shown by Google is of some other third party site. Why is Google showing third party url in our site's cached url. Did any of you guys faced any such issue. Regards,
Intermediate & Advanced SEO | | vivekrathore0 -
Spam Links? -115 Domains Sharing the Same IP Address, to Remove or Not Remove Links
Out of 250 domains that link to my site about 115 are from low quality directories that are published by the same company and hosted on the same ip address. Examples of these directories are: -www.keydirectory.net -www.linkwind.com -www.sitepassage.com -www.ubdaily.com -www.linkyard.org A recent site audit from a reputable SEO firm identified 125 toxic links. I assume these are those toxic links. They also identified about another 80 suspicious domains linking to my site. They audit concluded that my site is suffering a partial Penguin penalty due to low quality links. My question is whether it is safe to remove these 125 links from the low quality directories. I am concerned that removing this quantity of links all at once will cause a drop in ranking because the link profile will be thin with only about 125 domains remaining that point to the site. Granted those 125 domains should be of somewhat better quality. I am playing with fire by having these removed. I URGENTLY NEED ADVICE AS THE WEBMASTER HAS INITIATED STEPS TO REMOVE THE 125 LINKS. Thanks everyone!!! Alan
Intermediate & Advanced SEO | | Kingalan10 -
1 Ecommerce site for several product segments or 1 Ecommerce site for each product segment ?
I am currently struggling with the decision whether to create individual ecommerce sites for each of 3 consumer product segments or rather to integrate them all under one umbrella domain. Obviously integration under 1 domain makes link building easier, but I am not sure how far google will favor in rankings websites focussed on one topic=product segment. Product segments are medium competitive.Product segments are not directly related but there may be some overlap in customer demographics- Any thoughts ?
Intermediate & Advanced SEO | | lcourse1 -
Temporarily shut down a site
What would be the best way to temporarily shut down a site the right way and not have a negative impact on SEO?
Intermediate & Advanced SEO | | LibertyTax1 -
Googlebot Can't Access My Sites After I Repair My Robots File
Hello Mozzers, A colleague and I have been collectively managing about 12 brands for the past several months and we have recently received a number of messages in the sites' webmaster tools instructing us that 'Googlebot was not able to access our site due to some errors with our robots.txt file' My colleague and I, in turn, created new robots.txt files with the intention of preventing the spider from crawling our 'cgi-bin' directory as follows: User-agent: * Disallow: /cgi-bin/ After creating the robots and manually re-submitting it in Webmaster Tools (and receiving the green checkbox), I received the same message about Googlebot not being able to access the site, only difference being that this time it was for a different site that I manage. I repeated the process and everything, aesthetically looked correct, however, I continued receiving these messages for each of the other sites I manage on a daily-basis for roughly a 10-day period. Do any of you know why I may be receiving this error? is it not possible for me to block the Googlebot from crawling the 'cgi-bin'? Any and all advice/insight is very much welcome, I hope I'm being descriptive enough!
Intermediate & Advanced SEO | | NiallSmith1 -
How Important is the IP Address for SEO?
Hi Everyone, I am curious to know if IP Address plays any role in SEO....What if a website sharing an I.P with a porn site, BlueFart site, fake viagra pills site etc.? Does it affect the SEO? Please share your opinion on this. Thanks
Intermediate & Advanced SEO | | seodoz0