How a google bot sees your site
-
So I have stumbled across various websites like this:
http://www.smart-it-consulting.com/internet/google/googlebot-spoofer/
The concept here is to be able to view your site as a googlebot sees it. However, the results are a little puzzling. Google is reading the text on my page but not the title tags according to the results. Are websites like this accurate OR does Google not read title tags and H1 tags anymore?
Also on a slighly related note. I noticed the results show the navigation bar is being read first by google, is this bad and should the navigation bar be optimized for keywords as well? If it did, it would read a bit funny and the "humans" would be confused.
-
You need to pull this forum question. That link redirects to a spammy site about "Freeing Syria."
-
The instructions are near the bottom of the page:
In order to use Fetch as Googlebot, you'll need to have added and verified your site in Webmaster Tools. Then, follow these instructions:
- On the Webmaster Tools Home page, click the site you want.
- On the Dashboard, under Diagnostics, click Fetch as Googlebot.
- In the text box, type the path to the page you want to check.
- In the dropdown list, select the type of fetch you want. To see what our web crawler Googlebot sees, select Web. To see what our mobile crawler Googlebot-Mobile sees, select cHTML (this is used mainly for Japanese web sites) or Mobile XHTML/WML.
- Click Fetch.
Once googlebot has fetched your page you'll have a "success" link that you can click on to see what Googlebot saw.
This will be the header, including the server response code and then the html that googlebot received.
What this doesn't tell you is how this was interpreted by Google, of course this is where SEOMoz's on-page reports and crawl stats can help detect errors and way your can improve your on-page optimisation.
-
Thanks, but even under that Fetch as Googlebot link you posted, I don't see how to get an accurate tool of how Google views your site.
-
These sites are good for a quick scan of the contextual formatting of a website, but not for really telling how Google (or any specific search engine) sees your site. That specific one you linked to is horrible.
Google does see title, H1, other headers, meta description and most elements of your site. A more accurate way to see how google sees your site would be to:
- See how the page looks in the index. Type "site:myspecificurl.com" into google for the page you want to see and google will just return the results of what it has in its index. That is how google sees your site. If your site/pages are not in the index, get them in (#2 below).
- Verify through Google Webmaster Tools. In the webmaster tools you can see what pages of your site are being indexed/crawled through google, and you can also request specific pages to be crawled again if you need. This combined with an xml sitemap will usually get pages indexed pretty quick, and then you can verify with the same methodology as i mentioned above.
- Use the SEOmoz pro toolset here and set up a campaign and the tool will tell you if you are missing any title tags or other important on-page elements. the seomoz "bot" crawls similar to google, so that should give you a feel for how it works.
-
A lot of these sites are badly coded garbage. I would ignore these sites.
-
I don't know how far I'd trust such third party sites. Take a look at Google Webmaster Tools. There's a Fetch as Googlebot tool under diagnostics.
Here's the Google help about it::
Google certainly does read title and heading tags!
As far as the navigation bar goes - always think about humans first. Sometimes you can improve relevance by avoiding generic names. Avoid generic terms like "Articles" for instance and replace it with something that better describes the content behind the click "nutrition guide" or "food facts"...
The fact that your navigation is being read first isn't a problem - it's a convention that is hardly going to be penalised by a search engine.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google News problem
Hello to all. The latest Google algorithm changes have had a big impact on the way that Google news features stories, at least in my country. I've been featured heavily in Google News until about 6th of october, when the changes had the biggest impact, but since then, I haven't been featured at all. Prior to this, I would be featured for keywords on almost any article, not necessarily on the 1st position, but I was almost always there. Posts still show up in the dedicated News category, but not in the main search pages. I've seen a lot of websites being impacted, but some with lower ranks than mine still show up there. I haven't done any changes prior to the 6th of october, and I haven't done any link building campaings, just getting links from higher ranking news sites in my country, for articles I wrote. What I'd like to know is if there were any major changes for Google News and I'm not complying with any of them, or If i could check to see if there are any other problems. I don't have any penalties disclosed by Google, and no new errors in the Webmasters console, I'm just baffled by the fact that overnight the website was completely cut off from being featured in Google News. And one other strange thing, I'm now ranking better for searches that are kind of opposite to my website's main theme. Think about mainly writing about BMW, and less about AUDI, but ranking a lot better for the latter, and a lot less for the other. Thank you.
Technical SEO | | thefrost0 -
Webmaster tools not showing links but Moz OSE is showing links. Why can't I see them in the Google Search Console
Hi, Please see attached photos. I have a website that shows external follow links when performing a search on open site explorer. However, they are not recognised or visible in search console. This is the case for both internal and external links. The internal links are 'no follow' which I am getting developer to rectify. Any ideas why I cant see the 'follow' external links? Thanks in advance to those who help me out. Jesse T7dkL5s T7dkL5s OkQmPL4 3qILHqS
Technical SEO | | jessew0 -
Why is my site crawled so much more in Bing then in Google?
I recently setup Cloudflare so I can see how much my site is being crawled. It looks like Bing is crawling me about 3 times as much as Google. Any ideas on why that would be or what I should check?
Technical SEO | | EcommerceSite0 -
See Different Landing page for my main keyword in google search result
I have a website like http://www.bannerbuzz.com, i am promoting home page with vinyl banners keyword, but currently i can see my website's review page for vinyl banners result in google, i want to display my home page instead of review page for my keyword result in google, its frequently change, some time i can see home page for it and some time it shows review page as i attached image. i want to show my home page, so can you please help me to solve it, how can i stable my home page with main keywords. OtOXxiE.png
Technical SEO | | CommercePundit0 -
I noticed all my SEOed sites are getting attacked constantly by viruses. I do wordpress sites. Does anyone have a good recommendation to protect my clients sites? thanks
We have tried all different kinds of security plugins but none seem to work long term.
Technical SEO | | Carla_Dawson0 -
A site is not being indexed by Google Yahoo or Bing
This site - http://adoptionconnection.org/ is not being indexed by any of the search engines. I checked the easy stuff - robots text is: <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">all, index, follow</a>" /> <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noodp</a>" /> <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noydir</a>" /> I have checked what I can determine would cause the issue but have found nothing to prevent it from being indexed. I'm thinking it may be re-directs etc. Any answer would be great. Thanks in advance,
Technical SEO | | Intergen0 -
Google and QnA sites
My website has a QnA site - a bit like this one except it's not private to premium members. It is a page with a left colomn for category links and it has a list of recently asked questions, each question is a link to view the full question and answers etc. Does google know this is a QnA ? Or will it say - hey, there are far too many links on this page, tut tut. Is there anything I can do to help it understand what the page is.
Technical SEO | | borderbound0 -
How to setup tumblr blog.site.com to give juice to site.com
Is it possible to get a subdomain blog.site.com that is on tumblr to count toward site.com. I hoped I could point it in webmaster tools like we do www but alas no. Any help would be greatly appreciated.
Technical SEO | | oznappies0