How a google bot sees your site
-
So I have stumbled across various websites like this:
http://www.smart-it-consulting.com/internet/google/googlebot-spoofer/
The concept here is to be able to view your site as a googlebot sees it. However, the results are a little puzzling. Google is reading the text on my page but not the title tags according to the results. Are websites like this accurate OR does Google not read title tags and H1 tags anymore?
Also on a slighly related note. I noticed the results show the navigation bar is being read first by google, is this bad and should the navigation bar be optimized for keywords as well? If it did, it would read a bit funny and the "humans" would be confused.
-
You need to pull this forum question. That link redirects to a spammy site about "Freeing Syria."
-
The instructions are near the bottom of the page:
In order to use Fetch as Googlebot, you'll need to have added and verified your site in Webmaster Tools. Then, follow these instructions:
- On the Webmaster Tools Home page, click the site you want.
- On the Dashboard, under Diagnostics, click Fetch as Googlebot.
- In the text box, type the path to the page you want to check.
- In the dropdown list, select the type of fetch you want. To see what our web crawler Googlebot sees, select Web. To see what our mobile crawler Googlebot-Mobile sees, select cHTML (this is used mainly for Japanese web sites) or Mobile XHTML/WML.
- Click Fetch.
Once googlebot has fetched your page you'll have a "success" link that you can click on to see what Googlebot saw.
This will be the header, including the server response code and then the html that googlebot received.
What this doesn't tell you is how this was interpreted by Google, of course this is where SEOMoz's on-page reports and crawl stats can help detect errors and way your can improve your on-page optimisation.
-
Thanks, but even under that Fetch as Googlebot link you posted, I don't see how to get an accurate tool of how Google views your site.
-
These sites are good for a quick scan of the contextual formatting of a website, but not for really telling how Google (or any specific search engine) sees your site. That specific one you linked to is horrible.
Google does see title, H1, other headers, meta description and most elements of your site. A more accurate way to see how google sees your site would be to:
- See how the page looks in the index. Type "site:myspecificurl.com" into google for the page you want to see and google will just return the results of what it has in its index. That is how google sees your site. If your site/pages are not in the index, get them in (#2 below).
- Verify through Google Webmaster Tools. In the webmaster tools you can see what pages of your site are being indexed/crawled through google, and you can also request specific pages to be crawled again if you need. This combined with an xml sitemap will usually get pages indexed pretty quick, and then you can verify with the same methodology as i mentioned above.
- Use the SEOmoz pro toolset here and set up a campaign and the tool will tell you if you are missing any title tags or other important on-page elements. the seomoz "bot" crawls similar to google, so that should give you a feel for how it works.
-
A lot of these sites are badly coded garbage. I would ignore these sites.
-
I don't know how far I'd trust such third party sites. Take a look at Google Webmaster Tools. There's a Fetch as Googlebot tool under diagnostics.
Here's the Google help about it::
Google certainly does read title and heading tags!
As far as the navigation bar goes - always think about humans first. Sometimes you can improve relevance by avoiding generic names. Avoid generic terms like "Articles" for instance and replace it with something that better describes the content behind the click "nutrition guide" or "food facts"...
The fact that your navigation is being read first isn't a problem - it's a convention that is hardly going to be penalised by a search engine.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On our site by mistake some wrong links were entered and google crawled them. We have fixed those links. But they still show up in Not Found Errors. Should we just mark them as fixed? Or what is the best way to deal with them?
Some parameter was not sent. So the link was read as : null/city, null/country instead cityname/city
Technical SEO | | Lybrate06060 -
About how google works
Hey guyz,
Technical SEO | | atakala
I want to ask a basic question. If I search for Larry Page lets say.
I think google look for it's index for word larry and page distinct.
And mix it up. But the question ;
Can google show a result which only Larry exist on the page but any of the synonym or the stem of the Page not exist .
If it can happen how this page can be showned in larry page query. Thank you.0 -
How can I make Google Webmaster Tools see the robots.txt file when I am doing a .htacces redirec?
We are moving a site to a new domain. I have setup an .htaccess file and it is working fine. My problem is that Google Webmaster tools now says it cannot access the robots.txt file on the old site. How can I make it still see the robots.txt file when the .htaccess is doing a full site redirect? .htaccess currently has: Options +FollowSymLinks -MultiViews
Technical SEO | | RalphinAZ
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?michaelswilderhr.com$ [NC]
RewriteRule ^ http://www.s2esolutions.com/ [R=301,L] Google webmaster tools is reporting: Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 100.0%.0 -
How can you get the right site links for your site?
Hello all, I have been trying to get Google to list relevant site links for my site when you type in our brand name, Loco2 or for when Loco2 comes up in a search result. Different things come up when you search Loco2 and Loco 2. We would like site links to look like how they do when you search Loco 2. However Loco2 is our brand name, NOT Loco 2. Does anyone know why Google is doing this and whether we can influence results? We have done as much as possible via Google webmaster, in terms of specifying the links we DO NOT want Google to list for Loco2. However, when you search "Loco2", results only show simple site links. Ideally what we want is: Loco2 to be recognised as the brand NOT Loco 2 The same results (substantial, identical) for Loco2 as for Loco 2 (think o2 and o 2) For the site links to reflect the main pages of our site (Times & Tickets, Engine Room forum etc.) Many thanks in advance! Anila
Technical SEO | | anilababla0 -
Sitemaps for Google
In Google Webmaster Central, if a URL is reported in your site map as 404 (Not found), I'm assuming Google will automatically clean it up and that the next time we generate a sitemap, it won't include the 404 URL. Is this true? Do we need to comb through our sitemap files and remove the 404 pages Google finds, our will it "automagically" be cleaned up by Google's next crawl of our site?
Technical SEO | | Prospector-Plastics0 -
My site cannot be found by google at all
I don't know why but our company site can not be found by google at all. I have submitted to google webmaster, have social media point to, etc, Is there any reason for this? url for our website is www.bistosamerica.com Thank you
Technical SEO | | BistosAmerica0 -
Site problem
I moved a site earlier on in the year to a better server www.keyrs.co.uk, my main keywords being equity release - equity release calculator and equity release schemes. Since this happened the ranking have gone down and the schemes and calculator terms and have hit positions 7-8 when they were 2-3. basically my question is open to all, i am looking to see what the problem is with these pages as it is driving me nuts. All tools on SEO moz show the pages are doing well, however i must be missing something. Mike
Technical SEO | | TomBarker820 -
How long does it take for customized Google Site Search to show results from pdf files?
The site in question is http://www.ejmh.eu I am pretty unsatisfied with the results I am getting from the Site Search provided by Google. We have over 160 pdf files in this subfolder: http://www.ejmh.eu/mellekletek The files are the digital versions of articles. When I search for content in those pdf files, Google does not show results. It does show results from older pages, dating back 1-2 years but it is certainly not showing anything from pdf files that I have just put up 3 weeks ago. My questions: If I place a Google Search on a site, does it not automatically display results from ALL the content in the root domain? Is there any correlation between how the Site Search is indexing the files and how Google is indexing the urls in general? Should I just wait and see whether site search performance improves or should I switch to another Search software like Zoom Search? It is vital to have a proper, high-quality search functioning on that site in the very near future. What are your experiences? Any tips are greatly appreciated.
Technical SEO | | Lauroca0