Can Googlebot read the content on our homepage?
-
Just for fun I ran our homepage through this tool:
http://www.webmaster-toolkit.com/search-engine-simulator.shtml
This spider seems to detect little to no content on our homepage. Interior pages seem to be just fine. I think this tool is pretty old. Does anyone here have a take on whether or not it is reliable? Should I just ignore the fact that it can't seem to spider our home page?
Thanks!
-
Thanks all! Yes, I was familiar with the "Text-only" version and the Fetch as Googlebot, so I wasn't overly concerned. It just seemed odd that this particular spider couldn't get to the content. I think it is a very unsophisticated spider!
-
Assuming you've verified your site in Google Webmaster Tools, you can go in there and to go Crawl > Fetch as Googlebot. Put that page, and have Googlebot fetch it. Once it's done, you can click on the "Success" link, and this will show you exactly what Googlebot fetched with regards to that page. Make sure the source code you're seeing here is what you expect.
-
Hi Dana,
We would normally check through something like Website Auditor... I've run the tool on our home page and it seems to be missing some parts of our content, not sure why. Never had an issue before though with other tools, so would put it down to this tool....
Hope that helps.
-
Take a look at the text-only cached version of the page. If you are unsure how to do that follow my crude instructions below.
What I do to test if Googlebot can view the content of my homepage:
Do a Google search for 'site:example.com' and find your homepage. Next to the green URL in the SERP listing for your homepage there is a green arrow. Click that and select 'cached'. Then, when viewing the cached version of the homepage, click 'Text-only version' in the bottom right corner of the grey bar that appears at the top of the browser.
If the content you are questioning shows up, there is a good chance Google has obviously been able to crawl and index it. If the content is not there, there is a good chance they can't. If the content is in a hidden div it will likely still not show up in the text-only cache.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap For Static Content And Blog
We'll be uploading a sitemap to google search console for a new site. We have ~70-80 static pages that don't really chance much (some may change as we modify a couple pages over the course of the year). But we have a separate blog on the site which we will be adding content to frequently. How can I set up the sitemap to make sure that "future" blog posts will get picked up and indexed. I used a sitemap generator and it picked up the first blog post that's on the site, but am wondering what happens with future ones? I don't want to resubmit a new sitemap each time that has a link to a new blog post we posted.
Technical SEO | | vikasnwu0 -
Can Anybody Understand This ?
Hey guyz,
Technical SEO | | atakala
These days I'm reading the paperwork from sergey brin and larry which is the first paper of Google.
And I dont get the Ranking part which is: "Google maintains much more information about web documents than typical search engines. Every hitlist includes position, font, and capitalization information. Additionally, we factor in hits from anchor text and the PageRank of the document. Combining all of this information into a rank is difficult. We designed our ranking function so that no particular factor can have too much influence. First, consider the simplest case -- a single word query. In order to rank a document with a single word query, Google looks at that document's hit list for that word. Google considers each hit to be one of several different types (title, anchor, URL, plain text large font, plain text small font, ...), each of which has its own type-weight. The type-weights make up a vector indexed by type. Google counts the number of hits of each type in the hit list. Then every count is converted into a count-weight. Count-weights increase linearly with counts at first but quickly taper off so that more than a certain count will not help. We take the dot product of the vector of count-weights with the vector of type-weights to compute an IR score for the document. Finally, the IR score is combined with PageRank to give a final rank to the document. For a multi-word search, the situation is more complicated. Now multiple hit lists must be scanned through at once so that hits occurring close together in a document are weighted higher than hits occurring far apart. The hits from the multiple hit lists are matched up so that nearby hits are matched together. For every matched set of hits, a proximity is computed. The proximity is based on how far apart the hits are in the document (or anchor) but is classified into 10 different value "bins" ranging from a phrase match to "not even close". Counts are computed not only for every type of hit but for every type and proximity. Every type and proximity pair has a type-prox-weight. The counts are converted into count-weights and we take the dot product of the count-weights and the type-prox-weights to compute an IR score. All of these numbers and matrices can all be displayed with the search results using a special debug mode. These displays have been very helpful in developing the ranking system. "0 -
2 Versions of Same Homepage
We want to show new and returning visitors different versions of our homepage (same URL) What, if anything, should we use as the markup to tell Google what we are doing?
Technical SEO | | theLotter
Any danger that Google will think we are cloaking? Thanks!0 -
Can I canonical the same page?
I have a site where I have 500+ Page listing pages and I would like to rel=canonical them to the master page. Example: http://www.example.com//articles?p=18 OR http://www.example.com/articles?p=65 I plan on adding this to the section from of the page template so it goes to all pages - When I do this, I will also add the canonical to the page I am directing the canonical. Is this a bad thing? Or allowed?
Technical SEO | | JoshKimber0 -
Duplicate Content
Hi, we need some help on resolving this duplicate content issue,. We have redirected both domains to this magento website. I guess now Google considered this as duplicate content. Our client wants both domain name to go to the same magento store. What is the safe way of letting Google know these are same company? Or this is not ideal to do this? thanks
Technical SEO | | solution.advisor0 -
If two websites pull the same content from the same source in a CMS, does it count as duplicate content?
I have a client who wants to publish the same information about a hotel (summary, bullet list of amenities, roughly 200 words + images) to two different websites that they own. One is their main company website where the goal is booking, the other is a special program where that hotel is featured as an option for booking under this special promotion. Both websites are pulling the same content file from a centralized CMS, but they are different domains. My question is two fold: • To a search engine does this count as duplicate content? • If it does, is there a way to configure the publishing of this content to avoid SEO penalties (such as a feed of content to the microsite, etc.) or should the content be written uniquely from one site to the next? Any help you can offer would be greatly appreciated.
Technical SEO | | HeadwatersContent0 -
If I 301 re-direct a piece of content (A) to another piece of content (B) and B is unrelated in subject matter to A, will the referring search keywords to content piece A hold for content piece B?
For example, I have a piece of content about furniture and it ranks in top 5 in the SERPs for the phrase "furniture". If I were to 301 redirect that piece of furniture content to a piece of content about trucks, would the referring keyword "furniture" continue to rank over time for the trucks content? My instincts tell me that in the short term the content piece about trucks would receive traffic for the term "furniture", but over the long term, the trucks content would lose rankings for the term "furniture" since the piece has to do with trucks and not furniture. Any thoughts?
Technical SEO | | pbrothers240 -
Content Delivery Network
Anyone have a good reference for implementing a content delivery network? Any SEO pitfalls with using a CDN (brief research seems to indicate no problems)? I seem to recall that SEOmoz was using Amazon Web Services (AWS) for CDN. Is that still the case? All CDN & AWS experiences, advice, references welcomed!
Technical SEO | | Gyi0