How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Question on site structure
My client is a nationwide company. They provide building maintenance services in 7 different cities. In each city they provide a different range of services. They currently have a single service page for each service and no mention on that page of the cities they offer the service. The service pages are getting no SERP visibility. We are running Paid Search and recommending SEO. I'm wondering whether it would be beneficial to build out specific service pages for each city so the content is more relevant to both users and search engines. What is best practice in this situation? Client wants to dominate SERPs in each market for the services they offer.
On-Page Optimization | | SEOinSunnyNelson0 -
Google Answer Box
I optimized several pages using Rand's post on Google Answer Box: https://moz.com/blog/how-to-appear-in-googles-answer-boxes-whiteboard-friday How long after the page is indexed should it appear? Lastly, how long should I wait before determining it will not get an answer box and reconfigure the page? No bad answers 🙂 TY KJr
On-Page Optimization | | KevnJr0 -
Can I robots.txt an entire site to get rid of Duplicate content?
I am in the process of implementing Zendesk and will have two separate Zendesk sites with the same content to serve two separate user groups (for the same product-- B2B and B2C). Zendesk does not allow me the option to changed canonicals (nor meta tags). If I robots.txt one of the Zendesk sites, will that cover me for duplicate content with Google? Is that a good option? Is there a better option. I will also have to change some of the canonicals on my site (mysite.com) to use the zendesk canonicals (zendesk.mysite.com) to avoid duplicate content. Will I lose ranking by changing the established page canonicals on my site go to the new subdomain (only option offered through Zendesk)? Thank you.
On-Page Optimization | | RoxBrock0 -
Content in Tabs
I speed read an article recently and forgot to save it regarding Contents on a page in tabs. Is it correct that now Google is rendering the entire page it's better not to have content in tabs hidden by Javascript? As it stands at the moment, we've got the tabs set-up so that the main part of the page containing the keyword rich text is in a tab and not the first thing presented to the user
On-Page Optimization | | Ham19790 -
Does CloudFlare Benefit my site?
I am using the $20 / month CloudFlare service. My site is not running faster, so my question is: is this service adding real value? I do not have a lot of highly sensitive customer data (some email addresses and customer names and nothing more - no credit cards). I am already using Amazon Cloud Server and it appears to work fine. Any thoughts appreciated.
On-Page Optimization | | khi50 -
Google Custom Search
Hello, Is there anyone use goolge custom search? and will it improve my website ranking?
On-Page Optimization | | JohnHuynh0 -
Creating Duplicate Content on Shopping Sites
I have a client with an eCommerce site that is interested in adding their products to shopping sites. If we use the same information that is on the site currently, will we run into duplicate content issues when those same products & descriptions are published on shopping sites? Is it best practice to rewrite the product title and descriptions for shopping sites to avoid duplicate content issues?
On-Page Optimization | | mj7750 -
Duplicate content because of content scrapping - please help
We manage brands websites in a very competitive industry that have thousands of affiliate links We see that more and more websites (mainly affiliates websites) are scrapping our brand websites content and it generate many duplicate content (but most of them link to us back with an affiliate link). Our brand websites still rank for any sentence in brackets you search in Google, Will this duplicate content hurt our brand websites ? If yes, should we take some preventive actions ? We are not able to add ongoing UGC or additional text to all our duplicate content and trying to stop those websites of stealing our content is like playing cat and mouse... Thanks for your advices
On-Page Optimization | | Tit0
