How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO and dynamic content
I am working on a project right now and I am looking for some advice on the SEO implications. The site is an e-commerce site and on the category pages it is using an external call to retrieve the products after the page is loaded. How it works is all content on the site is loaded, then after that a js script appends an ID and loads all of the product information. I am unsure how Google will see this, anyone have any insights?
On-Page Optimization | | LesleyPaone0 -
Any idea how Google is doing this? Is it schematic? http://techcrunch.com/2014/02/28/google-adds-full-restaurant-menus-to-its-search-results-pages/
Google is now showing menus on select searches. Any idea how they are getting this information? I would like to make sure my clients get visibility this way.
On-Page Optimization | | Ron_McCabe0 -
Pre-launch site or not
We are going to set up a new site in four months. Historically we always set up a simple Wordpress "Pre-launch-site" with relevant texts to start ranking in the SERP. Anyone with experience of doing/not doing this and what is had led to? A site with relevant texts also should have incoming links, which needs more work.
On-Page Optimization | | fredrikahlen0 -
Why isn't our site being shown on the first page of Google for a query using the exact domain, when its pages are indeed indexed by Google
When I type our domain.com as a query into Google, I only see one of our pages on the homepage, and it's in 4th position. It seems though, that all pages of the site are indexed by google when I type in the query "site:domain.com". There was an issue at the site launch, where the robots.txt file was left active for around two weeks. Would this have been responsible for the fact that another domain ranks #1 when we type in our own domain? It has been around a couple of months now since the site was launched. Thanks in advance.
On-Page Optimization | | featherseo0 -
Duplicate content - what to do?
Hi, We have a whole lot of articles on our site. In total 5232 actually. The web crawler tells me that in the articles we have a lot of duplicate content. Which is sort of nonsense, since each article is unique. Ah, some might have some common paragraphs because they are recurring news about a weekly competition. But, an example: http://www.betxpert.com/artikler/bookmakere/brandvarme-ailton-snupper-topscorerprisen AND http://www.betxpert.com/artikler/bookmakere/opdaterede-odds-pa-sportschef-situationen-pa-vestegnen These are "duplicate content", however the two article texts are not the same. The menu, and the widgets are all the same, but highly relevant to the article. So what should I do? How can i rid myself of these errors? -Rasmus
On-Page Optimization | | rasmusbang0 -
What is causing Bing and Google Rankings to Differ by so much?
Does anyone know the trick to have Google Rankings to be as good as Bing/Yahoo Results?
On-Page Optimization | | hfranz0 -
On my site, www.myagingfolks.com, only a small number of my pages appear to be indexed by google or yahoo. Is that due to not having an XML sitemap, keywords, or some other problem?
On my site, www.myagingfolks.com, only a small number of my pages appear to be indexed by google or yahoo. I have thousands of pages! Is that due to not having an XML sitemap, keywords, or some other problem?
On-Page Optimization | | Jordanrg0 -
Google and display:none
Hi Guys, i want to know what you think about solution which i have to switch content in tab on my page. Here: http://www.exprestlac.sk/beta/produkt/vizitky i have some important content in tabs, which are switching via javascript. So when you click there on O produkte next to Ceny it will show you product description. My problem is that in source code when page is loaded i have this: Product description.. And after user click on O produkte javascript remove that display:none and show content. But Google will see only display:none as i think. Can i get penalty from Google? Will it index this text? Thanks for your suggestions how to resolve this.
On-Page Optimization | | xman870