How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Inbound Linking from your own sites
Good evening, On each of the sites I have made, I have a link with the anchor text 'Build and Design by Christoper Davies' to my own website. This link is in the footer of every page each of all the sites. Should I have a 'no follow' rel added to these links, or does linking from all the sites (on all pages) help my ranking? I am concerned that having so many inbound links from the same sites, with the same anchor text may be doing me more damage than good.
On-Page Optimization | | chrisdavieswebdesign0 -
Google is not taking the title I set
Hi Everyone, I have the following website I am trying to SEO www.forcor.com.ar. As usual, the client decided to SEO the website after it was made by a developer that is not specialized in SEO. I am having the following issue. Google is not picking up the title I defined for the homepage. I have changed it multiple times and Google continues to take the title that was defined when the website was launched which was about 6 months ago. I also tried to verify if the title is being defined by some other website and so far I do not find any evidence that supports this. This is the title seen in the SERP (GOOGLE) as of 6 months ago - Forcor S.A: Repuestos Ford - Concesionario Oficial de Ford This is the one that is programmed via Yoast - <title>Repuestos Ford - Concesionario Oficial de Ford - Forcor S.A</title> The funny thing is that Google is picking up the correct Meta Description. Any suggestions on how to get Google to pick up the programmed title??? Another interesting fact is that Yahoo is picking up the correct title as you can see here
On-Page Optimization | | Carla_Dawson0 -
Mass Duplicate Content
Hi guys Now that the full crawl is complete I've found the following: http://www.trespass.co.uk/mens-onslow-02022 http://www.trespass.co.uk/mens-moora-01816 http://www.trespass.co.uk/site/writeReview?ProductID=1816 http://www.trespass.co.uk/site/writeReview?ProductID=2022 The first 2 duplicate content is easily fixed by writing better product descriptions for each product (a lot of hours needed) but still an easy fix. The last 2 are review pages for each product which are all the same except for the main h1 text. My thinking is to add no index and no follow to all of these review pages? The site will be changing to magento very soon and theres still a lot of work to do. If anyone has any other suggestions or can spot any other issues, its appreciated. Kind regards Robert
On-Page Optimization | | yournetbiz1 -
Duplicate Content
Is making tabs with general product information on similar products considered duplicate content?
On-Page Optimization | | BridalHotspot0 -
Why isn't our site being shown on the first page of Google for a query using the exact domain, when its pages are indeed indexed by Google
When I type our domain.com as a query into Google, I only see one of our pages on the homepage, and it's in 4th position. It seems though, that all pages of the site are indexed by google when I type in the query "site:domain.com". There was an issue at the site launch, where the robots.txt file was left active for around two weeks. Would this have been responsible for the fact that another domain ranks #1 when we type in our own domain? It has been around a couple of months now since the site was launched. Thanks in advance.
On-Page Optimization | | featherseo0 -
What is the best way to manage industry required duplicate Important Safety Information (ISI) content on every page of a site?
Hello SEOmozzer! I have recently joined a large pharmaceutical marketing company as our head SEO guru, and I've encountered a duplicate content related issue here that I'd like some help on. Because there is so much red tape in the pharmaceutical industry, there are A LOT of limitations on website content, medication and drug claims, etc. Because of this, it is required to have Important Safety Information (ISI) clearly stated on every page of the client's website (including the homepage). The information is generally pretty lengthy, and in some cases is longer than the non-ISI content on each page. Here is an example: http://www.xifaxan.com/ All content under the ISI header is required on each page. My questions are: How will this duplicated content on each page affect our on-page optimization scores in the eyes of search engines? Is Google seeing this simply as duplicated content on every page, or are they "smart" enough to understand that because it is a drug website, this is industry standard (and required)? Aside from creating more meaty, non-ISI content for the site, are there any other suggestions you have for handling this potentially harmful SEO situation? And in case you were going to suggest it, we cannot simply have an image of the content, as it may not be visible by all internet users. We've already looked into that 😉 Thanks in advance! Dylan
On-Page Optimization | | MedThinkCommunications0 -
On-Site Optimization and Repeat Customers
Hello, One of my clients has repeat customers. All of his surface level categories are optimized. None of his 2nd or 3rd level categories are optimized at all. Is there any harm in optimizing these 2nd or 3rd level categories. Is there any way it could cause a problem with repeat customers? I didn't think so but I want to make sure. Thanks!
On-Page Optimization | | BobGW0 -
Major update to site architecture (outline)-Is Google going to drop?
I'm working with a lawyer client who has a table-based, outdated site. Her nav links consist of a jumble of topics and static pages in one long sidebar list on the home page. I'm moving her site to Wordpress and I've recommended that she organize the site based on categories that roughly match the topics/keywords she wants to rank highest for in Google. The site will be much better organized and coded and the URLs for the new launch will be much stronger for SEO by being targeted and coded properly. So the site should rank better after, right? Right??? I know that when Google crawls the new architecture, it's not going to find the expected long sidebar list of internal nav links. It'll find better, more keyword targeted internal nav links. But will that keep the site from getting dropped off page 1? I'm speaking w/ the client tomorrow and if she's going to drop or get bounced around, I feel like I should prepare her and let her know roughly what might happen. I'm thinking based on my current understanding that I should tell her to expect to be bounced around for a few weeks, but in the end she should rank higher than before. What would you do/say?
On-Page Optimization | | bvrob0
