How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO and dynamic content
I am working on a project right now and I am looking for some advice on the SEO implications. The site is an e-commerce site and on the category pages it is using an external call to retrieve the products after the page is loaded. How it works is all content on the site is loaded, then after that a js script appends an ID and loads all of the product information. I am unsure how Google will see this, anyone have any insights?
On-Page Optimization | | LesleyPaone0 -
2000 Active pages 404 on LIVE Ecommerce site - what will google do now?
Hi All, One of my ecommerce site having more than 20,000 pages from that one of the categories having 2000 pages showing 404 and still taking time for developer to fix this issue and may be they will be able to fix after 2-3 days so is this okay with google or google will take any action during this period? Thanks! Dev
On-Page Optimization | | devdan0 -
Pre-launch site or not
We are going to set up a new site in four months. Historically we always set up a simple Wordpress "Pre-launch-site" with relevant texts to start ranking in the SERP. Anyone with experience of doing/not doing this and what is had led to? A site with relevant texts also should have incoming links, which needs more work.
On-Page Optimization | | fredrikahlen0 -
90 days for Google
Hi, I'm new to Moz so still getting a feel of the forums. If my question has been answered then please point me in the right direction. I have noticed with many SEO companies they advertise that they can get you on google front page in 90 days. I'm not really interested in their techniques but more of why google takes 90+ to even appear. I have been working on my site for over a month, adding content, building good links, social media, blogs etc... but have not even come close to appearing in the top 50 pages for google. Is this normal? Is it just a matter of time before it starts to appear? Also, I have checked my backlinks and there is about 8 links that are coming from random pages in the US and some from China and india which i have no idea of. I tried to visit on of the sites but it had malware. I added all these back links to google disavow so hopefully that will fix it. Could that be the reason google would not even list my site? Thanks... Rick
On-Page Optimization | | pureozone0 -
"Issue: Duplicate Page Content " in Crawl Diagnostics - but these pages are noindex
Saw an issue back in 2011 about this and I'm experiencing the same issue. http://moz.com/community/q/issue-duplicate-page-content-in-crawl-diagnostics-but-these-pages-are-noindex We have pages that are meta-tagged as no-everything for bots but are being reported as duplicate. Any suggestions on how to exclude them from the Moz bot?
On-Page Optimization | | Deb_VHB0 -
Google Index HTTPS
Hi,
On-Page Optimization | | JohnHuynh
I had a HTTP protocol file which indexed. Now I want to change this file to HTTPS protocol. I wonder that is there any effects?
I don't know HTTPS would be indexed by google or not? Thanks,0 -
Google Drop
I started using SEOMOZ due to a sudden and huge drop in Google for two main keywords (hair bows and baby headbands). Our site (BloomingBows.com) has held a top three spot for years with these words and then in the last few months has dropped down on the first page and now they are completely off the charts. Is there any insight as to why? Also, we have been very active using the data from here in the last week or so to clean up and improve anything listed, but I am still seeing keywords drop into the 40 - 60 position and our traffic is drying up. Starting to panic and wondering if I am missing something or going about this in the wrong way. ANY insight is appreciated at this point!! Thank you!!
On-Page Optimization | | bloomingB0 -
Website Content
Is it bad to have html pages on a blog? I converted a completely HTML site to wordpress, but havd hundreds of article pages that are still html.
On-Page Optimization | | azguy0
