How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When making content pages to a specific page; should you index it straight away in GSC or let Google crawl it naturally?
When making content pages to a specific page; should you index it straight away in GSC or let Google crawl it naturally?
On-Page Optimization | | Jacksons_Fencing0 -
Google Answer Box
I optimized several pages using Rand's post on Google Answer Box: https://moz.com/blog/how-to-appear-in-googles-answer-boxes-whiteboard-friday How long after the page is indexed should it appear? Lastly, how long should I wait before determining it will not get an answer box and reconfigure the page? No bad answers 🙂 TY KJr
On-Page Optimization | | KevnJr0 -
Would you consider this to be thin content
I always struggle with these pages I have on my site going back and forth debating what I want to do with them. On one side Google was content, yet at the same time its all about user experience. http://www.freescrabbledictionary.com/word-lists/words-that-start-with/letter/h/ I used to have all my words listed on one page which could have been well over 10,000. Now I pagination them as you can see. I debate writing a header of content for these pages, but honestly users just want the words. Get in, get what you need and get out. What is the recommendation on these pages. Should I write content? Should I not?
On-Page Optimization | | cbielich0 -
On-Site Optimization Issue!
Hello, I have some confusion about how to structure my site to better in on-site optimization. I am using WordPress. Therefore, there are many things that I need to consider as following: Static Page for homepage OR Latest posts? Archive, Category, Author, Attachment and Tag pages - To put meta robots (no index, follow) or not to prevent duplication? If I use Static Page for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? If I use Latest Posts for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? To have breadcrumb or not? To have recent posts, comment, tag clouds or popular posts/comments widget or not? To have social sharing icons and related posts in single post or not? If you don't mind adding more tips that I don't know it would be very great! Thanks!
On-Page Optimization | | dinabrokoth0 -
"Turning off" content to a site
One site I manage has a lot of low quality content. We are in the process of improving the overall site content but we have "turned off" a large portion of our content by setting 2/3 of the posts to draft. Has anyone done this before or had experience with doing something similar? This quote from Bruce Clay comes to mind: “Where a lot of people don’t understand content factoring to this is having 100 great pages and 100 terrible pages—they average, when the quality being viewed is your website,” he explained. “So, it isn’t enough to have 100 great pages if you still have 100 terrible ones, and if you add another 100 great pages, you still have the 100 terrible ones dragging down your average. In some cases we have found that it’s much better, to improve your ranking, to actually remove or rewrite the terrible ones than add more good ones.” What are your thoughts? Thanks
On-Page Optimization | | ThridHour0 -
How Google differentiates web sites like directories?
Hi, I want to ask how google differentiates web sites like directories or company listing websites? How it understands that is a normal thing to have many links in a directory site? Are there some guides links about what to do and avoid and how to make SEO optimization for a directory web site.
On-Page Optimization | | vladokan0 -
Can you have more than 1 site on the first page if site look and content is completely different but keywords are the sam.
I have a client that wants to build another completely different site than his main site and optimize it to have 2 websites on the first page for his keywords. The content and look and feel of the website would be completely different. One of his competitors is doing it and getting away with it. What is your advice.
On-Page Optimization | | Roots70 -
Google Reconsideration
Our site fell from grace last July and landed on page five of the Google search results for our primary keyword. For 6 months I tried a number of strategies with no results, including reconfiguring our site based on the SOEmoz on-page grading tool. More recently, after receiving your advice in a Q&A, I took down all of my paid links and submitted a reconsideration request to Google. Interestingly, 3 days later we popped up 20 spots. This left us on the top of page three. Better than page 5, but still not prime time! A few days ago (two weeks after our reconsideration request was submitted) I got a message back in my Webmaster Tools, that they had completed a review of our site - but oddly enough they provided no info on the outcome, positive or negative. And there has been no additional movement in the rankings since I received the message. Was the original 20 point jump the result of the reconsideration request, or just a coincidence? Or, is it possible that they did a review and the results will only occur later during some organic re-indexing process? What do you think?
On-Page Optimization | | JimSkychief0