Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Can too many pages hurt crawling and ranking?
-
Hi,
I work for local yellow pages in Belgium, over the last months we introduced a succesfull technique to boost SEO traffic: we have created over 150k of new pages, all targeting specific keywords and all containing unique content, a site architecture to enable google to find these pages through crawling, xml sitemaps, .... All signs (traffic, indexation of xml sitemaps, rankings, ...) are positive. So far so good.
We are able to quickly build more unique pages, and I wonder how google will react to this type of "large scale operation": can it hurt crawling and ranking if google notices big volumes of content (unique content)?
Please advice
-
Hi,
I don't believe having toooooooo many pages will hurt crawling and ranking. Actually having a lot of pages will give crawl bots more pages to crawl and when someone searches for keywords related to your pages, your pages might show up.
The only 2 problems I see from having too many pages are
-
With all these pages, are they all unique? With a lot of pages, it will be hard to manager and to keep track if all of them are unique. If you don't have unique pages and have a lot of duplicate, that will hurt your ranking.
-
The second problem is are you inter-linking all your pages? Can the bot crawl all your pages? You will need to have a good linking system and direct bots to different pages for them to crawl. Having a lot of pages will be difficult to manage as I mentioned above. Can you interlink all of them so the bots can crawl all of them? One solution I see to this is submitting a Sitemap but I am not sure if they will index everything since I had a problem with Google only indexing 4% of my sitemap and still can't find solution.
Hope this helps!
-
-
This is really just speculation...
It sounds like you're solid on the on-page, site architecture side. I would assume that crawling and indexation will slow down though if your offsite signals don't keep up though. By this, I mean that Google might see that you're doing everything right on your end, but that over time you're not creating content that very many people care to link to, share, etc, so they'll stop wasting resources on you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I want to move some pages of my website to a folder and nav menu in those pages should only show inner page links, will it hurt SEO?
Hi, My website has a few SaaS products, to make my website simple i want to move my website some pages to its specific folder structure , so eg website.com/product1/features
Technical SEO | | webbeemoz
website.com/product1/pricing
website.com/product1/information and same for product2 and so on, the website.com/product1/.. menu will only show the links of product1 and only one link to homepage (possibly in footer). Please share your opinion will it be a good idea, from UI perspective it will be simple , but i am not sure about SEO perspective, please help thanks1 -
Can an external firewall affect rankings?
For security reasons we are now routing traffic through an external firewall cum CDN. Our server and domain IPs remain the same, but any request is routed through an external IP, which then forwards the traffic. Would our rankings be affected because of IP changes? Thanks Sam
Technical SEO | | samgold0 -
Will Google crawl and rank our ReactJS website content?
We have 250+ products dynamically inserted and sorted on our site daily (more specifically our homepage... yes, it's a long page). Our dev team would like to explore rendering the page server-side using ReactJS. We currently use a CDN to cache all the content, which of course we would like to continue using. SO... will Google be able to crawl that content? We've read some articles with different ideas (including prerendering): http://andrewhfarmer.com/react-seo/
Technical SEO | | Jane.com
http://www.seoskeptic.com/json-ld-big-day-at-google/ If we were to only load the schema important to the page (like product title, image, price, description, etc.) from the server and then let the client render the remaining content (comments, suggested products, etc.), would that go against best practices? It seems like that might be seen as showing the googlebot 1 version and showing the site visitor a different (more complete) version.0 -
Home Page Ranking Instead of Service Pages
Hi everyone! I've noticed that many of our clients have pages addressing specific queries related to specific services on their websites, but that the Home Page is increasingly showing as the "ranking" page. For example, a plastic surgeon we work with has a page specifically talking about his breast augmentation procedure for Miami, FL but instead of THAT page showing in the search results, Google is using his home page. Noticing this across the board. Any insights? Should we still be optimizing these specific service pages? Should I be spending time trying to make sure Google ranks the page specifically addressing that query because it SHOULD perform better? Thanks for the help. Confused SEO :/, Ricky Shockley
Technical SEO | | RickyShockley0 -
Is there a limit to how many URLs you can put in a robots.txt file?
We have a site that has way too many urls caused by our crawlable faceted navigation. We are trying to purge 90% of our urls from the indexes. We put no index tags on the url combinations that we do no want indexed anymore, but it is taking google way too long to find the no index tags. Meanwhile we are getting hit with excessive url warnings and have been it by Panda. Would it help speed the process of purging urls if we added the urls to the robots.txt file? Could this cause any issues for us? Could it have the opposite effect and block the crawler from finding the urls, but not purge them from the index? The list could be in excess of 100MM urls.
Technical SEO | | kcb81780 -
404 error - but I can't find any broken links on the referrer pages
Hi, My crawl has diagnosed a client's site with eight 404 errors. In my CSV download of the crawl, I have checked the source code of the 'referrer' pages, but can't find where the link to the 404 error page is. Could there be another reason for getting 404 errors? Thanks for your help. Katharine.
Technical SEO | | PooleyK0 -
Crawling image folders / crawl allowance
We recently removed /img and /imgp from our robots.txt file thus allowing googlebot to crawl our image folders. Not sure why we had these blocked in the first place, but we opened them up in response to an email from Google Product Search about not being able to crawl images - which can/has hurt our traffic from Google Shopping. My question is: will allowing Google to crawl our image files eat up our 'crawl allowance'? We wouldn't want Google to not crawl/index certain pages, and ding our organic traffic, because more of our allotted crawl bandwidth is getting chewed up crawling image files. Outside of the non-detailed crawl stat graphs from Webmaster Tools, what's the best way to check how frequently/ deeply our site is getting crawled? Thanks all!
Technical SEO | | evoNick0 -
Is a redirect based on a session cookie hurting rankings?
My clients business is divided in chain stores. All stores are set under the same franchise. There is one domain www.company.com with branches like www.company.com/location1/content and www.company.com/location2/content etc. I've taken care of duplicate content issues with rel="canonical" and duplicate page titles are also not a concern, anymore. Right now the concept is like this: If you visit the site for the first time you get to choose between the locations. Then a cookie is set and once you revisit www.company.com it will redirect you via a php header command to the location stored in your cookie: www.company.com/location1/content. My question is if this might hurt rankings in some kind of way as these aren't permanent redirects with a 301 but rather individual ones, based on your cookie.
Technical SEO | | jfkorn0