Max Amout Of HTML Pages In A Folder
-
What's the maximum amount of html pages that one should put in a folder, to get the best SEO GoggleBot crawl? I'm aware that there's a limit of 10,000 on most servers, but was curious to know if a lesser amount of pages would be better, for crawling and indexing purposes. Also curious on peoples opinions on whether .jpg and .gif files should follow similiar rules.
-
Thanks for all the input. Google does seem to crawl everything these days, so I'm also in conclusion if the files fit, they'll get crawled. Sitemaps, internal links and optimized images are all a must.
-
For images, you want to make sure they're optimized for the web: small file sizes for easy download, but still a resolution that shows the image clearly. Your graphic designer and a good graphic design program (Photoshop, Gimp, etc.) should help with this.
-
Hi,
As Ray-pp said, there isn't an optimal number of pages that are going to serve you better.
However, if you want to help Google discover more about your site and pages of importances, look to create a good internal linking strategy. This doesn't mean that you should just add footer or sidebar links though - these are context links that talk about a different subject, along with a link to the appropriate page.
If you get this right, you can gain a lot in terms of Google understanding more about what you have to offer, and the links to primary pages can also lead to an increase in the SERPs for various phrases.
-Andy
-
AFAIK there is no efficient # of files to include in a folder directory for maximum crawl effectiveness. If you folder legitimately warrants 5k html pages in a directory, then Google will crawl all the pages. Make sure to create value-added pages with high quality content - Google will recognize them and crawl them as appropriate.
If you have the options, use your Google Webmaster Tools account to adjust crawl settings. Once your site is a specific size, Google will take-over crawl rate settings for you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Would My Page Have a Higher PA and DA, Links & On-Page Grade & Still Not Rank?
The Search Term is "Alcohol Ink" and our client has a better page authority, domain authority, links to the page, and on-page grade than those in the SERP for spaces 5-10 and we're not even ranked in the top 51+ according to Moz's tracker. The only difference I can see is that our URL doesn't use the exact text like some of the 5-10 do. However, regardless of this, our on-page grade is significantly higher than the rest of them. The one thing I found was that there were two links to the page (that we never asked for) that had a spam score in the low 20's and another in the low 30's. Does anyone have any recommendations on how to maybe get around this? Certainly, a content campaign and linking campaign around this could also help but I'm kind of scratching my head. The client is reputable, with a solid domain age and well recognized in the space so it's not like it's a noob trying to get in out of nowhere.
Intermediate & Advanced SEO | | Omnisye0 -
Home page vs inner page?
do you believe that the advantage of targeting a search term on the home page is now worse off than before? as I understand it ctr is a big factor now And as far as i can see if two pages are equal on page etc the better ctr will win out, the issue with the home page is the serp stars cannot be used hence the ctr on a product page will be higher? I feel if you where able to get a home page up quicker (1 year instead of two) you still lost out in the end due to the product page winning on ctr? do you think this is correct?
Intermediate & Advanced SEO | | BobAnderson0 -
Why is page still indexing?
Hi all, I have a few pages that - despite having a robots meta tag and no follow, no index, they are showing up in Google SERPs. In troubleshooting this with my team, it was brought up that another page could be linking to these pages and causing this. Is that plausible? How could I confirm that? Thanks,
Intermediate & Advanced SEO | | SSFCU
Sarah0 -
Please help with page
We used to use this page http://www.discountbannerprinting.co.uk/banners/vinyl-pvc-banners.html to rank for the words vinyl banner and PVC banner but we have tried to focus the page only on PVC banners and move the vinyl banners word to http://www.discountbannerprinting.co.uk/ yet for some reason even though they have both been spidered google has now chosen to rank this page http://www.discountbannerprinting.co.uk/stickers/vinyl-stickers.html for the vinyl banner words- how do I stop this from happening I thought the home page would be powerful enough to rank for the word with a title inclusion and a spread of the word on the page. Also if anyone can give their opinion on why they thinkhttp://www.discountbannerprinting.co.uk/banners/vinyl-pvc-banners.html does not rank very well I would be truly appreciative.
Intermediate & Advanced SEO | | BobAnderson0 -
Can too many "noindex" pages compared to "index" pages be a problem?
Hello, I have a question for you: our website virtualsheetmusic.com includes thousands of product pages, and due to Panda penalties in the past, we have no-indexed most of the product pages hoping in a sort of recovery (not yet seen though!). So, currently we have about 4,000 "index" page compared to about 80,000 "noindex" pages. Now, we plan to add additional 100,000 new product pages from a new publisher to offer our customers more music choice, and these new pages will still be marked as "noindex, follow". At the end of the integration process, we will end up having something like 180,000 "noindex, follow" pages compared to about 4,000 "index, follow" pages. Here is my question: can this huge discrepancy between 180,000 "noindex" pages and 4,000 "index" pages be a problem? Can this kind of scenario have or cause any negative effect on our current natural SEs profile? or is this something that doesn't actually matter? Any thoughts on this issue are very welcome. Thank you! Fabrizio
Intermediate & Advanced SEO | | fablau0 -
On-page optimization - Am I doing it well?
Hi Mozzers, I'm sitting here going through our site and optimizing all of our content.
Intermediate & Advanced SEO | | Travis-W
For the most part we've just written without correct keyword research, so the content lacks focus. Here is a page I would consider finished - http://www.consumerbase.com/international-mailing-lists.html I have our KWs in the: URL Title Tag Meta Description Bolded in Content Image Alt Attribute. If I optimize my other pages like this, will I be good?
It feels a tiny bit stuffed to me, but SEOmoz's on-page tool gives me glowing numbers. Thanks!0 -
What constitutes a duplicate page?
Hi, I have a question about duplicate page content and wondered if someone is able to shed some light on what actually constitutes a "duplicate". We publish hundreds of bus timetable pages that have similar, but technically with unique urls and content. For example http://www.intercity.co.nz/travel-info/timetable/lookup/akl The template of the page is oblivious duplicated, but the vast majority of the content is unique to each page, with data being refreshed each night. Our crawl shows these as duplicate page errors, but is this just a generalisation because the urls are very similar? (only the last three characters change for each page - in this case /akl) Thanks in advance.
Intermediate & Advanced SEO | | BusBoyNZ0 -
301 redirect every pages?
Good evening, my question might sound stupid but please forgive me, I am still learning SEO. If I build a new site that will replace an existing site. Is there any point to do a 301 redirect for pages that had no inbound link so, no juice to pass? I kind of think that it would be a better practice to 301 redirect each pages to a page that make sense on the new web site .... but here is why I think that. Why I say that If I am lucky, many of my old web site pages will be indexed, many of them having no inbound links. So once the new web site online, until all my new web sites pages are indexed, I could imagine Google would send people to the index pages (the old ones that do not exist anymore)... I am right? So in that case, if I do a 301 redirect only for pages that have inbound links, the user would end up on a 404 page. Could you tell me if it make sense how I think? Thanks a lot !! Nancy P.S. I would not redirect if it make no sense to the user. I fully understand that we must always keep the user experience in mind in any 404 and 301 redirect decisions. But to simplify the question, just suppose it is ok from a user perspective to map every old site pages to a page in new web site.
Intermediate & Advanced SEO | | EnigmaSolution0
