When you add 10.000 pages that have no real intention to rank in the SERP, should you: "follow,noindex" or disallow the whole directory through robots? What is your opinion?
-
I just want a second opinion
The customer don't want to loose any internal linkvalue by vaporizing link value though a big amount of internal links. What would  you do?
-
Hi Jeff,
Thanks for your answer. Please take a look to the reply above on Fredrico
-
Hi Federico,
In this case it's an affiliate website and the 10.000 pages are all prodcutpages. It's all coming from datafeeds so it's duplicate content.
We don't want to index this that's for sure.
So noindex,follow or disallow the whole directory or both...
We have our own opinion about this but I want to hear what others are thinking about this
Thanks in advanced!
-
Yep, I agree with belt and suspenders.
-
Wesley - I do agree with Federico.
That said, if they really don't want those pages indexed, use the belt-and-suspender method (if you wear both a belt and suspenders, chances are greater that your pants won't fall down).
I'd put a robot.txt file to disallow the indexing of the directory, and also no-index / no-follow each of the pages, too.
That way when they have someone working on the pages in the site and they change things to followed, you're still covered. Â Likewise, if someone blows away the robot.txt file.
Just my $0.02, but hope it helps…
-- Jeff -
What do they have? 10,000 pages of uninteresting content? a robots tag noindex,follow will do to leave them our of engines. But to decide you really need to know what those pages have. 10,000 isn't a few, and if there's value content worth sharing, a page could get a link, that if you disallow it through the robots, won't even flow pagerank.
It all comes down to what are those pages for...?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Robots.txt was set to disallow for 14 days
We updated our website and accidentally overwrote our robots file with a version that prevented crawling ( "Disallow: /") We realized the issue 14 days later and replaced after our organic visits began to drop significantly and we quickly replace the robots file with the correct version to begin crawling again. With the impact to our organic visits, we have a few and any help would be greatly appreciated - Will the site get back to its original status/ranking ? If so .. how long would that take? Is there anything we can do to speed up the process ? Thanks
Intermediate & Advanced SEO | | jc42540 -
"WWW" versus non "WWW" on domain
We plan on migrating our site to a new shorter domain name. I like the idea of removing "www" to gain an additional 3 letters in the URL display. Is there any disadvantage of doing so from a technical or SEO perspective? Thanks,Â
Intermediate & Advanced SEO | | Kingalan1
Alan0 -
Confused with rankings for keywords that are not on the actual page?
Hey Guys, So I went to search a keyword, my competitor came up, when I looked for that keyword on their page i didn't see it at all, and the page is content thin, how is it they are ranking for the keyword at all? Bit confused at how this works. Â So having the keyword in your content is not necessary a ranking factor? Please advise and sorry if this question is really amateur, I'm sure based on this search there is more to it then that.... could you other ranking factors as to why they rank for a keyword that is not in their content? Does the whole exact keyword have to be in the content to rank? Thanks all
Intermediate & Advanced SEO | | edward-may0 -
Homepage not ranking for main keyword, all other pages ranking slightly for their own keyword phrases.
Dear Mozzers, We have an ecommerce website (www.pashmina-boutique.com), we want to rank the homepage for the keyword "pashmina". Problem is, we are nowhere to be found, not even in the top 100 search results of Google. It is indexed, we have no crawl errors, except that we had some problems with our hosting (503's), the crawler of Google bumped in a few of those. And we are fixing it. Our other pages, e.g. (http://www.pashmina-boutique.com/15-white-pashminas) ranks for "white pashminas" 24th and "white cashmere pashminas" 23rd. We have done no linkbuilding, in Majestic, you can check that. We have been offline for a long time (over 6 months) and about 3 weeks ago, we did a redesign (SEO). We relaunched, everything is looking fine except the homepage isn't ranking for the main keyword. Could you guys check it out? Is it over-optimization? It can't be Penguin, Panda would be a surprise too. Or do we still have to wait for the monthly Panda data-refresh? We are currently busy with this issue, once this is solved - that the homepage is ranking - we will start linkbuilding. Mr.1000
Intermediate & Advanced SEO | | Mr.10000 -
How much does "overall site semantic theme" influence rankings?
OK. I've optimized sites before that are dedicated to 1, 2 or 3 products and or services. These sites inherently talk about one main thing - so the semantics of the content across the whole site reflect this. I get these ranked well on a local level. Now, take  an e-commerce site - which I am working on - 2000 products, all of which are quite varied - cookware, diningware, art, decor, outdoor, appliances... there is a lot of different semantics throughout the site's different pages. Does this influence the ranking possibilities? Your opinion and time is appreciated. Thanks in advance.
Intermediate & Advanced SEO | | bjs20100 -
Redirect or Rewrite? 2 pages ranking
We have two pages ranking for "Custom Web Design" http://www.imageworksstudio.com/custom-web-design ranks higher (consistently 9-13) and http://www.imageworksstudio.com/content/custom-web-design ranks around 35-40 the latter is actually an older version of the article that never was replaced or taken down - but it has the majority of the links to it Wondering if we should update the old content so it is not similar to the one that is ranking better or if we should redirect everything to /custom-web-design to see if it can secure better rankings?
Intermediate & Advanced SEO | | imageworks-2612900 -
Optimize root domain or a page in a sub directory?
Hi My root domain is already optimized for keywords, i would say branded keywords, which i do not really need, as the traffic from these does not give me any revenue ( mostly consists of our employees/returning visitors). Now i have run on page optimization for set of keywords for root domain which i like and got good grades (hurray!). But yet my website does not show up on search engines for those keywords. I have got pretty good link building done to my root domain but this is not done for all keywords (but done for branded keywords). It just happened, please do not ask why. So i decided to optimize inside pages in sub directory with new set of keywords i like. Starting with link building, giving anchor text on various other website linking to this particular page. These pages are not ranked in top 50 in google. Is that a good practice? or I would not need those branded keywords, hence should I re-optimize my root domain to suite my new keywords by giving less preference to branded keywords? Is this a good practice?
Intermediate & Advanced SEO | | MiddleEastSeo0 -
Why are new pages not being indexed, and old pages (now in robots.txt) remain in the index?
I currently have a site that was recently restructured, causing much of its content to be reposted, creating new URL's for each page. To avoid duplicates, all of the existing pages were added to the robots file. That said, it has now been over a week - I know Google has recrawled the site - and when I search for term X, it is stil the old page that is ranking, with the new one nowhere to be seen. I'm assuming it's a cached version, but why are so many of the old pages still appearing in the index? Furthermore, all "tags" pages (it's a Q&A site, like this one) were also added to the robots a few months ago, yet I think they are all still appearing in the index. Anyone got any ideas about why this is happening, and how I can get my new pages indexed?
Intermediate & Advanced SEO | | corp08030