Best way to stop pages being indexed and keeping PageRank
-
If for example on a discussion forum, what would be the best way to stop pages such as the posting page (where a user posts a topic or message) from being indexed AND not diluting PageRank too? If we added them to the Disallow on robots.txt, would pagerank still flow through the links to those blocked pages or would it stay concentrated on the linking page? Your ideas and suggestions will be greatly appreciated.
-
Hi Peter,
pages blocked by robots.txt would be considered to be not there, thus not flowing pagerank. You might want to use "noindex, follow" on these pages: pages are crawled and links on the page would be followed, by that any recieved linkjuice would flow from these pages to others. Noindex would mean that these pages wouldn't dilute PR (and ranking).
Furthermore is "noindex,follow" on a page to page basis faster and more secure keeping pages nonindexed than by robots.txt (which is only crawled every 12 hours or so).
You might want to use noindex,follow on all non-important pages such as legal etc.
Sebastian
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Search Console - Indexed Pages
I am performing a site audit and looking at the "Index Status Report" in GSC. This shows a total of 17 URLs have been indexed. However when I look at the Sitemap report in GSC it shows 9,000 pages indexed. Also, when I perform a site: search on Google I get 24,000 results. Can anyone help me to explain these anomalies?
Intermediate & Advanced SEO | | richdan0 -
Is it best to condense 2 similar category pages?
After reading Rand's great article about building Seo focused pages to serve topics, not keywords (http://moz.com/blog/topics-people-over-keywords-rankings-whiteboard-friday ) I started looking at my site. Question: I have 2 very similar category pages, orginally built to go after similar but different keyword terms. For example, one is: domain.com/blue-rings.html and the other is domain.com/blue-bands.com. (bridal jewelry) "Blue" is just a hypothetical type. At one time I could rank for "blue-rings" and "blue-bands". But with google changes, I think it's better to focus on a general term, right? Not set up similar pages, with same product, for very similar keywords. I'm thinking that having these 2 pages could be actually hurting, as they are competing with each other. Any recommendations? Thanks folks! Ron
Intermediate & Advanced SEO | | yatesandcojewelers0 -
Best way for Google and Bing not to crawl my /en default english pages
Hi Guys, I just transferred my old site to a new one and now have sub folder TLD's. My default pages from the front end and sitemap don't show /en after www.mysite.com. The only translation i have is in spanish where Google will crawl www.mysite.com/es (spanish). 1. On the SERPS of Google and Bing, every url that is crawled, shows the extra "/en" in my TLD. I find that very weird considering there is no physical /en in my urls. When i select the link it automatically redirects to it's default and natural page (no /en). All canonical tags do not show /en either, ONLY the SERPS. Should robots.txt be updated to "disallow /en"? 2. While i did a site transfer, we have altered some of the category url's in our domain. So we've had a lot of 301 redirects, but while searching specific keywords in the SERPS, the #1 ranked url shows up as our old url that redirects to a 404 page, and our newly created url shows up as #2 that goes to the correct page. Is there anyway to tell Google to stop showing our old url's in the SERP's? And would the "Fetch as Google" option in GWT be a great option to upload all of my url's so Google bots can crawl the right pages only? Direct Message me if you want real examples. THank you so much!
Intermediate & Advanced SEO | | Shawn1240 -
What's the best way to redirect categories & paginated pages on a blog?
I'm currently re-doing my blog and have a few categories that I'm getting rid of for housecleaning purposes and crawl efficiency. Each of these categories has many pages (some have hundreds). The new blog will also not have new relevant categories to redirect them to (1 or 2 may work). So what is the best place to properly redirect these pages to? And how do I handle the paginated URLs? The only logical place I can think of would be to redirect them to the homepage of the blog, but since there are so many pages, I don't know if that's the best idea. Does anybody have any thoughts?
Intermediate & Advanced SEO | | kking41200 -
Best way to preserve site authority / juice when moving a property to Facebook?
Hi, so, I have a website. Let's call it a cooking website with about 300 pieces of content cross-listed among 20 categories. I want to move my entire site, hook line and sinker, to Facebook. My first thought was to do this with a domain-wide 301, as that would preserve most of the authority and juice my site has built over the years... but would this have a corollary effect of unfocusing my keyword strategy? E.g. is there a risk in doing a sitewide 301 to a single landing page, in that some of the juice I'd be passing to my new home page would be from, say, "recipes for jelly donuts?" Has anyone had an experience making a large product transition like this, and are there any current best practices? Thanks!
Intermediate & Advanced SEO | | Kenn_Gold1 -
Any way to make my page on top of results again?
Hello Everyone, For a long time now I've had many of my website pages ranked on first page of results for some popular searches. On friday I noticed the traffic coming from google dropped dramatically and I realized that I was not on first page anymore. My SEO is focused for Google Spain. Here are some examples: http://msn-messenger.hispazone.com/ This page was on position 9 (first page) if we perform the search "Descargar msn" on Google. Now it's on page 3. http://http://adobe-acrobat-reader.hispazone.com/ This was position 5 (first page) for the search "descargar adobe". Now it's also page 3. Any suggestions? Best Regards.
Intermediate & Advanced SEO | | HispaZone0 -
Best Product URL For Indexing
My proposed URL: mydomain.com/products/category/subcategory/product detail Puts my products 4 levels deep. Is this too deep to get my products indexed?
Intermediate & Advanced SEO | | waynekolenchuk0 -
Best way to host multiple sites for maximum seo
We have over 100 websites we built for clients that we currently host on 1 shared godaddy hosting account. They each have a link to us but since they are all under one shared account, we feel that we are not maximizing the inbound link potential. I've looked into c class hosting but found that either the ip's were flagged as spam, or they shared nameservers which defeats the purpose. I've also been told that since the c class ip's a hosting company gives to you are all owned by them, that also defeats the purpose. Anyone have any solutions besides opening 130 accounts with different hosting companies? Also, will it make any difference changing existing sites onto different hosts now or are they already tainted?
Intermediate & Advanced SEO | | seopet0