Best way to stop pages being indexed and keeping PageRank
-
If for example on a discussion forum, what would be the best way to stop pages such as the posting page (where a user posts a topic or message) from being indexed AND not diluting PageRank too? If we added them to the Disallow on robots.txt, would pagerank still flow through the links to those blocked pages or would it stay concentrated on the linking page? Your ideas and suggestions will be greatly appreciated.
-
Hi Peter,
pages blocked by robots.txt would be considered to be not there, thus not flowing pagerank. You might want to use "noindex, follow" on these pages: pages are crawled and links on the page would be followed, by that any recieved linkjuice would flow from these pages to others. Noindex would mean that these pages wouldn't dilute PR (and ranking).
Furthermore is "noindex,follow" on a page to page basis faster and more secure keeping pages nonindexed than by robots.txt (which is only crawled every 12 hours or so).
You might want to use noindex,follow on all non-important pages such as legal etc.
Sebastian
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New Subdomain & Best Way To Index
We have an ecommerce site, we'll say at https://example.com. We have created a series of brand new landing pages, mainly for PPC and Social at https://sub.example.com, but would also like for these to get indexed. These are built on Unbounce so there is an easy option to simply uncheck the box that says "block page from search engines", however I am trying to speed up this process but also do this the best/correct way. I've read a lot about how we should build landing pages as a sub-directory, but one of the main issues we are dealing with is long page load time on https://example.com, so I wanted a kind of fresh start. I was thinking a potential solution to index these quickly/correctly was to make a redirect such as https://example.com/forward-1 -> https:sub.example.com/forward-1 then submit https://example.com/forward-1 to Search Console but I am not sure if that will even work. Another possible solution was to put some of the subdomain links accessed on the root domain say right on the pages or in the navigation. Also, will I definitely be hurt by 'starting over' with a new website? Even though my MozBar on my subdomain https://sub.example.com has the same domain authority (DA) as the root domain https://example.com? Recommendations and steps to be taken are welcome!
Intermediate & Advanced SEO | | Markbwc0 -
Pillar pages and blog pages
Hello, I was watching this video about pillar pages https://www.youtube.com/watch?v=Db3TpDZf_to and tried to apply it to my self but find it impossible to do (but maybe I am looking at it the wrong way). Let's say I want to rank on "Normandy bike tou"r. I created a pillar page about "Normandy bike tour" what would be the topics of the subpages boosting that pillar page. I know that it should be questions people have but in the tourism industry they don't have any, they just want us to make them dream !! I though about doing more general blog pages about things such as : Places to rent a bike in Normandy or in XYZ city ? ( related to biking) Or the landing sites in Normandy ? (not related to biking) Is it the way to do it, what do you recommend ? Thank you,
Intermediate & Advanced SEO | | seoanalytics0 -
Best way to go about merging 2 sites with significant search volume?
Hi everyone! A client of ours ('Company A') recently acquired another company ('Company B') - both brands carry weight within their industry. Company A's brand name currently registers over 6,500 searches per month, while Company B's brand name draws about 2,500 searches per month. While Company B is smaller, their search volume isn't insignificant. The powers that be plan to discontinue Company B's site at an unspecified date in the future, but it's on the backburner. We'd obviously like to transfer as much of their current ranking as possible, but we also don't want to confuse users. There's additional search volume for term variations such as 'Company B jobs' & 'Company B locations' that we'd like to capture for as long as there's still volume there. Would a microsite with Company B's look & feel (to make it easier to house pages built to capture careers/locations searches) justify its inherent cost, or would it be just as valuable to build a series of landing pages on Company A's site? (Obviously assuming that valid redirects would be in place once Company B's site is taken down.) Thanks in advance!
Intermediate & Advanced SEO | | wilcoxcm0 -
Best way to move a page without 301
I have a page that currently ranks high for its term. That page is going away for the main website users, meaning all internal site links pointing to that page are going away and point to a new page. Normally you would just do a 301 redirect to the new URL however the old URL will still need to remain as a landing page since we send paid media traffic to that URL. My question is what is the best way to deal with that? One thought was set up a canonical tag, however my understanding is that the pages need to be identical or very close to the same and the landing page will be light on content and different from the new main page. Not topically different but not identical copy or design, etc.
Intermediate & Advanced SEO | | IrvCo_Interactive0 -
If I only Link to Page via Sitemap, can it still get indexed?
Hi there! I am creating a ton of content for specific geographies. Is it possible for these pages to get indexed if I only put them in my sitemap and don't link to them through my actual site (though the pages will be live). Thanks!
Intermediate & Advanced SEO | | Travis-W
Travis0 -
What is the best way to handle special characters in URLs
What is the best way to handle special characters? We have some URL's that use special characters and when a sitemap is generate using Xenu it changes the characters to something different. Do we need to have physically change the URL back to display the correct character? Example: URL: http://petstreetmall.com/Feeding-&-Watering/361.html Sitmap Link: http://www.petstreetmall.com/Feeding-%26-Watering/361.html
Intermediate & Advanced SEO | | WebRiverGroup0 -
Indexing specified entry pages
Hi,We are currently working on location based info.Basically, when someone searches from Florida they will get specific Florida results and when they search from California they will specific California results.How does this location based info affect crawling and indexing?Lets say we have location info for googlebot, sometimes they crawl from a New York ip address, sometimes they do it from Texas and sometimes from California. In this case google will index 3 different pages with 3 different prices and a bit different text, and I'm afraid they might see these as some kind of cloaking or suspicious movement because we serve different versions of the page. What's the best way to handle this?
Intermediate & Advanced SEO | | SEODinosaur0 -
How long until Sitemap pages index
I recently submitted an XML sitemap on Webmaster tools: http://www.uncommongoods.com/sitemap.xml Once Webmaster tools downloads it, how long do you typically have to wait until the pages index ?
Intermediate & Advanced SEO | | znotes0