Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?

[Jacksons_Fencing]

I'm really not sure what is best practice for this query?

[Jacksons_Fencing]

Thank you for your answer John!

[johnnybgunn]

I would definitely not block these pages.  You want to block as few pages as possible.

1. These pages can be used to boost internal links by linking to your important pages.

2. Google crawls thousands of pages...it will likely crawl all your important and unimportant files.

3. You can de-prioritize these page in the XML sitemap, telling the spiders that there are more important pages to crawl.

4. If these are similar pages, then use the URL parameter tool in Search Console to indicate a page might be a filtered version of a more important page.

[Alick300]

Hi,

Yes you can block such pages in robots.txt. I would also like to let you know that If you don't want to index some pages you can use .

I would go for in your case.

Hope this helps.

Thanks

[JordanLowry]

Is it possible to beef up those lower quality pages with better content? If they are important main content pages I would imagine you would want to improve those pages.

However, if you were going to block them I would recommend a  tag within the header of those pages.

Hope that helps some.