How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why few pages have more than 100% bounce rate?
Hello All, For my ecommerce site approx more than 30k products I have. In Google Analytic approx daily for few products approx 10-15 products bounce rate show 300%, 200%, 150%, 140%, 125% how? and what is the solution? Product page daily change. Thanks!
Reporting & Analytics | | Johny123450 -
In Google Search Console: Total Clicks differ from the sum clicks on search queries
Hi.This may be a google technical question, but I've searched a lot and I couldn't find any certain information about that.The problem appears when you look at two stats in Search Console ( WebMasters Tools, some months ago), particulary in the Search Analysis.1- Total clicks2- Sum of clicks at every search query.I attached an image to make me clear.8IZsxs6.png
Reporting & Analytics | | NachoRetta2 -
Why only a few pages of my website are being indexed by google
Our website www.navisyachts.com has in its sitemap over 3000 pages of information, and this is all unique content written by our team. Now Google Webmaster central shows only 100 urls indexed from 3500 submitted. Can you help me understand why and how I can fix this issue? The website has 4 years old, is a Joomla 3.3 up to date. It has part of the content in the Joomla core content systems and part in K2. Thank you. Pablo
Reporting & Analytics | | FWC_SEO0 -
Google Analytics - Average Position
Hi Just trying to get some clarity on Google Analytics Average Positions in "Aquistions/Search Engine Optimisation". For a very competitive keyword Google Analytics is saying i am on average position of 6. Is this Page 6? I am assuming position six would be 1.6?
Reporting & Analytics | | Cocoonfxmedia0 -
What is the difference between "Organic Traffic" and the "Non-Paid Search Traffic" default segment in Google Analytics?
These two filtering options ("organic traffic" in the left sidebar and "non-paid search traffic" in the advanced segments) give me slightly different numbers. Any idea why this would be the case?
Reporting & Analytics | | FPD_NYC1 -
Changed URL's, traffic dropped from 2k week to 1K week. Need advice!
Hi Mozers, I recently changed my URLs for my ecommerce site and my traffic went from 2,000 visitors a week to 1,000 visitors a week, over a 3 week period. Traffic is down, so are unique Kwds. I need advice on why this happened and what I should do moving forward. To brief, I have a ecommerce website, www.ecustomfinishes.com. I noticed pattern that a lot of my URLs with a unique URL structure (URL.Com/ProductDescription/ProductName) were getting a lot of entrances ~30-50 a month, and others that followed the path of my subcategory (URL.com/SubCat/Product) were getting 0-3 entrances a month. The seo pattern was that those with unique product URLs were hitting long tail Kwds, and those URLs with /subcategory/product were getting far less traffic. I changed 150 or so urls to be unique. Good idea, I thought. Since then: CON: Since then my traffic dropped from 2200 visitors a week to 1100 visitors a week. -25% week to week, over 3 weeks CON: # of non-paid keywords sending visits: -25% week to week, over 3 weeks PRO: my Urls receiving entrances +10% week to week, over 3 weeks REF: http://imgur.com/GwZT8 Question: What are your best suggestions moving forward? Any advice is much appreciated, Thank you!!! abBN3
Reporting & Analytics | | longdenc_gmail.com0 -
Too Many On-Page Links
I'm getting the warning of "Too many on-page links". I have a number of affiliate marketing sites, all Wordpress, all with sidebars. In the header navigation bar is a link that offers "reviews" and links to each product/brand. This is not a drop down link but leads to a page with product links. Also, in the sidebar I have links that also lead to the products/brands. Redundant yes, but in the beginning this seemed to be a good practice in site design. It would be easy to simply remove the widget that contains the link process in the sidebar. The problem may simply be that the sidebar is the same for every page in the site(s). Is this hurting total SEO as my sites are all over 95-percent indexed. One site in particular does very well in traffic and sales so is removing these links potentially going to improve my SEO and ultimately by success? Thanks, Don
Reporting & Analytics | | JavaManOne0