How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What does 'Safari (in-app)' mean in Google Analytics browser traffic?
Hi, can anyone explain what 'Safari (in-app)' refers to in my browser sources? Also, it has a very high bounce rate - any ideas why?
Reporting & Analytics | | b4cab1 -
Variables in Google Tag Manager
Last week I got a great answer here on how to implement GTM and cross domain tracking, now that we have that in place I'm looking for some more advice 😉 I'm wondering that if we push variabels into the data-layer of GTM, can we actually use those variables in reports as well? Do they get recorded in Google Analytics? I'd like to use some data that we push to the data layer for segmentation purposes. Anybody an idea how to achieve this?
Reporting & Analytics | | jorisbrabants0 -
Google Search Console - Why is my average mobile position better than my average desktop position?
I'm wondering why my average mobile position is much better than my average desktop position. I'm wondering if Google is comparing the same queries for both mobile vs desktop or if they're only showing me the top ranked for each type of search. Is it example 1 or 2? Example 1: Desktop may have 5,000 ranking queries that average to 21.6
Reporting & Analytics | | Pauly_Gigs
Mobile may have 1,500 ranking queries that average to 8.5 OR Example 2: example.com has 5,000 total ranking keywords, those queries' average ranking in a desktop search 21.6 and mobile search 8.5. I'm curious to know exactly what I'm seeing in Google's Search Console. https://08875344305734164866.googlegroups.com/attach/777ae98664ed418f/Mobile%20VS%20Desktop.png?part=0.1&view=1&vt=ANaJVrEHOjRLlPH43i00NnC8PxaG3ct7bsHum_TWnUoa7xVamCpRp8jrvRQJL-gz4n7Q0otqKcKxcAJA5z1VySs2naQU_Zy5tDps6bJhUSZsLRQq4uU-tJQ0 -
In-Page Analytics
Hey folks, In the in-page analytics section of GA I am wanting to get the % of clicks on a particular link over a large selection of dates. Does anyone know how to pull the data without having to do it day by day? Ideally I want to pull it all at once. There might of course be other ways to do this, if so please let me know. Thanks for your help. Cheers,
Reporting & Analytics | | CraigAddyman0 -
Google Analytics censoring keywords?
The top keyword on our real time GA is showing as a blank. There's a topic of a sexual nature in our talk threads trending on Twitter right now which is driving a lot of traffic and I'm wondering whether Google have censored the keyword. The next few keywords are our name and variants of. Could there be some other reason why this word isn't showing up? Has anyone else experienced this?
Reporting & Analytics | | CecilyP0 -
Google URL Builder Extension showing up as indexed pages.
Hello, I was reviewing my PRO member campaign report. I see that I am getting warnings for too long of URLs. However, these URLs are my website URL with the Google URL builder tracking code that I set up for my marketing campaings. Why are these being indexed? For example: www.website.com/?utm_source=Oct+Newsletter&utm_medium=e.... Thank you, Kristen
Reporting & Analytics | | KLFeichtner0 -
Get Stats on Search Box Usage, Using Google Analytics?
Let me preface this by first saying I am NOT using the Google Adsense Search feature. The question I am asking is pertaining to the standard search box that comes already coded in most .php blogs. In the Google Analytics, is there a way to tell how many people are searching using the search box? I've looked through the navigation and pathway options in analytics and I don't seem to see anything that definitively points to the number of people using the search box but, perhaps, I am overlooking it. Any insight?
Reporting & Analytics | | ChristineCadena0