How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Phantom search results showing up in analytics
Hi, I'm getting these weird phantom search results showing up in my analytics account which is skewing my results. I have a wordpress site, and I do have some custom search results pages, but analytics is telling me these hits are coming from google- which I doubt. The end of the query string (&cat=plus-5-results) is not coming from a link created within my site. Anyone have any ideas? Failing that, would somebody be kind enough to explain how I can ignore these hits, but retain other, valid search result requests? Thanks reading! Will Screenshot%202019-05-16%20at%2022.32.10.jpg
Reporting & Analytics | | madegood0 -
Google Search Console
To the Moz Community, Should we be considering the information that Google Search Console is telling us? It is showing a dramatic drop in our SEO and our pages are not being indexed, however it is showing differently in our Moz Analytics section. Any clarification will be greatly appreciated. Many thanks Dawn
Reporting & Analytics | | DawnQ0 -
Do modal pop-ups impact the Google Analytics of the host page?
We have an instance of a page where visitors can click a button to start an interactive quiz. The quiz pops up in a modal window that references another domain (the interactive content provider). Will the person completing the quiz in the modal pop-up still be counted as an active visitor on the original host page during the time they are completing the quiz?
Reporting & Analytics | | MuhammadInc0 -
How can I easily combine moz page difficulty, google search volume and SERPS position?
I want to produce an excel spreadsheet that I can use to identify the best use of my content Writing time. So Looking at a keyword list I want Current SERPS to show me where I am now? moz page difficulty score to show how hard I'll have to work google traffic estimate so that I can see the potential payoff. I can can generate all these separately but combining them is a huge time waster as invariably the results don't come back in quiet the same order and a line by line check is required. Part of the reason for doing this is keyword exploration so that we can find new niches by generating hundreds or thousands of keywords to test.
Reporting & Analytics | | Zippy-Bungle0 -
Analytics - 'other'
When i look in GA under Channels (under Acquisition) 'other' is listed What is 'other' ? I have been told its other unidentified channels as they did not allow 3rd party cookies or surfers were in anonymous/private mode. Other is usually organic traffic that couldn't be identified for the aformentioned reasons. This data is encrypted and available but it violates Google guidelines as they are not allowed to pass personal info//data to third parties so it is automatically filtered. But they are not 'Not Provided' (since that still shows under organic) but is usually/mainly some form of organic visits. Hence Seo can take credit for much of that traffic, is this correct ? Many Thanks
Reporting & Analytics | | Dan-Lawrence0 -
Google Webmasters DNS error
Hi, In my webmaster tools I have a yellow triangle stating that there is a DNS error that is preventing Google crawling my sites. The site is indexed and I have checked fetch as Google and that seems ok but the triangle is still there every time I check it. The whois sites all have the correct information and point to Hostgator who I am using. I have contacted them and they said everything seems ok. Should I just carry on as normal with my link building as the site is indexed or investigate even further? Cheers, Stuart
Reporting & Analytics | | stuart420 -
Google URL Builder Extension showing up as indexed pages.
Hello, I was reviewing my PRO member campaign report. I see that I am getting warnings for too long of URLs. However, these URLs are my website URL with the Google URL builder tracking code that I set up for my marketing campaings. Why are these being indexed? For example: www.website.com/?utm_source=Oct+Newsletter&utm_medium=e.... Thank you, Kristen
Reporting & Analytics | | KLFeichtner0 -
Google +1 and ranking effect .
Anyone have any experience as to what effect google +1 has on ranking? Trying to figure out what impact it might have. Ecommerce site in norway selling childrens clothing. Atm we are ranking 25th for main targetted word childrens clothing (barneklær in norwegian). on google.no this has a difficulty of 34. None of main competitors have any amount of google +1's on their site. Lets say if I in some way could manage to get 100 ppl to push +1 button. Would this affect our ranking at all ? Might be impossible to answer but if anyone has thoughts about it I am interested. Thanks
Reporting & Analytics | | danlae0