Should I block google indexing "search.php"
-
My question is I have a search page on our website , you can search by date, number of people staying and so on, I am just wondering should block this in the robots.txt ? Because we have pretty URL'S already for searching by county and searching by towns. I cannot see any benefit of having e.g
"search/search.php?sp_dateFrom=16%2F12%2F2015&sp_dateTo=23%2F12%2F2015&sec_drop%5B%5D=727&spesh_town_id=764&q=&occupants=5&bedrooms=3&submit=SEARCH#search" indexed. Would I be correct in doing this ?
-
Second confirmation ;-), he's right. It's one of the things that for bigger sites really could get you in trouble.
-
That is the answer I was looking for, I was thinking along those lines. The problem was that Seo software was coming up with duplicate content because of different variations of the search parameters. Thanks very much for a clear and precise answer and taking time to explain this.
-
You want to block Google from any URL that produces a search result that is essentially a resorting or refiltering of a master list of search results that they have already crawled/indexed.
If you already have a set of pages that lets Google crawl all the pages in your site (could be all the products in your store, all the articles in your blog, etc), having Google crawl through variants of that same page causes a couple of problems. 1) You are wasting Google's time in spidering pages that it has already seen, vs having Google crawl your more important pages. Depending on how you have these setup, you may end up sending Google into an endless loop of non-important pages to crawl 2) You are creating pages that are generally low quality, have nothing truly original on them, they will not rank for anything anyway and may give the impression that your site consists of primarily low quality pages.
What I show Google is a single simple path to browse my content. For a blog this would be a chronological listing of articles that is paginated so that Google and the user can browse from my most recent to my oldest articles. For an ecommerce page, I might setup basic category pages, make sure the category pages have great content on them and then allow Google to crawl back through all the products based on those main category pages. If I have some products in 2 or 3 categories I do not sweat it. If on either of these examples, I show the user options to resort, filter, etc the results, I block Google with a nofollow or with robots.txt.
In your example, you already have "pretty" URLs by country and town, keep those, that will let Google and your users find your content and also provide context around that content. The crazy a$$ search URL you show is handy for your PHP code to give a search result, but would just waste Google's time. Unless you think it would be useful for a user to save the search URL results, I would see if there is a way to simply hide all the parameters from the user (submit the parameters using a POST vs a GET request for example) so that all they see in the URL result is /search/search.php
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What to do to index all my links of my website?
Ok, i have a new website, with only 14.000 page indexed by google, but the potential is big, 1-2 million pages. What i have to do, to force somehow google to index my website faster? This is my website: https://vmag.ro/
On-Page Optimization | | TeodorMarin0 -
Should you aim for Google to use your meta tags?
When updating meta titles and descriptions, I'm taking note of whether Google is displaying the set tag or changing it to copy from the page. Does this affect the ranking position if Google is having to change the tag? How much should I worry if Google is choosing to change every other page? Thanks!
On-Page Optimization | | Omar_aw0 -
Removing non www and index.php
Hi, I'm green when it comes to altering the htaccess file to remove non www and index.php. I think I've managed to redirect the urls to www however not sure if I've managed to remove the index.php. I'm pasting the contents of the htaccess file here maybe someone can identify if I have unwanted lines of code and if it is up to standard (there are a lot of comments in #) not sure if needed but I've left them as I don't want to screw up anything. Thanks 🙂 @package Joomla @copyright Copyright (C) 2005 - 2016 Open Source Matters. All rights reserved. @license GNU General Public License version 2 or later; see LICENSE.txt READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE! The line 'Options +FollowSymLinks' may cause problems with some server configurations. It is required for the use of mod_rewrite, but it may have already been set by your server administrator in a way that disallows changing it in this .htaccess file. If using it causes your site to produce an error, comment it out (add # to the beginning of the line), reload your site in your browser and test your sef urls. If they work, then it has been set by your server administrator and you do not need to set it here. No directory listings IndexIgnore * Can be commented out if causes errors, see notes above. Options +FollowSymlinks
On-Page Optimization | | KeithBugeja
Options -Indexes Mod_rewrite in use. RewriteEngine On
RewriteCond %{REQUEST_URI} ^/index.php/
RewriteRule ^index.php/(.*) /$1 [R,L] Begin - Rewrite rules to block out some common exploits. If you experience problems on your site then comment out the operations listed below by adding a # to the beginning of the line. This attempts to block the most common type of exploit attempts on Joomla! Block any script trying to base64_encode data within the URL. RewriteCond %{QUERY_STRING} base64_encode[^(]([^)]) [OR] Block any script that includes a0 -
Why are my pages de-indexed?
<form id="form-t3_37nfib9dz" class="usertext" action="http://www.reddit.com/r/SEO/comments/37nfib/why_were_my_pages_deindexed/#"> Hello all, I am very new to SEO. For some reason many of the pages on my site were de-indexed. Specifically the ones linked from this page: However other pages, like the ones linked from this page and this page were not de-indexed. http://www.lawyerconnection.ca/practice-areas/car-accident-injury-lawyers/[1] However the pages linked from this page were not de-indexed: http://www.lawyerconnection.ca/practice-areas/slip-and-fall-lawyers/[2] http://www.lawyerconnection.ca/podcastresources/[3] That first page itself was not de-indexed, just the site that it links to. It just happened today, so maybe I am jumping the gun but I doubt it. When I enter the page into google webmaster tools again and press fetch, one of the child pages, it re-indexes. What could be the problem here? I had someone re-write the content for every city but I have a feeling that there is less differences in the car accidents pages? Is this considered duplicated content do you think? Am I making some other mistake I can't think of? Is it just a one day blip (I doubt it) Let me know, thanks. </form>
On-Page Optimization | | RafeTLouis0 -
Google is NOT showing up the right META DESCRIPTION
Hi, Recently I changed titles and meta descriptions of some pages. The problem is that google have updated the titles but not the meta descriptions.. Ive also checekd the source code of the google cached version of the pages crawled and the meta description reflect the changes i did...but the changes don't appear in google. Do you have some solution/advice for the issue? Tx so Much
On-Page Optimization | | tourtravel0 -
Does Google Bias Against Homepages for Search Queries that are a Question?
If you are trying to rank for a keyword phrase that is a question (e.g. how to ___) does it matter if your page is the home page of your site or an article on your site? I suspect that Google would treat question search queries differently and would show preference for articles over a home page in most cases. An article would be more likely to satisfy the searcher by providing the answer to their question, whereas the home page usually doesn't provide specific answers. I looked at one keyword phrase that is a question and only 1 of the first 20 results was a home page. Any thoughts or experience with this?
On-Page Optimization | | Charlessipe0 -
Incoming Search Terms
Hi guys, I saw a blog post recently where the author added a list of "incoming search queries" to the bottom of his post, obviously to improve the post's ranking for those terms. On one hand, I suppose it it does help users find that post. On the other, it seems lazy and somewhat dodgy, but I haven't found any opinions on it elsewhere and have not seen this practice in my experience. What're your thoughts? Outright search engine manipulation? Cheers, Carlo SCWYt
On-Page Optimization | | mtgconsulting0 -
Getting pages indexed by Google
Hi SEOMoz, I relaunched a site back in February of this year (www.uniquip.com) with about 1 million URL's. Right now I'm seeing that Google is not going past 110k indexed URL's (based on sitemaps). Do you have any tips on what I can do to make the site more likeable by Google and get more indexed URL's? All the the part pages can be browsed to by going to: http://www.uniquip.com/product-line-card/suppliers/sw-a/p-1 I've tried to make the content as unique as possible by adding random testimonials and random "related part numbers" see here: http://www.uniquip.com/id/246172/electronic-components/infineon/microcontrollers-mcu/sabc161pilfca Do I need to wait more time and be more patient with Google? It just seems like I'm only getting a few thousand URL's per day at the most. Would it help me if I implemented a breadcrumb on all part pages? Thanks, -Carlos
On-Page Optimization | | caneja0