URL Parameters
-
On our webshop we've added some URL-parameters. We've set URL's like min_price, filter_cat, filter_color etc. on "don't Crawl" in our Google Search console. We see that some parameters have 100.000+ URL's and some have 10.000+
Is it better to add these parameters in the robots.txt file? And if that's better, how can we write it down so the URL's will not be crawled.
Our robotos.txt files shows now:
# Added by SEO Ultimate's Link Mask Generator module User-agent: * Disallow: /go/ # End Link Mask Generator output User-agent: * Disallow: /wp-admin/
-
Hi,
You might want to read this article on faceted navigation on the google webmaster blog which gives some good advice on how to handle the situation. What to use depends a bit on your actual situation.
Options include using a nofollow links / use a separate subdomain or block in robots.txt (using a separate folder).On Moz there is this article (the part of faceting) - its mainly about listing sites - but the core problem is more or less similar.
Hope this helps,
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Language Specific Characters in URLs for
Hi People, would really appreciate your advice as we are debating best practice and advice seems very subjective depending if we are talking to our dev or SEO team. We are developing a website aimed at the South American market with content entirely in Spanish. This is our first international site so our experience is limited. Should we be using Spanish characters (such as www.xyz.com/contáctanos) in URLs or should we use ASCII character replacements? What are the pros and cons for SEO and usability? Would really be great to get advice from the Moz community and make me look good at the same time as it was my suggestion 🙂 Nick
Technical SEO | | nickspiteri0 -
Changing URL of posts
HI, I need to change the urls and permalink structure of my blogposts. How I have to deal all this with google? Do I have to re-submit the pages to google with fetch as google? Will google display duplicate content of the same article ( having changed the url) or will it automatically replace the old url with the new ones? Tx for your support guys!
Technical SEO | | tourtravel0 -
Affiliate Link is Trumping Homepage - URL parameter handling?
An odd and slightly scary thing happened today: we saw an affiliate string version of our homepage ranking number one for our brand, along with the normal full set of site-links. We have done the following: 1. Added this to our robots.txt : User-agent: *
Technical SEO | | LawrenceNeal
Disallow: /*? 2. Reinserted a canonical on the homepage (we had removed this when we implemented hreflang as had read the two interfered with each other. We haven't had canonical for a long time now without issue. Is this anything to do with the algo update perhaps?! The third thing we're reviewing I'm slightly confused about: URL Parameter Handling in GWT. As advised - with regard to affiliate strings - to the question: "Does this parameter change page content seen by the user?" We have NO selected, which means they should be crawling one representative URL. But isn't it the case that we don't want them crawling or indexing ANY affiliate URLs? You can specify Googlebot to not crawl any of particular string, but only if you select: "Yes. The parameter changes the page content." Should they know an affiliate URL from the original and not index them? I read a quote from Matt Cutts which suggested this (along with putting a "nofollow" tag in affiliate links just in case) Any advice in this area would be appreciated. Thanks.0 -
301: Dynamic URL to Static Page
I've been going around trying to get this dynamic url to redirect in the .htaccess file. I know I'm missing something but can't figure it out. Code: RewriteEngine on
Technical SEO | | ohlmanngroup
RewriteCond %{QUERY_STRING} ^/dynamic-url.php?id=43$
RewriteRule ^$ http://static/page/url/inserted/here? [R=301,L] Suggestions?0 -
Compare URLs with 302 redirects
Hello I have a store which was developed in Magento. I have about 8300 errors like this: URL: http://www.theprinterdepo.com/catalog/product_compare/add/product/100/uenc/aHR0cDovL3d3dy50aGVwcmludGVyZGVwby5jb20vcHJpbnRlci1wYXJ0cy5odG1sP3A9NA,,/ 1 Warning 302 (Temporary Redirect) Found 3 days ago <dl> <dt>Redirects to</dt> <dt>http://goo.gl/XMaZg</dt> <dd>Description</dd> <dd>Using a 302 redirect will cause search engine crawlers to treat the redirect as temporary and not pass any link juice (ranking power). We highly recommend that you replace 302 redirects with 301 redirects.</dd> </dl> <a class="more expanded">Minimize</a> These URLs, are generated by magento by the COMPARE feature. In my store we bought an extension called SEO Enterprise Suite and I asked the developers(www.mageworx) about this error. Their answer is: Sorry for the late reply. Our extension adds NOINDEX,FOLLOW tag to compare and cookies pages so that they won't be indexed. I do not think that these redirects can hurt your SEO because these pages won't be indexed at all. The question is: What should I do? Is there anyway that SEOMOZ ignores these URLs? What should I do next, I just dont like to have that HIGH number of errors and warnings. Thank you
Technical SEO | | levalencia10 -
URL parameter reduction plug in
Anyone know of a good plug-in that reduces the amount parameters used in URLs? I need one for an ASP based system and a PHP based system
Technical SEO | | matmox0 -
URL Rewrite
We are trying to convince a client to do a massive rewrite from all URL's looking like this: "www.company.com/category/categoryId=82374" to something like "www.company.com/womens/jackets/rain" How would you describe the importance and impact of doing URL rewrites to an ecommerce site? What evidence/research can we share with them to convince them it is worth the time and effort to do?
Technical SEO | | Hakkasan0