URL Parameters
-
On our webshop we've added some URL-parameters. We've set URL's like min_price, filter_cat, filter_color etc. on "don't Crawl" in our Google Search console. We see that some parameters have 100.000+ URL's and some have 10.000+
Is it better to add these parameters in the robots.txt file? And if that's better, how can we write it down so the URL's will not be crawled.
Our robotos.txt files shows now:
# Added by SEO Ultimate's Link Mask Generator module User-agent: * Disallow: /go/ # End Link Mask Generator output User-agent: * Disallow: /wp-admin/
-
Hi,
You might want to read this article on faceted navigation on the google webmaster blog which gives some good advice on how to handle the situation. What to use depends a bit on your actual situation.
Options include using a nofollow links / use a separate subdomain or block in robots.txt (using a separate folder).On Moz there is this article (the part of faceting) - its mainly about listing sites - but the core problem is more or less similar.
Hope this helps,
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Doubts about the technical URL structure
Hello, first we had this structure Categorie: https://www.stoneart-design.de/armaturen/ Subcategory: https://www.stoneart-design.de/armaturen/waschtischarmaturen/ Oft i see this https://www.xxxxxxxx.de/badewelt/badmoebel/ But i have heard it has something to do with layers so google can index it better, is that true ? "Badewelt" is an extra layer ? So i thought maybe we can better change this to: https://www.stoneart-design.de/badewelt/armaturen/ https://www.stoneart-design.de/badewelt/armaturen/waschtischarmaturen/ and after seeing that i thought we can do it also like this so the keyword is on the left, and make instead "badewelt" just a "c" and put it on the back https://www.stoneart-design.de/armaturen/c/ https://www.stoneart-design.de/armaturen/waschtischarmaturen/c/ I dont understand it anymomre which is the best one, to me its seems to be the last one The reason was about this: this looks to me keyword stuffing: Attached picture Google indexed not the same time the same url, so i thougt with this we can solve it Also we can use only the word "whirlpools" in de main category and the subs only the type without "whirlpools" in text thanks Regards, Marcel SC9vi60
Technical SEO | | HolgerL0 -
URL Parameters as pagination
Hi guys, due to some changes to our category pages our paginated urls will change so they will look like this: ...category/bagger/2?q=Bagger&startDate=26.06.2017&endDate=27.06.2017 You see they include a query parameter as well as a start and end date which will change daily. All URLs with pagination are on noindex/follow. I am worrying that the products which are linked from the category pages will not get crawled well when the URLs on which they are linked from change on a daily basis. Do you have some experience with this? Are there other things we need to worry about with these pagination URLs? cheers
Technical SEO | | JKMarketing0 -
URL Structure for Product Pages
Hi Moz Community. I'm in need of some URL structure advice for product pages. We currently have ~4,000+ products and I'm trying to determine whether I need a new URL structure from the previous site owners. There are two current product URL structures that exist in our website: 1.http://www.example.com/bracelets/gold-bracelets/1-1-10-ct-diamond-tw-slip-on-bangle-14k-pink-gold-gh-i1-i2/ (old URL structure)
Technical SEO | | IceIcebaby
2. http://www.example.com/gemstone-bracelet-prd-bcy-121189/ (new URL structure) The problem is that half of our products are still in the old structure (no one moved them forward), but at the same time I'm not sure if the new structure is optimized as much as possible. Every single gemstone bracelet, or whatever product will have the same url structure, only being unique with the product number at the end. Would it be better to change everything over to more product specific URLS. I.e. example.com/topaz-gemstone-dangle-bracelet. Thanks for your help!
-Reed0 -
Friendly URLS (SEO urls)
Hello, I own a eCommerce site with more than 5k of products, urls of products are : www.site.com/index.php?route=product/product&path=61_87&product_id=266 Im thinking about make it friend to seo site.com/category/product-brand Here is my question,will I lost ranks for make that change? Its very important to me know it Thank you very much!
Technical SEO | | matiw0 -
Exact URL Match For Ranking
Has anyone else run into this issue? I have a competitor that purchases domain names for popular inner pages we are trying to rank for. We are trying to build a brand, our competitors have a lower domain authority but rank higher for inner pages in the serps with VERY little content, backlinks/seo work, they host a single page and do a re-direct to their main site. Would this be a good long term strategy? EX. We sell golf clubs our brand name is golfcity (Ex only) and we carry callaway clubs, our competitor is also building a brand but they purchased callawayclubs.net and do a re-direct. They rank on page one for keywords callaway clubs. If I do try to do this does one have an advantage over another? .com. net .org. because Ive seem them all used and rank on page 1. Thank you!!!
Technical SEO | | TP_Marketing0 -
301 on certain url string
I have a few thousand old urls with the string /content/ in them and are looking for a way to 301 batch redirect them. So for all the urls that contain the word 'content' I would like to redirect to 1 specific page. I have tried the methods below without success. Regular 301's are working fine but this particular method is not working for me. I am running a Joomla site but I don't imagine that would have any impact. Any suggestions would be greatly appreciated. Redirect 301 ^content/.*$ http://www.mysite.com Redirect 301 ^content/ http://www.mysite.com
Technical SEO | | omega0 -
How much of a hit to changing urls?
Hi, We have a few pages that from an SEO perspective have poor URLs. We are planning on changing them (and 301 redirecting) the old page to the new page. I heard in the past, this can temporarily negatively impact your SERP rank etc. Since the old URLs are bad, even if there is a temporary negative hit, changing them in the long run it is better, but curious if anyone has any experience on what to expect.
Technical SEO | | NicB10 -
Canonical URL
In our campaign, I see this notices Tag value
Technical SEO | | shebinhassan
florahospitality.com/ar/careers.aspx Description
Using rel=canonical suggests to search engines which URL should be seen as canonical. What does it mean? Because If I try to view the source code of our site, it clearly gives me the canonical url.0