Ignore Urls with pattern.
-
I have 7000 warnings of urls because of a 302 redirect.
http://imageshack.us/photo/my-images/215/44060409.png/
I want to get rid of those, is it possible to get rid of the Urls with robots.txt.
For example that it does not crawl anything that has /product_compare/ in its url?
Thank you
-
in case they do not all start with /category
Disallow: product_compare
-
Then you simply add this to your robots.txt:
Disallow: /catalog/product_compare/
That should leave out all pages starting with:
https://www.theprinterdepo.com/catalog/product_compare/ -
-
Could you perhaps post a URL which has product_compare in it?
You could alter your robots.txt file to disallow robots to index pages in
http//www.domain.com/product_compare/ by adding this line to your robots.txt file: Disallow: /product_compare/
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect_to in URLs?
I've never seen this before. I'm assuming that it's not SEO friendly and that these should be 301s or 302s instead? http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/cars-for-sale-search/results/central/riyadh/ford/explorer/2010/ford-explorer-2010-1038353 http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/account/my-saved-searches/
Technical SEO | | KatherineWatierOng0 -
XML Sitemap and unwanted URL parameters
We currently don't have an XML sitemap for our site. I generated one using Screaming Frog and it looks ok, but it also contains my tracking url parameters (ref=), which I don't want Google to use, as specified in GWT. Cleaning it will require time and effort which I currently don't have. I also think that having one could help us on Bing. So my question is: Is it better to submit a "so-so" sitemap than having none at all, or the risks are just too high? Could you explain what could go wrong? Thanks !
Technical SEO | | jfmonfette0 -
Affiliate urls and duplicate content
Hi, What is the best way to get around having an affiliate program, and the affiliate links on your site showing as duplicate content?
Technical SEO | | Memoz0 -
What is your opinion on ideal url structure?
Which url structure do you think is better... website.com/CO/denver/555-your-address-way-denver-co-55678/98347578 or website.com/classifying-keyword/555_your_address_way-denver_colorado-55678-98347578 orrr website.com/classifying-keyword/555-your-address-way-denver-colorado-55678-98347578
Technical SEO | | jessefriedman0 -
Crawl reveals hundreds of urls with multiple urls in the url string
The latest crawl of my site revealed hundreds of duplicate page content and duplicate page title errors. When I looked it was from a large number of urls with urls appended to them at the end. For example: http://www.test-site.com/page1.html/page14.html or http://www.test-site.com/page4.html/page12.html/page16.html some of them go on for a hundred characters. I am totally stymied, as are the people at my ISP and the person who talked to me on the phone from SEOMoz. Does anyone know what's going on? Thanks So much for any help you can offer! Jean
Technical SEO | | JeanYates0 -
Automatic redirect to external urls
Hi all, I'm developing a dynamic qr code service.. The service works in the following way: You create an account with an associated QR CODE pointing to a url like:
Technical SEO | | raulo79
- http://domain.me/username The user can change the target of this url.. he can:
- point to an external url ( his website for example)
- point to a vCard download page
- a mobile ready webpage ( no redirection in this case)... Visiting http://domain.me/username my company logo is displayed and we redirect the visitor with a: header("Refresh: 5;URL=http://userdomain.tld"); Google is indexing many user's URLs, this is good for those users pointing to the mobile ready webpage, in this case there is no redirection, but Google is indexing many urls that redirect to an external url and I don't know how to avoid this.. I can't do an header('Location: http://www.example.com/'); because I need to display our logo after redirection.. how can I do google friendly? Sorry for my english, I hope you can undestand the problem. Best regards.
Mauro.0 -
Need Help with MAGENTO - URL rewrite
Hello... Hopefully a Magento expert will stumble across this question and help me out. I have noticed that my site is no longer as prominent as it once was for specific product pages... I am looking for help in rewriting the URL's for the product pages. I want it to have xyz.com/product (which exists if you hard code it into the site) If you wind up on the product by clicking throught the categories the url looks like: xyz.com/category/subcategory/product. Does anyone know how to make it so when you land on a product page it is just xyz.com/product ? My Site is : http://goo.gl/JgK1e Thanks for the help...
Technical SEO | | Prime850 -
Rel=Canonical being ignored?
Hi all, We have a toys website that has several categories. It's setup such that each product has a primary category amongst the categories within it can be found. For example... Addendum's primary url is http://www.brightminds.co.uk/childrens-toys/board-games/addendum.htm but it can also be found here http://www.brightminds.co.uk/learning-toys/maths-learning/addendum.htm. Hence, in the for that url it has a rel=canonical that points to the first url. For some reason though seomoz ignores this and reports duplicate page content. It doesn't seem to record the canonical tag either. Any ideas what's going on? Thanks, Josh.
Technical SEO | | joshgeake_gmail.com0