How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How effective are 301 redirects in passing page rank?
I have a blog which is ranking well for certain terms, and would like to repurpose it to better explain these terms it is ranking for, including updating the url to the new term the blog will be about. The plan being to 301 redirect the old url to new. In the past, I've done this with other pages, and have actually lost much of the rankings that I had earned on the original URL. What is your take on this? Maybe repurpose blog, but maintain original URL just to be on the safe side? Thanks
Technical SEO | | CitimarineMoz0 -
Is there anyway to automatically find complete urls for 301 redirects?
Hello all, I'm working on a large site that is being prepared for a migration. After scraping on screaming frog, WMT, and deepcrawl, and deduping I have a list of 404 pages that need redirected. Many of these URLs are just incomplete URLs. For example the incomplete URL would be: "https://moz.com/beginners-guide-to" when it should be "https://moz.com/beginners-guide-to-seo" I have been finding the correct page manually searching google. With hundreds more to go, I am looking for a way the would be a little less laborious. Thanks! https://moz.com/beginners-guide-to-seo
Technical SEO | | AdamBohr0 -
301 redirect file question
Hi Everyone, I am creating a list of 301 redirects to give to a developer to put into Magento. I used Screaming Frog to crawl the site, but I have noticed that all of their urls 302 to another page. I am wondering if I should 301 the first URL to the url on the new site, or the second. I am thinking the first, but would love some confirmation. Thank you!
Technical SEO | | mrbobland0 -
Should I consolidate multiple domains to a single site with 301 redirects?
Our client wants all his sites to be re-designed and perhaps consolidated into one domain. What are the dangers of using 301s on all his already ranking and established domains to their new forward-slash location? If there are some good articles that describe this exact issue please post a link.
Technical SEO | | dsmdesign0 -
How does this rank? - a page that is 301 redirected
How does a 301ed page rank in google? In google I searched for" ikea.ca" which is set up as a 301 redirect to www.ikea.com/ca/en and was surprised to see the url --> www.ikea.ca actually ranking. IKEA Canada <cite>ikea.ca/</cite>IKEA Featuring Scandinavian modern style furniture and accessories. Include storage options, lighting, decor products, kitchen appliances and beds. Bedroom - Kitchen - Living Room - IKEA North York
Technical SEO | | Morris770 -
Need Help writing 301 redirects in .htaccess file
SEOmoz tool shows me 2 errors for duplicate content pages (www.abc.com and www.abc.com/index.html). I believe, the solution to this is writing 301 redirects I need two 301 redirects 1. abc.com to www.abc.com 2. /index.html to / (which is www.abc.com/index.html to www.abc.com) The code that I currently have is ................................................... RewriteEngine On
Technical SEO | | WebsiteEditor
RewriteCond %{HTTP_HOST} ^abc.com
RewriteRule (.*) http://www.abc.com/$1 [R=301,L] Redirect 301 http://www.abc.com/index.html http://www.abc.com ...................................................... but this does not redirect /index.html to abc.com. What is wrong here? Please help.0 -
301 Redirect & Linking Root Domain Count
Howdy Mozzers! If I do a 301 redirect from a domain that say has 200 linking root domains, to a fresh domain will the fresh domain now (when updated) have a linking root domain count of 200? Also, is it beneficial or detrimental to do a 301 redirect for an unrelated website i.e. garden hose website, to a children's playgorund equipment website in order to capture the link juice? Best
Technical SEO | | clickfactory0 -
Product ratings causing 302 redirect problem
I am working on an ecommerce site and my crawl report came back with 7000+ 302 redirects and maxed out at 10,000 pages because of all the redirects. The site really only has maybe 1500 pages (dynamic content aside). After looking into it a little more I see it is because of the product rating system. They have a star rating system that kinda looks like amazons. The only problem is that each star is a link to a dynamic address that records the vote and then 302's back to the original page the vote was cast from. So virtually every page on this site links out anywhere from 15 to 45 times and 302's back to itself, losing virtually all of its PR. Am I correct in that assumption or am I missing something? I don't see the links being blocked by robots.txt or noindex, nofollowed. Also it is an anonymous rating system where a rating can be cast from any category page displaying a product or any product page. To make matters worse every page links to a printable version which duplicates the issue by repeating the whole thing over again. So assuming I am correct that is site has a major PR leak on virtually every page, what is the best recommendation to fix this. 1. Block all of those links in robots.txt, 2. no index, nofollow these links or 3. put the rating system behind a submit button or disallow anon ratings 4. something else??? Looking at their product ratings on the site virtually everything is between 2-3 starts out of 5 and has about the same number of votes except less votes on deeper pages. I dont believe this is real at all since this site gets almost no traffic and maybe 1 sale a week, there is no way that any product has been rated 50 times. I think the crawler is voting as it crawls and doing it 5 times for every product which is why everything is rated 2.5 out of 5. This is an x-cart site in case anyone cares. Any suggestions?
Technical SEO | | BlinkWeb0