How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
HTacess 301 redirect with special characters
Hello moz community ! I would to make a special 301 redirection through my htaccess file. I am a total noob concerning regexp and 301 redirection. I would like to redirect(301) this url : http://www.legipermis.com/stages-points/">http://www.legipermis.com/stages-points/</a></p>; yes yes it's in the index of google, this strange url includes the last ; to http://www.legipermis.com/stages-points/ I have already include a canonical tag by security, i would like to remove url with a 301 redirection and by remove this url through GWT (but the removal tool can't "eat' this kind of URL) Please consider the fact that i am not an expert about 301 redirections and regexps. No 301 redirect generator works properly for such a strange URL (which triggers content duplication corrected anyway with canonical tag). Thanks for your help.
Technical SEO | | LegiPermis0 -
301 redirect with Magento; still Page authority 0 after 6 weeks
Hi Mozzers! In December '14 I have execute a 301 redirect in the 'old' page in the admin of my Magento store. Now I was surprised to see that the Page Authority is still 0 of the new page 6 weeks after the execution. should I have seen the update of the PA on the new page already after 6 weeks of time? If yes, then I assume that my Magento didn't execute this properly? Old url: http://hippemamashop.nl/mama/boeken/fotoalbum.html
Technical SEO | | aznventure
New page: you will be redirected to the new page after clicking the old page Mm4uBEl0 -
To 301 or not to 301?
I have a client that is having a new site built. Their old site (WP) does not use the trailing / at the end of urls. The new site is using most of the same url names but IS using the /. For instance, the old site would be www.example.com/products and the new site, also WP, will be www.example.com/products/. WordPress will resolve either way, but my question is whether or not to go in and redirect each matching non / page to the new url that has the /. I don't want to leave any link juice on the table but if I can keep the juice without doing a few hundred 301s that certainly wouldn't suck. Any thoughts? Sleepless in KVegas
Technical SEO | | seorocket0 -
CNAME vs 301 redirect
Hi all, Recently I created a website for a new client and my next job is trying to get them higher in Google. I added them in OSE and noticed some strange backlinks. To my surprise the client has about 20 domain names. All automatically poiting to (showing) the same new mainsite now. www.maindomain.nl www.maindomain.be
Technical SEO | | Houdoe
www.maindomain.eu
www.maindomain.com
www.otherdomain.nl
www.otherdomain.com
... Some of these domains have backlinks too (but not so much). I suggested to 301 redirect them all to the main site. Just to avoid duplicate content. But now the webhoster comes into play: "It's a problem, client has only 1 hosting account, blablabla...". They told me they could CNAME the 20 domains to the main domain. Or A-record them to an IP address. This is too technical stuff for me. So my concrete questions are: Is it smart to do anything at all or am I just harming my client? The main site is ranking pretty well now. And some backlinks are from their copy sites (probably because everywhere the logo links to the full mainsite url). Does the CNAME or A-record solution has the same effect as a 301 redirect, from SEO perspective? Many thanks,
Hans0 -
Automatically write Mass 301 redirects for csv
Hi Guys Does anyone know if there is away to write say 30 x 301 redirects in one go? I have a list from a client with old links and new links and I want to do it all at once. Any suggestions would be appreciate?
Technical SEO | | Cocoonfxmedia0 -
Best practice for eCommerce site migration, should I 301 redirect or match URLs on new site
Hi Guys, I have been struggling with this one for quite some time. I am no SEO expert like many of you, rather just a small business owner trying to do the right thing, so forgive me if I say something that makes no sense 🙂 I am moving our eCommerce store from one platform to another, in the process the store is getting a massive face lift. The part I am struggling with is whether I should keep my existing URL structure in place or use 301 redirects to create a cleaner looking URLs. Currently the URLs are a little long and I would love to move to a /category/product_name type format. Of course the goal is not to lose ranking in the process, I rank pretty well for several competitive phrases and do not want to create a negative impact. How would you guys handle this? Thanks, Dinesh
Technical SEO | | MyFairyTaleBooks0 -
Redirect
Hello, I need to redirect just the homepage of my blog somewhere. So blog.com But there are other redirects in there and I don't want to override them. So blog.com/post.html How do I just redirect the homepage of wordpress page without redirecting all the other pages. This is what I did that recreated the issue. redirect 301 / blog2.com Thanks! Tyler
Technical SEO | | tylerfraser0 -
301 redirect problems on site not yet moved
I have re-designed one of my sites, the old site is all static pages on a Windows server, I have made the new sites and it is on a new server and is running on Wordpress. I have just finished testing it and so am nearly ready to switch over the nameservers to the new server, however I'm having some problems with 301 redirects. I have tried to set up a few 301 redirects on the new server to test before I change nameservers but they don't appear to be working. I would have imagined that they should (even though the actual page isn't hosted on the new server) or am I being very stupid here and I can't test a 301 redirect until the nameservers have been changed. Redirect 301 /magazines.htm http://.../~account/magazine-freebies Obviously the above with the stars (*) is the server address and route to my account.
Technical SEO | | Wardy0