How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 Redirects
Looking for the best way to do the following. Business has changed its name, and has also become a corporate store. The old domain name is now no longer needed as a website page has been created inside the main corporate site. Obviously i dont want to loose all the traffic that we had and want to redirect them but there is a problem, that im unable to redirect the old domain to the new one due to office 365 installed on the hosting platform, and the old emails will need to run for another 6 months. I can remove the old site and put a landing page up, but i still need to redirect all the pages to the new site, and there is approx 50+ of them. My main question is i currently have atleast 50+ redirects already in there due to seo changes over the years, some would go back atleast 5 years, whats a safe amount of time that i can remove the older redirects And am i going about this the right way so i dont loose all the hard work on rankings etc
Technical SEO | | Dunjoko0 -
Redirection loop. Best way to resolve...
Hi Guys Got a warning on a crawl today "Your page is redirecting to a page that is redirecting to a page that is redirecting to a page... and so on." In GWMT it is set to www. and also back-end in my server. I also have an SSL deployed and in my htaccess the rule is added to ensure all pages got to SSL. Any of you guys have advice regarding the best route to go or should I "IGNORE" this warning as all other aspects are clocking 95%+? Thanks in advance Daren
Technical SEO | | Daren-WebSupportLab0 -
To 301 or not to 301?
So, to cut a long story short, on our website we have a /product page that is very similar to our homepage and doesn't really serve much of a purpose. It doesn't really fit in with the rest of the website and our directors want to get rid of it and focus our efforts on our homepage. Problem is, the /product page has a little bit of PA and links to other important pages on the website. I personally don't want to completely cut this page off. Would a 301 redirect to the homepage be a good option or would I be better off redirecting users to our course library page (Our course pages are what bring in most of our organic traffic)? Any help or other suggestions would be appreciated here! http://www.ihasco.co.uk/
Technical SEO | | iHasco1 -
301 redirect chains
Hi everyone, I've had my site for a while now and have changed the structure a number of times. I'm confident my 301's work well and am not concerned about dead ends on my site. My question is, is there a way to find 301 redirect chains? i.e. can I export my link data from webmaster tools and run it through some software that tells me how many steps my 301's are taking to get to the final page? I don't know for sure that there are long 301 chains in my link structure, but I have a suspicion and it's very hard to check by going through them manually. Thanks in advance Will
Technical SEO | | madegood0 -
Remove html file extension and 301 redirects
Hi Recently I ask for some work done on my website from a company, but I am not sure what they've done is right.
Technical SEO | | ulefos
What I wanted was html file extensions to be removed like
/ash-logs.html to /ash-logs
also the index.html to www.timports.co.uk
I have done a crawl diagnostics and have duplicate page content and 32 page title duplicates. This is so doing my head in please help This is what is in the .htaccess file <ifmodule pagespeed_module="">ModPagespeed on
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule> <ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule> <ifmodule mod_rewrite.c="">Options +FollowSymLinks -MultiViews</ifmodule> DirectoryIndex index.html RewriteEngine On
# Rewrite valid requests on .html files RewriteCond %{REQUEST_FILENAME}.html -f RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]
# Return 404 on direct requests against .html files RewriteCond %{REQUEST_URI} .html$
RewriteCond %{QUERY_STRING} !rw=1 [NC]
RewriteRule ^ - [R=404] AddCharset UTF-8 .html # <filesmatch “.(js|css|html|htm|php|xml|swf|flv|ashx)$”="">#SetOutputFilter DEFLATE #</filesmatch> <ifmodule mod_expires.c="">ExpiresActive On
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years"
ExpiresByType image/x-icon "access plus 1 years"
ExpiresByType image/jpg "access plus 1 years"
ExpiresByType text/css "access 1 years"
ExpiresByType text/x-javascript "access 1 years"
ExpiresByType application/javascript "access 1 years"
ExpiresByType image/x-icon "access 1 years"</ifmodule> <files 403.shtml="">order allow,deny allow from all</files> redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html
redirect 301 /about_us.html http://www.timports.co.uk/about-us.html
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery.html redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery.html
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards.html
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs.html redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy.html redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed.html redirect 301 /payment_info.html http://www.timports.co.uk/payment-info.html1 -
Can I Get Penalized for 301 Redirects (Too Many or In Any Scenario)?
A client of ours owns several domain names that are keyword similar to the domain they actually use to run their site. They are asking us if we should 301 redirect all of these websites to the domain they use. However, I don't want this to work against them and their site get penalized later for this. I have heard buying out competitors and redirecting their domain to yours is frowned upon and penalized when you get caught (they did not do this). We are also wondering if there is a limit as to how many domains you can 301 redirect and what type (keyword similar, misspellings, .net's, etc.) and if you are penalized after too many (i.e. >50). All of the domains in question are keyword/brand name similar only and do not exist as actual websites. We just want to do the right thing. Thank you for your help.
Technical SEO | | JCunningham0 -
Where is the 301 redirect?
Hi, in the last week I take an issue for 301 permanent redirect for a subfolder in the main website! In that folder i have a index.php file for a google map fullscreen edition and the only link who connects the wordpress website with the subfolder is only a direct link! Is that an error of seomoz app or something else? Thanks 1.jpg
Technical SEO | | petrospan0 -
301 Redirects - SEO Benefit?
Hello, Years ago, our company started out as a Yahoo store. We've since moved onto another website with its own shopping cart but since the Yahoo store is almost 10 years old, there's a lot of history there and it still exists with the occasional order. We currently use it for reputation management purposes with links to our real ecommerce site but we're thinking of just redirecting the Yahoo store to our ecommerce site. Is there any SEO benefit in doing this? We were also kind of penalized by Panda. Would this help us out at all (the descriptions on both sites could be considered duplicate content).
Technical SEO | | airnwater0
