Tricky 301 question
-
A friend has relaunched a website but his web guys (he didn't consult me!) didn't do any 301s and now traffic unsurprisingly has tanked.
The old site and database no longer exists and there are now 2000+ 404's.
Any ideas how to do the 301s from old urls to new product urls WITHOUT it being a massive manual job?
-
That's my point, you only need to worry about the pages that had external links
Thanks -
Thanks
-
Pages dont just get equity from external links of course. If a category page has 10 links to it the product pages linked to on that page benefit. The wholesale drop in rankings isn't because every page had an external link to it.
-
I don't know what you mean about link equity, if there is no link pointing to the page then there is nothing lost.
As for search engines finding a lot of 404s, they will remove them from the index after a while, no problem there, you are returning the correct status code, that's what they want. This will allow them to clean up there index and stop crawling the pages. -
If the majority of URLs have no logic, then it makes things a bit tricky in regards to minimizing the amount of work.
I once had a very active and large website with about 500-1000 single lines of rewrite code (1 for each URL) in my htaccess. Surprisingly, it did not slow the server down at any noticeable rate, unless you are very sensitive to milliseconds and even then, one trial to the next could easily differ from regular internet congestion. My point is, nobody ever noticed.
Here's a few ways that I would handle this job to get through it as quickly and effortlessly as possible.
The more aggressive and time consuming approach:
I would output all the URLs that were changed from phpmyadmin or whatever mysql administration tool you might use to a spreadsheet. From that spreadsheet, I would add the original URL.
Then with the old URL (A1) and new URL (A2) I would write a formula to output the correct rewrite (A3.) Then simply copy and paste that formula down all the rows that it applies to. You might need to break up the URLs to grab the right pieces for your formula.Of course use, regex where you can, and keep your .htaccess rewrites to a minimum.
If that is still too much work, hire someone to do it through elance.com
The somewhat sloppy pace-yourself-approach:
Another approach you could take is to just monitor google webmaster tools for all the page not found errors. And once a day or once a week, grab those URLS, create the rewrite, and mark it as fixed in webmaster tools.
The reason I say this is somewhat sloppy is because, you might find that you could have used regex in a lot of instances to better handle all those missing URLs.
But it may be a good way of staying on track with google, and handling the issues only as they arise so it does not feel like such a mammoth task.
-
Thanks Alan, yes they have good external links to many pages. They retail a very niche product and have a lot of forum, review, social type links. It might be though if need be they just have to focus mostly on 301s for the pages with those links. As best practise I am in favour of 301'ing regardless of external links as the link equity gets messed up and causes ranking issues, as in this case, as well as sending a signal to the engines about the amount of wasted resource they will use crawling a site with 1000s of 404s.
-
Thanks Donna & Luis. Luis is right i'm looking for a way for this not to be a mammoth manual task for their developer.
-
Thanks, the regex is a good idea and might be part of the solution for some urls at least but there seems to be some discrepancies in logic between old and new product urls and some of the new product urls are actually still the same as the old (which of course is fine).
-
Thanks Luis, unfortunately neither 1 or 2 are ideal.
1. I don't think there is much logic in the change of url structure between old and new product urls which makes that idea impossible.
2. Thats going to be a last resort
Andy
-
do you know if they had any external links?
If they don't have external links then I would just let them 404.
some people have some wired thoughts of what 301's do. They simply redirect a request, so a request o A is told to remake the request to to B, so the crawler will follow it that way and award the pagerank to the new page with a small loss on each request.If no external links what is there to gain? don't complicate your site with unnesasary redirects, there is a small argument that the pages may have been bookmarked at old url, but I think that argument is so weak I would not bother
-
Yeah. I heard him. I guess I'm saying "probably not".
I like how you're keeping us honest though Luis. I don't like it when people respond with what they want to say rather than with an answer to the specific question.
-
Donna,
Andy has been very specific about this: "WITHOUT it being a massive manual job" hehe thanks for supporting my answer.
Luis
-
It really depends on the nature, link and traffic patterns of your site Andy. If the vast majority of those 2,000+ 404's are coming from pages that should never have been indexed in the first place, you can probably get away with Luis's 2nd suggestion. If they're differentiated, valuable, and show evidence of incoming links and traffic, you've got some work ahead of you.
You might be able to streamline the process by inventorying and grouping like pages, then doing group redirects. But I suggest you do some analysis first to determine whether the effort is warranted.
-
2000+ is a lot of URLs to work through. But you can most likely get through them quickly with a few good regular expression 301 redirects in your .htaccess
If you have a pretty consistent form from the old url to the new one, this will be a piece of cake.
ex:
old URL: this/was/coolnew URL: this/is/cool
However, if there is really no rhyme and reason to the newly formed URLs, this could end up taking a considerate amount of time.
I would look into writing 301 redirects with regular expressions in .htaccess (I'm assuming your server is and uses .htaccess)
There are a number of resources for doing this, and even one here at moz.com
https://moz.com/learn/seo/redirection -
Hello Andy,
1. Try this: http://webdesign.about.com/od/htaccess/ht/redirect-an-entire-site-using-htaccess.htm
2. Second/faster solution. You could add this line of code to your .htacess file (and all the current "404's users" will go to the homepage):
ErrorDocument 404 /
But pay attention... 404's are perfectly normal if the page no longer exists, for user experience you should only ever use a 301 redirect if the page that no longer exists is going to a equal page.. i.e about cars to cars, about rabbits to rabbits. Maybe the only solution is creating a 404 specific landing page for this (with links to different sections of your site)
Hope this helps,
Luis
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Scraping / Duplicate Content Question
Hi All, I understanding the way to protect content such as a feature rich article is to create authorship by linking to your Google+ account. My Question
Intermediate & Advanced SEO | | Mark_Ch
You have created a webpage that is informative but not worthy to be an article, hence no need create authorship in Google+
If a competitor comes along and steals this content word for word, something similar, creates their own Google+ page, can you be penalised? Is there any way to protect yourself without authorship and Google+? Regards Mark0 -
301 Redirect from now defunct website?
Hi guys Quick question about 301 redirection between domains. I currently manage a website, lets call it website A. Website A sells a particular product range, however the decision has been made by the powers that be to pull the plug on the business and sell the products previously sold via Website A via another website within the parent companies control.....lets call it Website B. I need to make it clear to customers of Website A that the company no longer operates but want to pass the SEO equity that has been built up over time to the relevant pages on Website B. My plan was to 1. 301 Redirect all key landing pages on Website A to the most relevant pages on Website B 2. Initially keep the website A homepage live but change the message to say "Website A no longer operates, but Website B can help etc. etc." Remove all sub links from navigation. 3. Monitor referral and direct traffic levels and consider 301 redirecting website A homepage to Website B homepage in the long term. My questions: Does this sound like the best approach? If not, what alternatives are there? Will Website A look like a link farm for Website B? I dont want this obviously!
Intermediate & Advanced SEO | | DHS_SH0 -
When is it time to kill 301 redirects
3 months we updated our site design design and as such lots of page urls changed. At the time we 301 redirected about 100 pages. (All pages are on the same domain - 301 redirects like .com/about-us/company to .com/company) Anyhow my question is should I leave these redirects active indefinitely or kill them assuming value has passed through by now? Your Thoughts are welcomed. Thanks, Glen.
Intermediate & Advanced SEO | | AdvanceSystems0 -
Question about best approach to site structure
I am curious if anyone can share some advice. I am working on planning architecture for a tour company. The key piece of the content strategy will be providing details on each of the tour destinations, with associated profiles for each city within those destinations. Lots of content, which should be great for the SEO strategy. With regards to the architecture, I have a ‘destinations’ section on the Website where users can access each of the key destinations served by the tour company. My question is – from a planning perspective I can organize my folder structure in a few different ways. http://www.companyurl.com/destinations/touring-regions/cities/ or http://www.companyurl.com/destinations/ http://www.companyurl.com/touring-regionA/ http://www.companyurl.com/touring-regionB/cities-profile/ I am curious if anyone has an opinion on what might perform best in terms of the site structure from an SEO perspective. My fear is taking all of this rich content and placing it so many tiers down in the architecture of the site. Any advice that could be offered would be appreciated. Thanks.
Intermediate & Advanced SEO | | VERBInteractive0 -
Quick htaccess question
Hi! I'm trying to do a 301 from www.stevesims.com/index.htm to www.stevesims.com. I know I need to use the request command to avoid an infinite loop, but I can't quite figure out the correct code. Here's the first part of the htaccess file. RewriteEngine On RewriteCond %{HTTP_HOST} ^stevesims.com
Intermediate & Advanced SEO | | Blink-SEO
RewriteRule (.*) http://www.stevesims.com/$1 [R=301,L] RewriteCond %{HTTP_REFERER} !^http://stevesims.com/.$ [NC]
RewriteCond %{HTTP_REFERER} !^http://stevesims.com$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.stevesims.com/.$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.stevesims.com$ [NC]
RewriteRule .*.(jpg|jpeg|gif|png|bmp)$ - [F,NC] Any suggestions would be much appreciated.0 -
Big 301 Redirect Help!
Hey guys I need a little help with setting up a big 301. Background: It's a bit of a mess as the old site is a total mess after being online for 10 years plus. It has html and php pages, and a mod rewrite to redirect old html links to the newer php version of those pages. It's now moving to a new site and as the domain name and URL structure has changed we can't use any fancy regex and have to do a page to page redirect. There are 1500 pages to redirect. However, the old site has thousands of linking root domains, and some of these are to the old html pages (which currently redirect to the php pages) and some to the newer php pages. Question: My initial plan was to leave the mod rewrite and only redirect the php pages. That means 1500 individual redirects instead of 3000 if I individually redirect both the php and html pages. I'm not sure what's best to be honest. We don't really want multiple hops in the redirect (html>php>new site), but surely 1500 redirects is better than 3000! Does anyone have any advice on which option may be best, or even a better option? Thanks 🙂
Intermediate & Advanced SEO | | HarveyP0 -
Spammy Link Profile Questions. What do you think?
I'm trying to dilute the link profile for a website. But have a couple of questions on the best way to achieve this. Current link profile, www.mysitename.com Keyword 1 Keyword 2 Keyword 3 Keyword 4 Keyword 5 Keyword 6 Keyword 7 Keyword 8 Keyword 9 Keyword 10 Keyword 12 Keyword 13 Keyword 14 Keyword 14 Keyword 15 mysitename.com Desired link profile, www.mysitename.com mysitename.com www.mysitename.com http://www. mysitename.com/ My Site Name http://mysitename.com Click Here my site name More Info mysitename.com/ www.mysitename.com/ Keyword 1 Keyword 2 Keyword 3 Keyword 4 Keyword 4 Keyword 5 Questions 1. Do you think Google looks at this on a domain level? Or do you think this needs to be done with every page on the site? 2. What would be a good way to build links fast to the pages, need to build lots of links to be able to dilute the profile. I was considering Dripable, or a similar service, but decided i really don't want to create more spam.What would you do? 3. What would you say the % threshold for anchor text is, i have read on different sources that at least 40% - 60% of links should be branded, url, or generic anchor links. Do you think this is accurate?
Intermediate & Advanced SEO | | 858-SEO0 -
A very basic seo question
Sorry, been a long day and wanted a second opinion on this please.... I am developing an affiliate store which will have dozens of products in each category. We will not be indexing the product pages themselves as they are all duplicate content. The plan is to have just the first page of the category results indexed as this will have unique content about the products in that section. The later pagnated pages (ie pages 2,3,4,5 etc) will have 12 products on each but no unique content. Would the best advice be to add a canonical tag to all pages in the 'chairs' category pointing to the page with the first 12 results and the descriptions? This would ensure that the visitors are able to browse many pages of product but google won't index products 13 and onwards. Am I right in my thinkings? A supplemental question. What is the best way to block google from indexing/crawling 90,000 product listings which are pulled direct from the merchant so are not unique in the least. I have previous played with banning google from the product folder but it reports health issues in webmaster tools. Would the best route be a no index tag on all the product pages and to no follow all the products in the category listings? Many thanks Carl
Intermediate & Advanced SEO | | Grumpy_Carl0