How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Found a cache of old domain names, should I link or 301 redirect
We have found a cache of about 10 URLs, some are ranking above our main URL in Google SERPS. What is the best course of action here? a. Redirect all to the homepage?
Intermediate & Advanced SEO | | moconn
b. Link all domains to the homepage?
c. Link all domains to select pages on on main site, being careful not to anchor text spam
d. 301 redirect all to the main site. Is there any disadvantage to your recommendation? Is there likely to be a penalty incurred? I feel like we'll get the strongest increase in rankings by following option c but it feels like option d may be safer. Thanks in advance for your help!0 -
Soft 404 error for a big, longstanding 301-redirected page
Hi everyone, Years ago, we acquired a website that had essentially 2 prominent homepages - one was like example.com and the other like example.com/htm... They served the same purpose basically, and were both very powerful, like PR7 and often had double listings for important search phrases in Google. Both pages had amassed considerable powerful links to them. About 4 years ago, we decided to 301 redirect the example.com/htm page to our homepage to clean up the user experience on our site and also, we hoped, to make one even stronger page in serps, rather than two less strong pages. Suddenly, in the past couple weeks, this example.com/htm 301-ed page started appearing in our Google Search Console as a soft 404 error. We've never had a soft 404 error before now. I tried marking this as resolved, to see if the error would return or if it was just some kind of temporary blip. The error did return. So my questions are:
Intermediate & Advanced SEO | | Eric_R
1. Why would this be happening after all this time?
2. Is this soft 404 error a signal from Google that we are no longer getting any benefit from link juice funneled to our existing homepage through the example.com/htm 301 redirect? The example.com/htm page still has considerable (albeit old) links pointing to it across the web. We're trying to make sense of this soft 404 observation and any insight would be greatly appreciated. Thanks!
Eric0 -
How does Google treat chained 301 redirects?
I did the following two chained 301 redirects (A->B->C) Plural to Singular to New Domain A. http://domain1.com/filenames B. http://domain1.com/filename C. http://domain2.com/filename To new domain without www and then back to origining domain A. http://www.domain1.com/filename B. http://domain2.com/filename C. http://domain1.com/fifilename How much link juicy will be rediectetoto URL C in above two scenarios?
Intermediate & Advanced SEO | | Bull1350 -
Multiple 301 Redirects on the same domain name
Hi, I'd appreciate some advice ont he below. I have a website, say www.site.co.uk that has just been redesigned using a new CMS. Previously it had URLs in the format /article.php?id=123, the new site has more friendly urls in the format /articles/article-slug. I have been able to import the old articles into my CMS using the same article IDs and I have created a unique slug for each post. So now in my database, I have the article id (from the querystring) and a slug. However, I have hundreds of old URLs indexed by Google in the format /article.php?id=123 and need to redirect these. My plan was to do the following. 301 Redirect /article.php?id=123 to an intermediate page, in this case /redirect/123. On this intermediate page I would do a database lookup for the article slug, based on the ID from the querystring, create a new URL and perform a second 301 redirect to my new URL E.g. /articles/article-slug-from-database. Whilst this works and keeps the site usable for visitors the two 301 redirects do worry me, as I don;t want Google indexing lots of /redirect/[article id] urls. The other solution is to generate hundreds of htaccess redirect rules that map old url to the new url. The first solution is much cleaner, but the two 301's worry me. Will Google work this out on it's own, is there a better way? Any advice is much appreciated. Cheers Rob
Intermediate & Advanced SEO | | AmyCrompton1 -
How many pages to 301 Redirect
Hi Mozzers, My site has 11,200 pages indexed in Google and I'm looking to remove some of the lesser content which should probably have been picked up by Panda. However these pages work out to about 1,100 in total and I'm not sure whether to remove these bit by bit or just do it in one fell swoop? Does Google not like a site's indexed pages fluctuating too quickly? Are there any other considerations I should be aware of? Thanks!
Intermediate & Advanced SEO | | panini0 -
301 Redirect for 2500 pages
Hi, We have an existing site done in DNN and we recreated it on a new platform (EPiServer) and now we're going live. However, there are 2500+ page URLs from the old site which is not exisitng on the new site. What do you reckon is the best way we can address this? Do we create a 301 redirect individually for each of these pages? These 2500+ pages have a domain authority 34-35 and I think it's best that we retain those. We'll be using the same domain name. Suggestions for ways to approach this issue would be greatly appreciated. I have access to the server and IIS. *Also, how do I create a virtual page in IIS? and redirect it to another URL within the site? Thanks.
Intermediate & Advanced SEO | | Peter.Huxley590 -
301 Redirect All Url's - WWW -> HTTP
Hi guys, This is part 2 of a question I asked before which got partially answered; I clicked question answered before I realized it only fixed part of the problem so I think I have to post a new question now. I have an apache server I believe on Host Gator. What I want to do is redirect every URL to it's corresponding alternative (www redirects to http). So for example if someone typed in www.mysite.com/page1 it would take them to http://mysite.com/page1 Here is a code that has made all of my site's links go from WWW to HTTP which is great, but the problem is still if you try to access the WWW version by typing it, it still works and I need it to redirect. It's important because Google has been indexing SOME of the URL's as http and some as WWW and my site was just HTTP for a long time until I made the mistake of switching it now I'm having a problem with duplicate content and such. Updated it in Webmaster Tools but I need to do this regardless for other SE's. Thanks a ton! RewriteEngine On RewriteBase / RewriteCond %{HTTP_HOST} ^www.yourdomain.com [NC] RewriteRule ^(.*)$ http://yourdomain.com/$1 [L,R=301]
Intermediate & Advanced SEO | | DustinX0 -
Should I 301 Redirect Old Pages to Newer Ones?
I know there is value having lots of unique content on our websites, but I'm wondering how long it should be kept for, and if there is any value in 301 redirecting it? So, for example we have a number of pages on our website that are dedicated to single products (blue widget x, blue widget y, red widget x, red widget y). Nice unique content, with some (but not many) links. These products are no longer available though and have been replaced. So I'm faced with three choices: 1. Leave it as it is, and hope it adds to the overall site authority (by value of being another page), and also perhaps mop up a few longer tail keywords. Add a link to the replacement product on these pages; 2. 301 redirect these pages to the replacement products to give these a bit of a boost, and lose the content; 3. 301 redirect these pages to the replacement products and move all the old content to a new 'blue widgets archive' and 'red widgets archive' page? Would appreciate everyones thoughts!
Intermediate & Advanced SEO | | BigMiniMan0