Tricky 301 question
-
A friend has relaunched a website but his web guys (he didn't consult me!) didn't do any 301s and now traffic unsurprisingly has tanked.
The old site and database no longer exists and there are now 2000+ 404's.
Any ideas how to do the 301s from old urls to new product urls WITHOUT it being a massive manual job?
-
That's my point, you only need to worry about the pages that had external links
Thanks -
Thanks
-
Pages dont just get equity from external links of course. If a category page has 10 links to it the product pages linked to on that page benefit. The wholesale drop in rankings isn't because every page had an external link to it.
-
I don't know what you mean about link equity, if there is no link pointing to the page then there is nothing lost.
As for search engines finding a lot of 404s, they will remove them from the index after a while, no problem there, you are returning the correct status code, that's what they want. This will allow them to clean up there index and stop crawling the pages. -
If the majority of URLs have no logic, then it makes things a bit tricky in regards to minimizing the amount of work.
I once had a very active and large website with about 500-1000 single lines of rewrite code (1 for each URL) in my htaccess. Surprisingly, it did not slow the server down at any noticeable rate, unless you are very sensitive to milliseconds and even then, one trial to the next could easily differ from regular internet congestion. My point is, nobody ever noticed.
Here's a few ways that I would handle this job to get through it as quickly and effortlessly as possible.
The more aggressive and time consuming approach:
I would output all the URLs that were changed from phpmyadmin or whatever mysql administration tool you might use to a spreadsheet. From that spreadsheet, I would add the original URL.
Then with the old URL (A1) and new URL (A2) I would write a formula to output the correct rewrite (A3.) Then simply copy and paste that formula down all the rows that it applies to. You might need to break up the URLs to grab the right pieces for your formula.Of course use, regex where you can, and keep your .htaccess rewrites to a minimum.
If that is still too much work, hire someone to do it through elance.com
The somewhat sloppy pace-yourself-approach:
Another approach you could take is to just monitor google webmaster tools for all the page not found errors. And once a day or once a week, grab those URLS, create the rewrite, and mark it as fixed in webmaster tools.
The reason I say this is somewhat sloppy is because, you might find that you could have used regex in a lot of instances to better handle all those missing URLs.
But it may be a good way of staying on track with google, and handling the issues only as they arise so it does not feel like such a mammoth task.
-
Thanks Alan, yes they have good external links to many pages. They retail a very niche product and have a lot of forum, review, social type links. It might be though if need be they just have to focus mostly on 301s for the pages with those links. As best practise I am in favour of 301'ing regardless of external links as the link equity gets messed up and causes ranking issues, as in this case, as well as sending a signal to the engines about the amount of wasted resource they will use crawling a site with 1000s of 404s.
-
Thanks Donna & Luis. Luis is right i'm looking for a way for this not to be a mammoth manual task for their developer.
-
Thanks, the regex is a good idea and might be part of the solution for some urls at least but there seems to be some discrepancies in logic between old and new product urls and some of the new product urls are actually still the same as the old (which of course is fine).
-
Thanks Luis, unfortunately neither 1 or 2 are ideal.
1. I don't think there is much logic in the change of url structure between old and new product urls which makes that idea impossible.
2. Thats going to be a last resort
Andy
-
do you know if they had any external links?
If they don't have external links then I would just let them 404.
some people have some wired thoughts of what 301's do. They simply redirect a request, so a request o A is told to remake the request to to B, so the crawler will follow it that way and award the pagerank to the new page with a small loss on each request.If no external links what is there to gain? don't complicate your site with unnesasary redirects, there is a small argument that the pages may have been bookmarked at old url, but I think that argument is so weak I would not bother
-
Yeah. I heard him. I guess I'm saying "probably not".
I like how you're keeping us honest though Luis. I don't like it when people respond with what they want to say rather than with an answer to the specific question.
-
Donna,
Andy has been very specific about this: "WITHOUT it being a massive manual job" hehe thanks for supporting my answer.
Luis
-
It really depends on the nature, link and traffic patterns of your site Andy. If the vast majority of those 2,000+ 404's are coming from pages that should never have been indexed in the first place, you can probably get away with Luis's 2nd suggestion. If they're differentiated, valuable, and show evidence of incoming links and traffic, you've got some work ahead of you.
You might be able to streamline the process by inventorying and grouping like pages, then doing group redirects. But I suggest you do some analysis first to determine whether the effort is warranted.
-
2000+ is a lot of URLs to work through. But you can most likely get through them quickly with a few good regular expression 301 redirects in your .htaccess
If you have a pretty consistent form from the old url to the new one, this will be a piece of cake.
ex:
old URL: this/was/coolnew URL: this/is/cool
However, if there is really no rhyme and reason to the newly formed URLs, this could end up taking a considerate amount of time.
I would look into writing 301 redirects with regular expressions in .htaccess (I'm assuming your server is and uses .htaccess)
There are a number of resources for doing this, and even one here at moz.com
https://moz.com/learn/seo/redirection -
Hello Andy,
1. Try this: http://webdesign.about.com/od/htaccess/ht/redirect-an-entire-site-using-htaccess.htm
2. Second/faster solution. You could add this line of code to your .htacess file (and all the current "404's users" will go to the homepage):
ErrorDocument 404 /
But pay attention... 404's are perfectly normal if the page no longer exists, for user experience you should only ever use a 301 redirect if the page that no longer exists is going to a equal page.. i.e about cars to cars, about rabbits to rabbits. Maybe the only solution is creating a 404 specific landing page for this (with links to different sections of your site)
Hope this helps,
Luis
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
To subdomain or to subfolder, that is the question.
Hi All, So I have a client that has two restaurants that they are wanting two sites for. Right now they have one site for their two locations that ranks pretty well for some bigger keywords for their style of food. With them wanting two sites, i'm struggling on whether we should just build them all within one site and just use separate folders on that site restaurant.com/location1 & restaurant.com/location2 with a landing page sending you to each, or if we should split it into subdomains. The content will be roughly the same, the menus are identical, i think each branch is just owned by a different family member so they want their own site. I keep leaning towards building it all into one site but i'm not sure. Any ideas?
Intermediate & Advanced SEO | | insitemoz10 -
Internal links question
I've read that Google frowns upon large numbers of internal links. We're building a site that helps users browse a list of shows via dozens of genres. If the genres are expose, say, as a pulldown menu as opposed to a list of static links, and selecting the pulldown option filters the list of shows, would those genres count against our internal links count?
Intermediate & Advanced SEO | | TheaterMania0 -
Ecommerce question - Should I use a CDN for my images. ?
Hi , We are currently in the process of re-developing out commerce website and I wondering should we use a CDN (content delivery nertwork) for our product images. My category pages are currently showing approx 21 product images per page and the page speed is okay but can be better but the page size is rather large ... anything between 600kb - 1 Meg. We do optimise the images already in photoshop. We also do things like minify etc to get the pages to load as fast as possible but I think the only thing left is using a CDN but I have heard mixed reports about using this.? We are also doing a mobile responsive version of the site to but I know that speed will be king with google and how it reflects on rankings. Whilst I can see a CDN will improve image page load speed etc, I guess there a negative SEO impact as well as images will be stored in the cloud ?.. as opposed on to on my site/database. Does anyone know how best to implement a CDN without impacting on SEO or know of any good SEO /implementation articles on this ?... Maybe do Ieave some images on my category pages so I can still do the alt image tags etc/ and have the remaining images on the CDN.? Many Thanks Sarah
Intermediate & Advanced SEO | | SarahCollins0 -
How to conduct catch 301 redirects & have the separate 301 redirects for the key pages
Hi, We've currently done a site migration mapping and 301 redirecting only the sites key pages. However two GWT (Google Webmaster Tools) is picking a massive amount of 404 areas and there has been some drop in rankings. I want to mitigate the site from further decline, and hence thought about doing a catch 301 - that is 301 redirecting the remaining pages found on the old site back to the home page, with the future aim of going through each URL one by one to redirect them to the page which is most relevant. Two questions, (1) can I do a catch 301 and if so what is the process and requirements that I have to give to the developer? (2) How do you reduce the number of increasing 404 errors from a site, despite doing 301 redirects and updating links on external linking sites. Note: The server is apache and the site is hosted on Wordpress platform. Regards, Vahe
Intermediate & Advanced SEO | | Vahe.Arabian0 -
Interesting site migration question.
Hi all. I'm looking for some thoughts on a migrations option we have. At the moment we have two E-Com sites ranking well for some of the same terms. An older site, and a nice new site. The older site is ranking very well for category and product terms, the new one is slowly coming up. Ideally we would like to have one site, the nice new one, and get rid of the old one. If I 301 the old site url's to the new sites will that bring the new site url's into the same position as the old ones? I'm just not sure how this effects sites that are already ranking well. Any ideas are welcomed but I'm really looking for a definitive answer. It's a big decision after all.
Intermediate & Advanced SEO | | PASSLtd0 -
Question about HTTP Vary for Mobile
I'm reviewing https://developers.google.com/webmasters/smartphone-sites/redirects, and wondering where exactly to add HTTP Vary: Desktop request which has a mobile page to add “Vary: User-Agent” to the response HEADER Or if the request came from mobile device, than add “Vary: User-Agent” to the response HEADER
Intermediate & Advanced SEO | | nicole.healthline0 -
What happens with a 301 redirected page?
Hi All, What happens with an indexed page that I 301 redirect?
Intermediate & Advanced SEO | | BeytzNet
Is it removed from the Google index after a while? Thanks0 -
Canonical Tag - Question
Hey, I will give a thumbs up and best answer to whoever answers my question correctly. The Canonical Tag is supposed to solve Duplication which is fine. My questions are: Does the Canonical Tag make the PR / Link Juice flow differently? If I have john.long.com/home and john.long.com but put a Canonical Tag on john.long.com/home reading john.long.com then what does this do? Does it flow the Link Equity back to john.long.com? Can you use the Canonical Tag to change PR flow in any means? If I had john.long.com/washing-machines and john.long.com/kids-toys... If I put a Canonical Tag on john.long.com/kids-toys reading john.long.com/washing-machines then would the PR from /kids-toys flow to /washing-machines or would Google just ignore this? (The pages are completely different in this example and content is completely different). Thank you.
Intermediate & Advanced SEO | | AdiRste0