Tricky 301 question
-
A friend has relaunched a website but his web guys (he didn't consult me!) didn't do any 301s and now traffic unsurprisingly has tanked.
The old site and database no longer exists and there are now 2000+ 404's.
Any ideas how to do the 301s from old urls to new product urls WITHOUT it being a massive manual job?
-
That's my point, you only need to worry about the pages that had external links
Thanks -
Thanks
-
Pages dont just get equity from external links of course. If a category page has 10 links to it the product pages linked to on that page benefit. The wholesale drop in rankings isn't because every page had an external link to it.
-
I don't know what you mean about link equity, if there is no link pointing to the page then there is nothing lost.
As for search engines finding a lot of 404s, they will remove them from the index after a while, no problem there, you are returning the correct status code, that's what they want. This will allow them to clean up there index and stop crawling the pages. -
If the majority of URLs have no logic, then it makes things a bit tricky in regards to minimizing the amount of work.
I once had a very active and large website with about 500-1000 single lines of rewrite code (1 for each URL) in my htaccess. Surprisingly, it did not slow the server down at any noticeable rate, unless you are very sensitive to milliseconds and even then, one trial to the next could easily differ from regular internet congestion. My point is, nobody ever noticed.
Here's a few ways that I would handle this job to get through it as quickly and effortlessly as possible.
The more aggressive and time consuming approach:
I would output all the URLs that were changed from phpmyadmin or whatever mysql administration tool you might use to a spreadsheet. From that spreadsheet, I would add the original URL.
Then with the old URL (A1) and new URL (A2) I would write a formula to output the correct rewrite (A3.) Then simply copy and paste that formula down all the rows that it applies to. You might need to break up the URLs to grab the right pieces for your formula.Of course use, regex where you can, and keep your .htaccess rewrites to a minimum.
If that is still too much work, hire someone to do it through elance.com
The somewhat sloppy pace-yourself-approach:
Another approach you could take is to just monitor google webmaster tools for all the page not found errors. And once a day or once a week, grab those URLS, create the rewrite, and mark it as fixed in webmaster tools.
The reason I say this is somewhat sloppy is because, you might find that you could have used regex in a lot of instances to better handle all those missing URLs.
But it may be a good way of staying on track with google, and handling the issues only as they arise so it does not feel like such a mammoth task.
-
Thanks Alan, yes they have good external links to many pages. They retail a very niche product and have a lot of forum, review, social type links. It might be though if need be they just have to focus mostly on 301s for the pages with those links. As best practise I am in favour of 301'ing regardless of external links as the link equity gets messed up and causes ranking issues, as in this case, as well as sending a signal to the engines about the amount of wasted resource they will use crawling a site with 1000s of 404s.
-
Thanks Donna & Luis. Luis is right i'm looking for a way for this not to be a mammoth manual task for their developer.
-
Thanks, the regex is a good idea and might be part of the solution for some urls at least but there seems to be some discrepancies in logic between old and new product urls and some of the new product urls are actually still the same as the old (which of course is fine).
-
Thanks Luis, unfortunately neither 1 or 2 are ideal.
1. I don't think there is much logic in the change of url structure between old and new product urls which makes that idea impossible.
2. Thats going to be a last resort
Andy
-
do you know if they had any external links?
If they don't have external links then I would just let them 404.
some people have some wired thoughts of what 301's do. They simply redirect a request, so a request o A is told to remake the request to to B, so the crawler will follow it that way and award the pagerank to the new page with a small loss on each request.If no external links what is there to gain? don't complicate your site with unnesasary redirects, there is a small argument that the pages may have been bookmarked at old url, but I think that argument is so weak I would not bother
-
Yeah. I heard him. I guess I'm saying "probably not".
I like how you're keeping us honest though Luis. I don't like it when people respond with what they want to say rather than with an answer to the specific question.
-
Donna,
Andy has been very specific about this: "WITHOUT it being a massive manual job" hehe thanks for supporting my answer.
Luis
-
It really depends on the nature, link and traffic patterns of your site Andy. If the vast majority of those 2,000+ 404's are coming from pages that should never have been indexed in the first place, you can probably get away with Luis's 2nd suggestion. If they're differentiated, valuable, and show evidence of incoming links and traffic, you've got some work ahead of you.
You might be able to streamline the process by inventorying and grouping like pages, then doing group redirects. But I suggest you do some analysis first to determine whether the effort is warranted.
-
2000+ is a lot of URLs to work through. But you can most likely get through them quickly with a few good regular expression 301 redirects in your .htaccess
If you have a pretty consistent form from the old url to the new one, this will be a piece of cake.
ex:
old URL: this/was/coolnew URL: this/is/cool
However, if there is really no rhyme and reason to the newly formed URLs, this could end up taking a considerate amount of time.
I would look into writing 301 redirects with regular expressions in .htaccess (I'm assuming your server is and uses .htaccess)
There are a number of resources for doing this, and even one here at moz.com
https://moz.com/learn/seo/redirection -
Hello Andy,
1. Try this: http://webdesign.about.com/od/htaccess/ht/redirect-an-entire-site-using-htaccess.htm
2. Second/faster solution. You could add this line of code to your .htacess file (and all the current "404's users" will go to the homepage):
ErrorDocument 404 /
But pay attention... 404's are perfectly normal if the page no longer exists, for user experience you should only ever use a 301 redirect if the page that no longer exists is going to a equal page.. i.e about cars to cars, about rabbits to rabbits. Maybe the only solution is creating a 404 specific landing page for this (with links to different sections of your site)
Hope this helps,
Luis
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Few question about SEO
HI guys, I have few questions and I always find good answer here. I tried many SEO companies some very expensive and well known some with medium prices and some from India. I’m not an SEO expert but I always get the same things from SEO companies. They're saying you have to stay with us for few months before you’ll see any results. I completely understand however I don’t see the result on the end.1. What exactly Do I need SEO company for, after I do on page optimisation if they don’t work on proper backlinks. Just letting you know I’m getting content from other people.2. Is there something else which is really important after your page is optimised than backlinks? Or we should fully focus on get backlinks from customers, guest post, sharing on social media etc. to increase our DA and PA?3. Any advice about some individual or company who is good in backlink services?
Intermediate & Advanced SEO | | Lukas-ST
Thank youLukasThanks a lot.Lukas0 -
Location in URLs question
Hi there, my company is a national theater news publisher. Quick question about a particular use case. When an editor publishes a story they can assign several discrete locations, allowing it to appear on each of those locations within our website. This article (http://www.theatermania.com/denver-theater/news/full-casting-if-then-tour-idina-menzel_74354.html), for example, appears in the Los Angeles, San Francisco, and Denver section. We force the author to choose a primary location from that list, which controls the location displayed in the URL. Is this a bad practice? I'm wondering if the fact that having 'Denver' in the URL is misleading and hurts SEO value, particularly since that article features several other cities.
Intermediate & Advanced SEO | | TheaterMania0 -
SEO Question re: Keyword Cannibalization
I know about Keyword Cannibalization, so I understand why it's generally a problem. If you have multiple versions of the same page, Google has to "guess" which one to display (as I understand it, unless you have a SUPER influential page you won't get both pages showing up on the SERP). To explain why I'm not sure if this applies to our page, we have a blog that we write about employment law issues on. So we might have 20 blog posts over the past year that all talk about recent pregnancy discrimination lawsuits employers might be interested in. Now, searching the Google Keyword tools, there aren't even close to 20 different focus keywords that would make any sense. "Pregnancy Discrimination lawsuit" is niche enough for us to be competitive, but anything more specific than that simply has very little search activity. My suggestion is to just optimize all of them for "pregnancy discrimination lawsuit". My understand of how Panda works is that if the content is different on each page (and it is!) then it will only display what it guesses is the most relevant "NLRB" post, but any link juice sent to the other 19 "NLRB" posts would still boost the relevancy for whatever post Google chooses. And it wouldn't get dinged as keyword stuffing because it's clearly not just the same page repeated over and over. I've found quite a few articles on Keyword Cannibalization but many are pre-Panda. I was CERTAIN I'd seen a post that explained my idea is a totally viable and good one, but of course now I can't find it. So before I go full steam ahead with this strategy I just want to make sure there's nothing I'm missing. Thanks!
Intermediate & Advanced SEO | | CEDRSolutions0 -
XML Sitemap Questions For Big Site
Hey Guys, I have a few question about XML Sitemaps. For a social site that is going to have presonal accounts created, what is the best way to get them indexed? When it comes to profiles I found out that twitter (https://twitter.com/i/directory/profiles) and facebook (https://www.facebook.com/find-friends?ref=pf) have directory pages, but Google plus has xml index pages (http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml). If we go the XML route, how would we automatically add new profiles to the sitemap? Or is the only option to keep updating your xml profiles using a third party software (sitemapwriter)? If a user chooses to not have their profile indexed (by default it will be index-able), how do we go about deindexing that profile? Is their an automatic way of doing this? Lastly, has anyone dappled with google sitemap generator (https://code.google.com/p/googlesitemapgenerator/) if so do you recommend it? Thank you!
Intermediate & Advanced SEO | | keywordwizzard0 -
Ecommerce SEO URL Structure Questions
| I am in the process of developing a new Magento ecommerce store. Take for instance this website is in the apparel industry and i have the following main categories. Clothing Shoes Accessories Beauty Sub categories for clothing would be: Dresses Pants jeans Tops Products would be: Kelly Maxi dresses What is the best SEO Structure for this? Main categories obviously: www.example.com/clothing Sub Categories:
Intermediate & Advanced SEO | | WayneRooney
www.example.com/clothing/dresses Or www.example.com/dresses (Zappos seem to pursue the second type) Products:
www.example.com/clothing/dresses/kelly-maxi-dresses/ Or www.example.com/kelly-maxi-dresses ? Which one would be the best way to structure your site? Also what about filters that available in category pages? Say if i were to filter by color. what would be the best URL? I am sure canonical tag is needed here. New to Ecommerce SEO so i need some guidance! |0 -
Permalink question
For 5 years I have used the permalink custom structure: /%postname% without the end backslash. I didn't think the difference was that big of a deal, yet last month I was curious of what benefits would happen if I made the change. To my surprise my rankings took a slight dive, but recovered stronger than before. As the URL itself doesn't require a redirect the posts and pages loaded the same with or wothout the "/" But now in Open Site Explorer, all my URL's have no page Authority. All the links i built were pointing to links without the backslash: example.com/post-name Questions: Did Google figure out the change, hence the dip in rankings and strong return? Will keeping /%postname%/ even though many links are pointing to a non backslash URL comeback to haunt me? Is there anything I can do to help lead Google to better see the changes I've made? thx
Intermediate & Advanced SEO | | MikePatch0 -
Multiple 301 Redirects for the Same Page
Hi Mozzers, What happens if I have a trail of 301 redirects for the same page? For example,
Intermediate & Advanced SEO | | Travis-W
SiteA.com/10 --> SiteA.com/11 --> SiteA.com/13 --> SiteA.com/14 I know I lose a little bit of link juice by 301 redirecting.
The question is, would the link juice look like this for the example above? 100% --> 90% --> 81% -->72.9%
Or just 100% -----------------------------------------> 90% Does this link juice refer to juice from inbound links or links between internal pages on my site? Thanks!0 -
301 to 404
Hello Everybody, I've got a technical question about server responses. Imagine this scenario: www.domain.com/not-existing-page/ --> 404 & domain.com/not-existing-page/ --> www.domain.com/not-existing-page/ --> 404 I use Wordpress for my websites and I can't seem to be able to configure it (or the server where I have total control) to stop it doing this. Ideally, a non-existent url should return 404 instantly, not first redirect to the "corect" url and then return 404. Anyone experiencing this and can help? Here's a neat tool that allows you to quickly check server response codes - for those of you who are new to this: http://responsetester.appspot.com/ Much appreciated! Alex
Intermediate & Advanced SEO | | pwpaneuro1