What to do about old urls that don't logically 301 redirect to current site?
-
Mozzers,
I have changed my site url structure several times.
As a result, I now have a lot of old URLs that don't really logically redirect to anything in the current site.
I started out 404-ing them, but it seemed like Google was penalizing my crawl rate AND it wasn't removing them from the index after being crawled several times. There are way too many (>100k) to use the URL removal tool even at a directory level.
So instead I took some advice and changed them to 200, but with a "noindex" meta tag and set them to not render any content. I get less errors but I now have a lot of pages that do this.
Should I (a) just 404 them and wait for Google to remove (b) keep the 200, noindex or (c) are there other things I can do? 410 maybe?
Thanks!
-
"So instead I took some advice and changed them to 200, but with a "noindex" meta tag and set them to not render any content. I get less errors but I now have a lot of pages that do this."
I would not recommend keeping it that way. You could mass redirect them to the sitemap page if they are passing PR and or some traffic, and there is no logical other place to point them.
404's are not really something that can hurt you, providing that they are coming from external sources and you aren't providing 404 links on your site to dead pages on your site, if there are these, then you should fix the internal links at the source.
-
I dont think 404 errors hurt your site. If you have that many pages, they are most likely crawling your site a lot anyway. Have you set your crawl frequency in your sitemap? On bigger sites that get frequent updates, we set the crawl frequency to daily rather than weekly.
If possible, try to see if there are any top level items you can submit a URL removal request for. Hopefully this can speed up the process fo getting the URL's removed. This process can take a long time for Google to take care of. After changing websites we still had 404 errors after 6 months, even after submitting the URL removal request.
Another option is to have the page render a 410 rather than a 404. A 410 states to the search engine the page is gone, and will not be coming back. If you are using some form of cart system or cms there might be a way to apply the code to a large number of pages at once, rather than trying to manually code 100k pages.
"410 Gone
The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know–or has no facility to determine–whether or not the condition is permanent, the status code 404 (Not Found) should be used instead of 410 (Gone). This response is cacheable unless indicated otherwise."Worse case scenero, you could set them to no-index, or just leave them be. Even if they dont lead anywhere logically, they could still bring you traffic. Or redirect them to the closest thing that is on the site currently.
-
JC,
When you say ...started out 404-ing them...seemed like Google was penalizing my crawl rate..... etc. I have not seen where Google even algorithmically had any real issues with 404's. I your site has 500K pages and 100K are 404'd I do not think it would be a problem for Google per se. (You might have a searcher problem if these were pages that were bookmarked, lots of links, etc.) My caution would be that if you have a lot of pages on the site with links that still go to the 404 pages you could run into UX issues.
For me, I would go with the 404's. I think they will get removed over time.Best
-
When necessary, redirect relevant pages to closely related URLs. Category pages are better than a general homepage.
If the page is no longer relevant, receives little traffic, and a better page does not exist, it’s often perfectly okay to serve a 404 or 410 status codes.
-
You could redirect them to something even remotely relevant even if its the homepage at the end of the day. What ever you do it going to take time and it's going to give you some sort of headache.
What would best suit a user who might land on an old link or somehow get to the page? That would be the best way to find a solution. A good soft 404 or redirect tends to help here.
Best of luck though.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirecting Ecommerce Site
Hi I'm working on a big site migration I'm setting up redirects for all the old categories to point to the new ones. I'm doing this based on relevancy, the categories don't match up exactly but I've tried to redirect to the most relevant alternative. Would this be the right approach?
Intermediate & Advanced SEO | | BeckyKey1 -
I've got duplicate pages. For example, blog/page/2 is the same as author/admin/page/2\. Is this something I should just ignore, or should I create the author/admin/page2 and then 301 redirect?
I'm going through the crawl report and it says I've got duplicate pages. For example, blog/page/2 is the same as author/admin/page/2/ Now, the author/admin/page/2 I can't even find in WordPress, but it is the same thing as blog/page/2 nonetheless. Is this something I should just ignore, or should I create the author/admin/page2 and then 301 redirect it to blog/page/2?
Intermediate & Advanced SEO | | shift-inc0 -
Should we use URL parameters or plain URL's=
Hi, Me and the development team are having a heated discussion about one of the more important thing in life, i.e. URL structures on our site. Let's say we are creating a AirBNB clone, and we want to be found when people search for apartments new york. As we have both have houses and apartments in all cities in the U.S it would make sense for our url to at least include these, so clone.com/Appartments/New-York but the user are also able to filter on price and size. This isn't really relevant for google, and we all agree on clone.com/Apartments/New-York should be canonical for all apartment/New York searches. But how should the url look like for people having a price for max 300$ and 100 sqft? clone.com/Apartments/New-York?price=30&size=100 or (We are using Node.js so no problem) clone.com/Apartments/New-York/Price/30/Size/100 The developers hate url parameters with a vengeance, and think the last version is the preferable one and most user readable, and says that as long we use canonical on everything to clone.com/Apartments/New-York it won't matter for god old google. I think the url parameters are the way to go for two reasons. One is that google might by themselves figure out that the price parameter doesn't matter (https://support.google.com/webmasters/answer/1235687?hl=en) and also it is possible in webmaster tools to actually tell google that you shouldn't worry about a parameter. We have agreed to disagree on this point, and let the wisdom of Moz decide what we ought to do. What do you all think?
Intermediate & Advanced SEO | | Peekabo0 -
Merging Sites: Will redirecting the old homepage to an internal page on the new site cause issues?
I've ended up with two sites which have similar content (but not duplicate) and target similar keywords, rather than trying to maintain two sites I would like to merge the sites together. The old site is more of a traditional niche site and targets a particular set of keywords on its homepage, the new site is more of an authority site with a magazine type homepage and targets the same set of keywords from an internal page. My question is: Should I redirect the old site's homepage to the relevant internal page on the new website...
Intermediate & Advanced SEO | | lara_dar
...or should I redirect the old site's homepage to the new site's homepage? (the old site's homepage backlinks are a mixture of partial match keyword anchor text, naked URLs and branded anchor text) I am in two minds (a & b!) (a) Redirecting to the internal page would be great for ranking as there are some decent backlinks and the content is similar (b) But usually when you do a 301 redirect the homepage usually directs to the new homepage and some of the old site's links are related to the domain rather than the keyword (e.g. http://www.site.com) and some people will be looking for the site's homepage. What do you think? Your help is much appreciated (and hope this makes sense...!)0 -
301 Redirect and Webmaster Central
I've been working on removing canonical issues. My host is Apache. Is this the correct code for my htaccess? RewriteEngine On
Intermediate & Advanced SEO | | spkcp111
RewriteCond %{HTTP_HOST} ^luckygemstones.com$ [NC]
RewriteRule ^(.*)$ http://www.luckygemstones.com/$1 [R=301,L] SECOND!!! I have two websites under Google's Webmaster Central; http://luckygemstones.com which gets NO 404 soft errors... AND http://www.luckygemstones.com which has 247 soft 404 errors... I think I should DELETE the http://luckygemstones.com site from Webmaster Central--the 301 redirect handles the"www" thing. Is this correct? I hate to hose things (even worse?) Help! Kathleen0 -
Canonicalization interact with 301 redirects?
This is a interesting one I think. I have recently taken down some product list pages from our website www.towelsrus.co.uk. These have canonicalisation in place to deal with pages where a query string is generated depending on the search criteria. When I put a 301 redirect in place the target page redirects fine, however webmaster tools then errors with 404 on all canonicalised pages. Is this correct behaviour and how do we get over this?
Intermediate & Advanced SEO | | Towelsrus0 -
301 redirect or Link back from old to new pages
Hi all, We run a ticket agent, and have multiple events that occur year after year, for example a festival. The festival has a main page with each event having a different page for each year like the below: Main page
Intermediate & Advanced SEO | | gigantictickets
http://www.gigantic.com/leefest-tickets (main page) Event pages:
http://www.gigantic.com/leefest-2010-tickets/hawksbrook-lane-beckenham/2009-08-15-13-00-gce/11246a
http://www.gigantic.com/leefest-2010-tickets/highhams-hill-farm-warlingham/2010-08-14-13-00-gce/19044a
http://www.gigantic.com/leefest-2011-tickets/highhams-hill-farm-warlingham/2011-08-13-13-00-gce/26204a
http://www.gigantic.com/leefest-2012-tickets/highhams-hill-farm-warlingham/2012-06-29-12-00-gce/32168a
http://www.gigantic.com/leefest-2013/highhams-hill-farm/2013-07-12-12-00 my question is: Is it better to leave the old event pages active and link them back to the main page, or 301 redirect these pages once they're out of date? (leave them there until there is a new event page to replace it for this year) If the best answer is to leave the page there, should i use a canonical tag back to the main page? and what would be the best way to link back? there is a breadcrumb there now, but it doesn't seem to obvious for users to click this. Keywords we're aming for on this example are 'Leefest Tickets', which has good ranking now, the main page and 2012 page is listed. Thanks in advance for your help.0 -
To canonicalize an old site to a VERY young one
Hi,
Intermediate & Advanced SEO | | Greywolf
The situation is I have an old e-commerce site with good Pagreank (13 yrs old) and a new 3 month old site. Currently with duplicate content because it's e-commerce site. The old site has two languages on each product page, the new site only English. Traffic to old site is half English, half the traffic is the other language.
Question is should I canonicalise to the new 3 month old domain that's only got English content at the moment? Soon it will also have both languages, but proper multi-lingual. The old site is meant to be for wholesalers, the new site for end-users. So the new site should get all the SE traffic it can get, I amnot worried about the old site not getting much because it has established userbase. If I canonicalise now, will I lose all the non English traffic to the old site? And is it a good idea to canonicalise to such a young domain? (on same IP C Block) What would you do?0