What should I do with a large number of 'pages not found'?
-
One of my client sites lists millions of products and 100s or 1000s are de-listed from their inventory each month and removed from the site (no longer for sale). What is the best way to handle these pages/URLs from an SEO perspective? There is no place to use a 301.
1. Should we implement 404s for each one and put up with the growing number of 'pages not found' shown in Webmaster Tools?
2. Should we add them to the Robots.txt file?
3. Should we add 'nofollow' into all these pages?
Or is there a better solution?
Would love some help with this!
-
I would leave the pages up but mark them as "no follow". When I worked in eCommerce, this was a great tactic. For UX purposes, you could try to steer people to similar-products, but keep the originating page as "no follow" or "no index".
-
Thanks Jane and Lesley for your responses. Great ideas from you both. I think I'll keep the pages but change the content/buying options, as you've both suggested.
I had considered 410s and might fall back on this for historical URLs in the instance that we can no longer retrieve the content.
-
I always take notes from giants on how to handle things like this. Amazon is the giant in this arena, what do they do? They do not disable the product, they leave it on the site as unavailable. I would do the same thing personally. What platform are you using, does it have a suggested products module / plugin? If so, it can be modified to be more promient on pages that are disabled from selling. But I would keep the page and keep the authority of the page.
If you 301 it to another product, the search satisfaction level goes down and your bounce rate will rise. I would be careful with this, because Google wants to serve results that are relevant and what people are looking for.
The other option I would give is to return a 410 status code to get them de-indexed.
-
Hi Claire,
If you really can't 301, consider serving a page providing alternative products, a search function and an explanation of why the page's former content is no longer available. Many estate websites are quite good at this. Using real estate as an example, some maintain the URLs of properties that regularly go on the market (big city apartments, for example) but grey out the information to show a user that the property is not currently for lease. Other URLs will show properties in the former listing's post code.
Your robots.txt file is going to get out of control if you are having to add millions of pages to it on a regular basis, so I would personally not pursue that route.
-
Why aren't 301s an option?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
A large number of high spam links are negatively affecting my DA, how do I remove them?
I have identified a large number of very high spam score links to "free wallpaper" coming into my site.
Technical SEO | | beckygh
I am running a wordpress blog and would like some advice on the best course of action. There are thousands of spam domains linking to various images on my site with the anchor text "get free high quality hd wallpaper" The webmasters for these domains are not contactable so I am planning to submit a disavow file to google. I am aware these links have negatively affected my DA so would like to do more to remove them. My questions are: will deleting the images they link to help?
As this is on a wordpress site deleting the images will result in a soft 404, should I force a hard 404 to properly break the link?
Will this positively improve my DA?1 -
Why is the Page Authority of my product pages so low?
My domain authority is 35 (homepage Page Authority = 45) and my website has been up for years: www.rainchainsdirect.com Most random pages on my site (like this one) have a Page Authority of around 20. However, as a whole, the individual pages of my products rank exceptionally low. Like these: http://www.rainchainsdirect.com/products/copper-channel-link-rain-chain (Page Authority = 1) http://www.rainchainsdirect.com/collections/todays-deals/products/contempo-chain (Page Authority = 1) I was thinking that for whatever reason they have such low authority, that it may explain why these pages rank lower in google for specific searches using my exact product name (in other words, other sites that are piggybacking of my unique products are ranking higher for my product in a specific name search than the original product itself on my site) In any event, I'm trying to get some perspective on why these pages remain with the same non-existent Page Authority. Can anyone help to shed some light on why and what can be done about it? Thanks!
Technical SEO | | csblev0 -
Wordpress & use of 'www' vs not for webmaster tools - explanation needed
I am having a hard time understanding the issue of canonization of site pages, specifically in regards to the 'www' or 'non-www' versions of a site. And specifically in regards to wordpress. I can see that it doesn't matter whether you type in 'www' or not in the url for a wordpress site, what is going on in the back end that allows this? When I link up to google webmaster tools, should i use www or not? thanks for any help d
Technical SEO | | dnaynay0 -
Are these 'not found' errors a concern?
Our webmaster report is showing thousands of 'not found' errors for links that show up in javascript code. Is this something we should be concerned about? Especially since there are so many?
Technical SEO | | nicole.healthline0 -
Https-pages still in the SERP's
Hi all, my problem is the following: our CMS (self-developed) produces https-versions of our "normal" web pages, which means duplicate content. Our it-department put the <noindex,nofollow>on the https pages, that was like 6 weeks ago.</noindex,nofollow> I check the number of indexed pages once a week and still see a lot of these https pages in the Google index. I know that I may hit different data center and that these numbers aren't 100% valid, but still... sometimes the number of indexed https even moves up. Any ideas/suggestions? Wait for a longer time? Or take the time and go to Webmaster Tools to kick them out of the index? Another question: for a nice query, one https page ranks No. 1. If I kick the page out of the index, do you think that the http page replaces the No. 1 position? Or will the ranking be lost? (sends some nice traffic :-))... thanx in advance 😉
Technical SEO | | accessKellyOCG0 -
Handling 301s: Multiple pages to a single page (consolidation)
Been scouring the interwebs and haven't found much information on redirecting two serparate pages to a single new page. Here is what it boils down to: Let's say a website has two pages, both with good page authority of products that are becoming fazed out. The products, Widget A and Widget B, are still popular search terms, but they are being combined into ONE product, Widget C. While Widget A and Widget B STILL have plenty to do with Widget C, Widget C is now the new page, the main focus page, and the page you want everyone to see and Google to recognize. Now, do I 301 Widget A and Widget B pages to Widget C, ALTHOUGH Widgets A and B previously had nothing to do with one another? (Remember, we want to try and keep some of that authority the two page have had.) OR do we keep Widget A and Widget B pages "alive", take them off the main navigation, and then put a "disclaimer" on the pages announcing they are now part of Widget C and link to Widget C? OR Should Widgets A and B page be canonicalized to Widget C? Again, keep in mind, widgets A and B previously were not similar, but NOW they are and result in Widget C. (If you are confused, we can provide a REAL work example of what we are talkinga about, but decided to not be specific to our industry for this.) Appreciate any and all thoughts on this.
Technical SEO | | JU19850 -
Why the number of crawled pages is so low¿?
Hi, my website is www.theprinterdepo.com and I have been in seomoz pro for 2 months. When it started it crawled 10000 pages, then I modified robots.txt to disallow some specific parameters in the pages to be crawled. We have about 3500 products, so thhe number of crawled pages should be close to that number In the last crawl, it shows only 1700, What should I do?
Technical SEO | | levalencia10 -
Why do I have one page showing as two url's?
My SEOMoz stats show that I have duplicate titles for the following two url's: http://www.rmtracking.com/products.php and http://www.rmtracking.com/products I have checked my server files, and I don't see a live page without the php. A while back, we converted our site from html to php, but the html pages have 301's and as you can see the page without the php is properly redirecting to the php page. Any ideas why this would show as two separate url's?
Technical SEO | | BradBorst0