Soft 404s for unpublished & 301'd content
-
Hi,
One site I work with unpublished a lot of thin content. Great idea, right?
These unpublished pages were then 301'd up to the main category page that they previously existed in.
Now Google Webmaster Tools calls them out as soft 404 errors. This seems unexpected since the pages
were 301'd. Here is my question; Is this a serious problem that may affect the site's overall organic results
and if so what should I do about it?
Thanks... Darcy
-
Short answer: create a custom 404 page, not just for these pages, but one that can show for everypage on your site.
A few resources:
https://support.google.com/webmasters/answer/93641?hl=en
Example: http://moz.com/sadfklfadsadfjs
-
Cyrus, thanks for hanging in there with my questions. If I just give back a 404, what am I showing them on the page?
I would think seeing the main questions page would be better than just sitting at the original url and looking at 404 page notice - seems like a bad user experience if Google wants to get all user-experiency about it.
Thanks... Darcy
-
Yes, it's possible, but that could be considered cloaking. I'd say best to return a 404.
-
Hi Cyrus,
Have not experienced a dip, but things have been a little static.
Can you do both... forward the page and give back a 404?
What would you do?
Thanks... Darcy
-
Yes, I would think that at the point Google crawls it and finds it forwarded it would drop it from the index and not waste resources crawling it again unless linked somewhere. I will keep an eye out for links, but don't believe that there are any.
Thanks, Dirk... Darcy
-
In that case, sounds like you should either:
- 404 them if you have evidence these have hurt your rankings/traffic (have you experienced a dip?)
- Ignore them and go about your day
-
Hi Cyrus,
Thanks for the info. These are forum pages where no one ever answered the question, so
there is no helpful info and very little content.
They were forwarded up to the main questions page (one / up the url structure).
The page they were forwarded to is like a questions category page, not specific to the subject of the
forwarded page. These forwarded pages don't get much/any traffic because they never ranked
and we didn't promote them.
If it doesn't hurt overall search on other pages, I'd rather not go to the substantial effort of finding subject-relevant pages to forward to, since no one will ever go to the original url and need to see something super relevant.
Your thoughts? Thanks! Best... Darcy
-
If Fetch like Google is also giving a 301 - I would mark them as solved in WMT & check if they re-appear.
If you click on the i next to the redirect message in Fetch like Google - it shows the type of redirect & the page it's redirecting to. I assume you checked that this is also a 301.I have a similar issue on one of my sites - if a user gets to a non-existing url - the server first tries to find out if the page exists - if it doesn't it's redirected to a 404 page. Although technically it is a 301 - WMT sees them as a soft 404 as the destination page is a "Page not found" type of page (called 404.php) - which (quite ironically) renders a 200 status.
On the destination page - do you mention somewhere a message like "page not found" or is it just a plain category page?
The SEO impact is difficult to assess - Google says these pages are mainly wasting the bot's time as it's indexing pages that do no longer exist, not sure if it is also affecting rankings. As you did the crawl with Screaming Frog, I guess you are also removing all internal links to these redirected pages? If these links disappear, and as the content was thin, I suspect you don't have many external links pointing to them, so the problem should disappear after a while.
rgds,
Dirk
-
If Google thinks the 301 leads to a page that isn't relevant enough, they may flag it as a "soft 404" even though it returns a 301. That's Google's way of saying they think you should 404 these pages instead.
How much will it hurt you? Probably not much, but it's hard to say.
Let's ask these questions:
- How much traffic goes to these pages? If not much, is it okay to 404 them?
- Are there more relevant pages you could redirect these to? (ideally, something with a similar title as the original page?)
- Have you seen much traffic loss overall? If not, it's likely this isn't hurting you.
Hope this helps! Best of luck with your SEO.
-
Okay, that is extra weird. It could be that GWT hasn't update your information since you made the changes. Since everywhere else is telling it's correct -- especially the fetch tool -- then you should wait a few more days and see if it updates.
-
Hi Erica,
I'm saying that the only place it shows a soft 404 is in GWT errors. Screaming Frog, web-sniffer and now Fetch As Google In GWT, all show them as 301 re-directs. I can't re-direct them more than they are. So, is GWT just goofy?
Thanks... Darcy
-
Hi Darcy,
Yeah, if it's still showing as a soft 404, there's still something wrong. I'd try using fetch and render as Google bot and see what happens.
Best of luck!
-
Hi Dirk,
Thanks for the suggestion. As noted above, I put the whole list thru screaming frog and a few thru your suggestion of web-sniffer.net.
95% of the whole list is 301s and 100% of the few put one at a time thru web-sniffer come back as 301s.
My question remains "Is this a serious problem that may affect the site's overall organic results
and if so what should I do about it?"
Thanks... Darcy
-
Hi Erica,
I put the list through screaming frog and 95% of the urls are shown as 301s.
Do you think screaming frog has it right or is there something they wouldn't catch?
Thanks... Darcy
-
Maybe an obvious question but did you check that the url's are indeed properly redirected - checking them with 'Fetch like Google' in WMT or by using a tool like web-sniffer.net?
rgds,
Dirk
-
I'd check to make sure your 301s were done correctly. If they are showing up as soft 404s, they are probably implemented wrong.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Tabbed Content Revisited
Hi-diddly-ho SEO gurus, quick question. I just saw this article and wanted to get thoughts from the people here. https://www.searchenginejournal.com/google-says-now-ok-put-content-behind-tabs/178020/ I am constantly at war with our UX guy on this subject because he believes, along with our CEO, that tabbed and accordion style information is better from THE UX standpoint. Less clutter on a page but with information still readily available. I am not here to argue that point but was wondering if you agree with the article posted here. I had to inform them their roll needed to be slowed until I could get something a little more concrete on the matter.
Intermediate & Advanced SEO | | spadedesign0 -
Technical 301 question
Howdy all, this has been bugging me for a while and I wanted to know the communities ideas on this. We have a .com website which has a little domain authority and is growing steadily. We are a UK business (but have a US office which we will be adapting too soon) We are ranking better within google.com than we do on google.co.uk probably down to our TLD. Is it a wise idea to 301 our .com to .co.uk for en-gb enquiries only? Is there any evidence that this will help improve our position? will all the link juice passed from 301s go to our .co.uk only if we are still applying the use of .com in the US? Many thanks and hope this isn't too complicated! Best wishes,
Intermediate & Advanced SEO | | TVFurniture
Chris0 -
Website with only a portion being 'mobile friendly' -- what to tell Google?
I have a website for desktop that does a lot of things, and have converted part of it do show pages in a mobile friendly format based on the users device. Not responsive design, but actual diff code with different formatting by mobile vs desktop--but each still share the same page url name. Google allows this approach. The mobile-friendly part of the site is not as extensive as desktop, so there are pages that apply to the desktop but not for mobile. So the functionality is limited some for mobile devices, and therefore some pages should only be indexed for desktop users. How should that page be handled for Google crawlers? If it is given a 404 not found for their mobile bot will Google properly still crawl it for the desktop, or will Google see that the url was flagged as 'not found' and not crawl it for the desktop? I asked a similar question yest, but it was not stated clearly. Thanks,Ted
Intermediate & Advanced SEO | | friendoffood0 -
Duplicate Content... Really?
Hi all, My site is www.actronics.eu Moz reports virtually every product page as duplicate content, flagged as HIGH PRIORITY!. I know why. Moz classes a page as duplicate if >95% content/code similar. There's very little I can do about this as although our products are different, the content is very similar, albeit a few part numbers and vehicle make/model. Here's an example:
Intermediate & Advanced SEO | | seowoody
http://www.actronics.eu/en/shop/audi-a4-8d-b5-1994-2000-abs-ecu-en/bosch-5-3
http://www.actronics.eu/en/shop/bmw-3-series-e36-1990-1998-abs-ecu-en/ate-34-51 Now, multiply this by ~2,000 products X 7 different languages and you'll see we have a big dupe content issue (according to Moz's Crawl Diagnostics report). I say "according to Moz..." as I do not know if this is actually an issue for Google? 90% of our products pages rank, albeit some much better than others? So what is the solution? We're not trying to deceive Google in any way so it would seem unfair to be hit with a dupe content penalty, this is a legit dilemma where our product differ by as little as a part number. One ugly solution would be to remove header / sidebar / footer on our product pages as I've demonstrated here - http://woodberry.me.uk/test-page2-minimal-v2.html since this removes A LOT of page bloat (code) and would bring the page difference down to 80% duplicate.
(This is the tool I'm using for checking http://www.webconfs.com/similar-page-checker.php) Other "prettier" solutions would greatly appreciated. I look forward to hearing your thoughts. Thanks,
Woody 🙂1 -
Brackets vs Encoded URLs: The "Same" in Google's eyes, or dup content?
Hello, This is the first time I've asked a question here, but I would really appreciate the advice of the community - thank you, thank you! Scenario: Internal linking is pointing to two different versions of a URL, one with brackets [] and the other version with the brackets encoded as %5B%5D Version 1: http://www.site.com/test?hello**[]=all&howdy[]=all&ciao[]=all
Intermediate & Advanced SEO | | mirabile
Version 2: http://www.site.com/test?hello%5B%5D**=all&howdy**%5B%5D**=all&ciao**%5B%5D**=all Question: Will search engines view these as duplicate content? Technically there is a difference in characters, but it's only because one version encodes the brackets, and the other does not (See: http://www.w3schools.com/tags/ref_urlencode.asp) We are asking the developer to encode ALL URLs because this seems cleaner but they are telling us that Google will see zero difference. We aren't sure if this is true, since engines can get so _hung up on even one single difference in character. _ We don't want to unnecessarily fracture the internal link structure of the site, so again - any feedback is welcome, thank you. 🙂0 -
Affiliate & canonicals
Hi, any help with this one would be great.... www.example.com sells widgets online. They are also promoted on a 3rd party website www.partner.com. Currently www.partner.com links to a page on www.example.com that is completely branded with the 'partners' design, style and unique copy (you would think you were still on 'partner' website). I saw this interesting article from 2011: http://www.seomoz.org/blog/getting-seo-value-from-your-affiliate-links (in particular idea 1) Do you think adding a rel=canonical on www.example.com's partner page is still safe? All the best & thank you, Richard
Intermediate & Advanced SEO | | Richard5550 -
How can I tell if a website is a 'NoFollow'?
I've been link building for a long time but have recently discovered that most of my links are from NoFollow links, such as twitter and Youtube. How can I tell if a website is a 'NoFollow'?
Intermediate & Advanced SEO | | Paul_Tovey0 -
Can literally any site get 'burned'?
Just curious what people think. The SEOMOZ trust on my site has gone up, all while Google is dropping us in rankings for lots of keywords. Just curious if this can happen to anyone or once you are 100% 'trusted' you're good. We went from 120,000 page views down to about 50,000. All while doubling content, improving the design(at least from a user perspective), and getting more natural links. Seems counter intuitive to Google's mantra of ranking quality. I would guess 'authority' sites never get hit by these updates right? So when you make it you've made it.(at least from a dropping like a rock perspective, obviously you have to keep working). I'm guessing we just need a bunch more quality links but would hate to work on building links, quality content, trust etc for it to be something so finicky long term.
Intermediate & Advanced SEO | | astahl110