Do I need to redirect soft 404s that I got from Google Webmaster Tools?
-
Hi guys,
I got almost 1000+ soft 404s from GWT. All of the soft 404s produce 200 HTTP status code but the URLs are something like the following:
http://www.example.com/search/house-for-rent
(query used: house for rent)
http://www.example.com/search/-----------rent
(query used:-------rent)
There are no listings that match these queries and there is an advanced search that is visible in these pages.
Here are my questions:
1. Do I need to redirect each page to its appropriate landing page?
2. Do I need to add user sitemap or a list of URLs where they can search for other properties?
Any suggestions would help.
-
Thanks guys for your inputs. By the way, this issue is already resolved last year. Thanks again!
-
It depends what you want to achieve. If the 404s are pages which no longer exist than it will be the fastest to use the GWMT removal tool to remove the page pattern and also add a noindex in robots.txt. In addition obviously returning a 404.
The soft 404 is a case where content is not found but HTTP-status 200 is returned - this needs to change if you currently serve non-existing pages.
We generally do the following:
- Content which we know does not exist anymore (i.e. a deleted product page or a deleted product category) is served with a SC_GONE (410) and we provide cross-selling information (i.e. display products from related categories). This works great and we have seen a boost in indexed content.
- URLs which don't exist will go through a standard 404 - this is intentional as our monitoring will pick this up. If it is a legitimate 404 put of SEO value, we will do a redirect if it makes sense, or just let Google drop it over time (takes sometimes up to 4 weeks)
You can have multiple versions of 404 pages, but this would need to be coded out - i.e. in your application server you would define 404page which then programmatically would display content depending on what you want to do.
-
I know I am way late to the party, but MagicDude4Eva, have you had success just putting a noindex header on the soft 404 pages?
That sounds like the easiest way to deal with this problem, if it works, especially since a lot of sites use dynamic URLs for product search that you don't want to de-index.
Can you have multiple 404 pages? Otherwise redirecting an empty search results page to your 404 page could be quite confusing..
-
Hi mate,
I already added the following syntax to my website's robots.txt:
User-agent: *
Disallow: /search/
I have checked the dynamic pages or URLs produced by search box (ex.http://www.domain.com/search/jhjehfjehfefe) but they are still showing in Google.com and there's still 1000+ soft 404s in my Google webmaster tools account.
I appreciate your help.
Thanks man!
-
I think if it is done carefully it adds quite a lot of value. A proper site taxonomy is obviously always better and more predictable.
-
I would never index or let google crawl search pages - very dangerous ground.
-
I would do the following:
- For valid searches returned create a proper canoncial URL (and then decide if you want to do a index,follow or a noindex,follow on the result pages). You might not necessarily want to index search results, but rather a structure of items/pages on your site.
- I would generally not index search results (rather have your pages being crawled through category structures, sitemaps and RSS feeds)
- It does sound though that the way you implemented the search is wrong - it should not result in a soft 404 - it could be as easy as making the canonical for your search just "/search" (without any search terms) and if no results are found display options to the user for search refinements
The only time I have seen soft 404s with us is in cases where we removed product pages and then displayed a generic "product not available" page with some upselling options. In this case we set a status of 410 (GONE) which resolved the soft 404 issue.
The advantage of the 410 is that your application makes the decision that a page is gone, whereas a 404 could really be just a wrong linked URL.
-
Yes Customize 404 whenever your database don't have have search results for user query then you can redirect them to that page.
Have you considered of blocking "search" results directory in Robots.txt because those pages are dynamic, they are not actually physical page so its better you block them.
-
What do you mean by default page? Is it a customized 404 page?
Thanks a lot man! I appreciate it.
-
Hi,
As per your URL, I think best solution is to block "search" directory in Robots.txt, then Google will not able to to access those pages so no error in GWT. OR you can also create default page for query which don't have any result in database.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is this is Wow HIT ME IN THE Face Google bug or am I missing something?
We have a page on our site https://www.spurshelving.co.uk/shop/bigimage.aspx?m=353&i=3436 which enders happily on all browsers as far as I am aware and is reasonably well optimised. So when google sent me a link to a new test tool I just had to check it out. https://testmysite.withgoogle.com/intl/en-gb/?utm_source=awareness&utm_medium=email&utm_campaign=tmsv1awareness&utm_content=header Well the result was shocking...... The page that renders in the results is a default missing product page and not the page that the link renders on a web page. I played a little and simply used the I=3436 attribute and the page appeared no problem I then reversed the attributes so that they were i=3436&m=353 and the page again resolved totally as expected. This indicates to me that Google have an issue with aspx attributes. Now I know what to do but is this same issue an issue in spidering and indexing pages. If is is wow that is a big smack in the face. Does it also harm search results in other engines. Keen for comments here
Web Design | | Eff-Commerce0 -
Site appears then disappears from Google
Hi, This is my first post to the Moz community. I hope someone can help me, as I'm at my wits end 🙂 Since mid to late April, my client's home page keeps dropping from Google rankings on and off on the desktop in incognito and non-incognito browsing. It seems to always be visible on mobile phones though. One day it's on page 4, the next day it's not anywhere to be found (even though some of her other pages rank). The site is here: evgeniaribinik.com I have done some blog writing (just text) for this client over the last few months, and recently she asked for some SEO help. After looking at her website, I noticed that she had a plugin called WordPress SEO on her site. She wasn't using this for the blogs, however. I don't think she did much of anything with SEO, but she did say that for the last few years, she was always on page 3 or 4 of Google. In mid/late April, she saw that she wasn't ranking at all for the keyword "boudoir photography nyc" anymore, despite a few years of ranking for it. I told her Yoast would be good to use. However, after she installed it, the same issues keep happening. Right now, she has Yoast and WordPress SEO plugins installed (I'm not sure if this is causing an issue as well). But I really can't figure out why she keeps going on and off page 4. She also asked me to optimize older blogs that she wrote herself for SEO. When I look at them, they don't have meta descriptions, good titles or good keywords. I realize this is hurting her, but why would her site be fine for years and all of a sudden not now? Thank you in advance for any help you can give me! Jill
Web Design | | lobeng770 -
Wordpress themes causing google penalty(need experts to settle a debate)
Hi, I have been having a disagreement with another online marketing company. We are both promoting the same product under a different brand name but we ended up using the same theme to build our WordPress sites off of but in no way is the content the same. They are telling me that using the same theme in the same industry will cause a Google penalty. I do not believe this and do not see this causing a problem. The sites are relatively new so there is no proof of traffic dropping or penalties as of yet. What is everyone's professional opinion on this? Can a WordPress theme cause duplicate content penalty? If so would that not mean that anyone using themes will have some sort of penalty?
Web Design | | impact891 -
Is it necessary to Remove 301 redirects from Wordpress after removing the 404 url from Google Webmaster?
There were many 404 urls in my site found by Google Webmaster. I've redirected these urls to the relevant urls with 301 redirect in wordpress. After that I removed these 404 urls from Google Index through Webmaster. "Should I cleanup these 301 redirects from Wordpress or not? ". Help Needed.
Web Design | | SangeetaC0 -
Display a message after 301 redirect.
I know this one is a little more technical than pure SEO, but it somewhat ties in. I'm redirecting 2 sites to one. The companies have combined, and I've imported all their content that has been created, and the url mapping is set up correctly. So it currently is set up like this: A.com --> C.com
Web Design | | WilliamBay
A.com/blogpost-1 --> C.com/blogpost-1 B.com --> C.com B.com/blogpost-20 --> C.com/blogpost-20 What i would like is some method a redirect when landing at C.com or C.com/blogpost-XX. And in turn display a message in a DIV so I can style it. And yes, I have nothing better to do on a Friday night 🙂0 -
How to make Address Text Clickable for Google Map Link for Mobile Device
How do I make the address text on the site a clickable link for mobile devices?
Web Design | | bozzie3110 -
Why is our sitemap not being indexed on Webmaster Tools?
Hi there, We have been having a problem with one of our websites. It appears as though someone has stolen our template and used it for themselves, but in the process also stole our analytics information. We have problems with the analytics, but are fixing that ourselves. The problem we have now is that when we tried to put in a sitemap into Google Webmaster Tools the URLs are submitted but have yet to be indexed. We have tried pinging them, but there has been no change. This is not a problem for our other websites which are very similar. What could be the problem here? For reference, the url is http://www.dentistinlittlerock.com Thank you for your responses in advance!
Web Design | | jid0