Why are "noindex" pages access denied errors in GWT and should I worry about it?
-
GWT calls pages that have "noindex, follow" tags "access denied errors."
How is it an "error" to say, "hey, don't include these in your index, but go ahead and crawl them."
These pages are thin content/duplicate content/overly templated pages I inherited and the noindex, follow tags are an effort to not crap up Google's view of this site.
The reason I ask is that GWT's detection of a rash of these access restricted errors coincides with a drop in organic traffic. Of course, coincidence is not necessarily cause.
Should I worry about it and do something or not?
Thanks... Darcy
-
I am a little surprised, because having those pages as "noindex, follow" should not bring GWT to flag them as errors.
Monica is correct in addressing google flag anything than 200 as errors, but... Your page with "noindex, follow" should return a HTTP code of 200. If it is returning anything else, it's probably wrong, and you should analyze why is doing it.
My religion has a law saying that GWT should return no errors, point. I have also witnessed few times a correlation between lowering GWT errors count to 0 and an improve in SERP ranking; but I have no proof one is causing the other.
-
I had a similar issue where my sitemap and my robots.txt didn't match properly and they were causing a slew of errors to show up. Everything falls under a crawler error but "should" clean itself up as its being indexed. I resubmitted an updated sitemap that matched my robots.txt and I have gotten rid of the errors.
Google also states that these errors don't directly hurt your ranking, but they can indirectly hurt because of user experience. You can always double check and see if the pages are being indexed by doing a "site:" search in google and checking if those pages exist.
Now, the errors are somewhat of a blessing. We had a design firm who redid our website and they had contracted an SEO "expert" to optimize the site before launch. They launched our website, and the next day I open up GWMT and our entire website was still under "noindex". The forgot to take the noindex from the dev site off of our main site.
Also I would consider just redirecting the thing content all together.
EDIT: And again Ryan sneaks in before me!!!!!!!!
-
Thumbs up to Monica's answer. I'd just add that you could redirect some of those pages to thin out the use of no index if possible, but it sounds like you've kept them around as they're marginally useful. You can also click the 'ignore' button for given error messages and they'll go away.
-
No. I wouldn't worry about it. Google calls them errors, the same as a 404 error. To them an error is anything that returns a code other than 200. I have hundreds of noindex pages on my site and it doesn't hurt. I believe it helps because it removes duplicate content and eliminates bad user experiences.
I have always thought that it is Google's way of double checking to make sure that the Webmaster is aware those pages are blocked. There have been times that I found URLs in there that weren't supposed to be, and contrarily found missing URLs as well. Its checks and balances in my opinion.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When i search for my domain name - google asks "did you mean" - why?
Hi all, I just noticed something quite odd - if i do a search for my domain name (see: http://goo.gl/LBc1lz) google shows my domain as first result, but it also asks "did i mean" and names another website with very similar name. the other site has far lower PA/DA according to Moz, any ideas why google is doing this? and more inportantly how i could stop it? please advise James
Intermediate & Advanced SEO | | isntworkdull0 -
Is there a tool to find out if a URL has been deemed "SPAM" by GOOGLE
I am currently doing a link audit on one of my sites and I am coming across some links that appear to be spam. Is there a tool that I can plug their URL into to see if they have been deemed spam by GOOGLE?
Intermediate & Advanced SEO | | Mozd0 -
Add noindex,nofollow prior to removing pages resulting in 404's
We're working with another site that unfortunately due to how their website has been programmed creates a bit of a mess. Whenever an employee removes a page from their site through their homegrown 'content management system', rather than 301'ing to another location on their site, the page is deleted and results in a 404. The interim question until they implement a better solution in managing their website is: Should they first add noindex,nofollow to the pages that are scheduled to be removed. Then once they are removed, they become 404's? Of note, it is possible that some of these pages will be used again in the future, and I would imagine they could submit them to Google through Webmaster Tools and adding the pages to their sitemap.
Intermediate & Advanced SEO | | Prospector-Plastics0 -
Received "Googlebot found an extremely high number of URLs on your site:" but most of the example URLs are noindexed.
An example URL can be found here: http://symptom.healthline.com/symptomsearch?addterm=Neck%20pain&addterm=Face&addterm=Fatigue&addterm=Shortness%20Of%20Breath A couple of questions: Why is Google reporting an issue with these URLs if they are marked as noindex? What is the best way to fix the issue? Thanks in advance.
Intermediate & Advanced SEO | | nicole.healthline0 -
Rel="canonical" questions?
On our site we have some similar pages for example in our parts page we have the link to all the electrical parts you can see here http://www.rockymountainatvmc.com/c/43/53/160/Electrical and we have a very similar page going from our accessories page to electrical here http://www.rockymountainatvmc.com/c/43/72/221/Electrical We are thinking about putting rel="canonical" from the accessories electrical page to the parts one. We would do this for several pages not just this one. Thoughts???
Intermediate & Advanced SEO | | DoRM0 -
RSS "fresh" content with static page
Hi SEOmoz members, Currently I am researching my competitor and noticed something what i dont really understand. They have hundreds of static pages that dont change, the content is already the same for over 6 months. Every time a customer orders a product they use their rss feed to publish: "Customer A just bought product 4" When i search in Google for product 4 in the last 24 hours, its always their with a new publishing date but the same old content. Is this a good SEO tactic to implant in my own site?
Intermediate & Advanced SEO | | MennoO0 -
Custom Error and page not found responses
When there is a 500 Internal Server Error, is it better to return an HTTP 500 response and custom error page from the requested URL, or is it better to return a 302 redirect? The redirect would send the browser to the custom error page, which would return the HTTP 500 result. We tell Google not to index or follow our error pages, so if Google sees an error at a URL, we don't necessarily want Google to think that the URL should be ignored. That's why the alternative would be to redirect to a custom error page with it's own URL. Similarly, what's the best approach if the response is a 404? Return HTTP 404 and custom 404 page from the requested URL, or redirect? Thanks.
Intermediate & Advanced SEO | | dbuckles0 -
Why my site is "STILL" violating the Google quality guidelines?
Hello, I had a site with two topics: Fashion & Technology. Due to the Panda Update I decided to change some things and one of those things was the separation of these two topics. So, on June 21, I redirected (301) all the Fashion pages to a new domain. The new domain performed well the first three days, but the rankings dropped later. Now, even the site doesn't rank for its own name. So, I thought the website was penalized for any reason, and I sent a reconsideration to Google. In fact, five days later, Google confirmed that my site is "still violating the quality guidelines". I don't understand. My original site was never penalized and the content is the same. And now when it is installed on the new domain becomes penalized just a few days later? Is this penalization only a sandbox for the new domain? Or just until the old URLs disappear from the index (due to the 301 redirect)? Maybe Google thinks my new site is duplicating my old site? Or just is a temporal prevention with new domains after a redirection in order to avoid spammers? Maybe this is not a real penalization and I only need a little patience? Or do you think my site is really violating the quality guidelines? (The domain is http://www.newclothing.co/) The original domain where the fashion section was installed before is http://www.myddnetwork.com/ (As you can see it is now a tech blog without fashion sections) The 301 redirect are working well. One example of redirected URLs: http://www.myddnetwork.com/clothing-shoes-accessories/ (this is the homepage, but each page was redirected to its corresponding URL in the new domain). I appreciate any advice. Basically my fashion pages have dropped totally. Both, the new and old URLs are not ranking. 😞
Intermediate & Advanced SEO | | omarinho0