Access denied errors in webmaster tools
-
I notice today Ihave 2 access denied errors. I checked the help which says:
Googlebot couldn’t access a URL on your site because your site requires users to log in to view all or some of your content. (Tip: You can get around this by removing this requirement for user-agent Googlebot.)
Therefore I think it may be because I have added a login page for users and googlebot can't access it. I'm using wordpress and presume I need to amend the robots.txt to remove the requirement for google to log in but how do I do that?
Unless I'm misunderstanding the problem altogether!
-
Samuel
Mat's right - you need to speratae the "access denied" pages into two buckets;
-
those which you want indexed
-
those which you don't want indexed.
For those you want indexed, take them on a page by page basis to figure out why access is being denied. They might not all be because of password protection.
-Dan
-
-
Personally I wouldn't remove the requirement for googlebot if you want to protect that content. Once you do that content is public for two reasons:
1. Any nefarious user can instantly change their user agent and view your content
2. It will appear in the public search results
Much better would be to consider what is and isn't protected by a login. If you want it indexed then it should be seen as public and not behind a login. If it should be private, then protect it.
If you want it to appear in the search to attract people to the site in order to encourage them to register consider having cut down versions that are public and have a strong call-to-action on those to get people to register.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Do you know if there is a tool that can tell you if a url have backlink?
Hi, Do you know if there is a tool that I can check backlinks for thousands of URLs Thanks Roy
Intermediate & Advanced SEO | | kadut0 -
Hreflang Tags with Errors in Google Webmaster Tools
Hello, Google Webmaster tools is giving me errors with Hreflang tags that I can't seem to figure out... I've double checked everything: all the alternate and canonical tags, everything seems to match yet Google finds errors. Can anyone help? International Targeting | Language > 'fr' - no return tags
Intermediate & Advanced SEO | | GlobeCar
URLs for your site and alternate URLs in 'fr' that do not have return tags.
Status: 7/10/15
24 Hreflang Tags with Errors Please see attached pictures for more info... Thanks, Karim KQgb3Pn0 -
Google disavow tool
I have an algorithmic penalty on one of my websites. I never received a notification of a manual penalty in GWMT and even sent in a reconsideration request 6 months ago ad they told me their were no manual penalties on the website. I have cleaned up my link profile and what I could not clean up I sent in using the Google disavow tool a few days ago. I've heard to just wait if it's algorithmic or should I send in another reconsideration request for disavow links tool?
Intermediate & Advanced SEO | | MarkHIggins0 -
How long for Google Webmaster tools to update/reflect link changes
Hi all, Does anyone know or have experience of how long GWMT takes to update its data?, we did some work on our link profile back in October/November but are still seeing old links (removed) showing in GWMT. Thanks in advance,
Intermediate & Advanced SEO | | righty0 -
URL errors in Google Webmaster Tool
Hi Within Google Webmaster Tool 'Crawl errors' report by clicking 'Not found' it shows 404 errors its found. By clicking any column headings and it will reorder them. One column is 'Priority' - do you think Google is telling me its ranked the errors in priority of needing a fix? There is no reference to this in the Webmaster tool help. Many thanks Nigel
Intermediate & Advanced SEO | | Richard5551 -
Disavowel tool.
Hello, One of my sites has a strange link profile; it has 40000 in bound links but 30000 of them are from the site http://ourlipsaresealed.skynetblogs.be/ with the anchor text "haarstijl (2)" which is dutch for hairstyles. I haven't paid for or even asked for these links and I don't think its negative seo. I think they just set up a template with hundreds of links they thought were useful to their visitors and produce several pages a day. So the question is do I use the new google disavowel tool? I've held off so far because A. they link to a competitor who haven't been anywhere near as affected as we have although they seem to have been affected to an extent by a drop for some reason and they have a much better link profile overall than mine. and B. in the video Matt cutts goes on over and over that this tool is for people that have done some dodgy link building in the past but I haven't. Thanks, Ian
Intermediate & Advanced SEO | | jwdl0 -
Error 403
Hi SEOmoz community, Today, I checked the google webmaster tool of one of my clients, and ithere are 18 403 errors, I was wondering on how to fix those since it is the first time I come across these errors? How can I avoid that in the future? Thank you,
Intermediate & Advanced SEO | | Ideas-Money-Art0 -
403, 301, 302, 404 errors & possible google penalty
William Rock ran a Xenu site scan on nlpca(dot)com and mentioned the following: ...ran a test with Xenu site scan and it found a lot of broken links with 403, 301, 302, 404 Errors. Other items found: Broken page-local links (also named 'anchors', 'fragmentidentifiers'): http://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#null anchor occurs multiple timeshttp://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#US not found Could somone give us an output of that list, and which ones of these errors do we need to clean up for SEO purposes? Thank you.
Intermediate & Advanced SEO | | BobGW0