Access denied in google webmaster tools
-
Hi I have just checked on my google webmaster tools and it is showing i 11 urls that are coming back as access denied. Now the urls are working, and they have been redirected using 301 redirect, so i have done everything right but for some reason google is not able to crawl them.
Does anyone know what i have done wrong for it to come back as access denied and how i can solve this problem. the site is www.in2town.co.uk many thanks
| | | |
| | 2 | Gardening/Gardening-Advice-What-is-Hydroponic-Gardening/menu-id-4991 | 403 | 4/11/13 |
| | 3 | Top-Showbiz-News/Super-Injunctions-Are-Right-Says-Hugh-Grant | 403 | 4/29/13 |
| | 4 | Entertainment-Tonight/Cheryl-Cole-wants-to-spice-up-The-X-Factor | 403 | 4/11/13 |
| | 5 | Tiger-Woods-paid-10000-a-time-for-sex | 403 | 4/20/13 |
| | 6 | Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/menu-id-4448 | 403 | 4/24/13 |
| | 7 | News-Showbiz/Doctor-Who-changed-my-life-says-Matt-Smith | 403 | 4/29/13 |
| | 8 | The-Latest-Health-News/Hypnosis-Hypnotherapy-for-Relationships/menu-id-4744 | 403 | 4/11/13 |
| | 9 | Soap-Gossip-Latest-News/Emmerdale-Marks-bit-on-the-side-comes-to-home-farm/menu-id-4615 | 403 | 4/11/13 |
| | 10 | news/eastenders/ | 403 | 4/11/13 |
| | 11 | entertainment-news/Prince-William-Stag-Do-To-Be-Held-in-Cape-Town | 403 | 3/24/13 || | |
-
cheers for this, i have contacted the company to see what they say about this issue and hopefully it will be resolved.
-
As I said the clue is likely in the message you get from the front end "Forbidden Access (flooding)"
If you search for this, the results all seem to mention joomla and that module. If you look through those results there are some mentions of the security features of this SEF module and how to turn them on/off. It is impossible to say if this is 100% the cause of your issue, but if your hosting company say everything is fine, and the message shown is specific to this joomla module, then it is a likely candidate. All things being equal, try turning off this security feature and see if the access denied errors in GWT go away.
-
just going into my webmaster tools and it says the following
| | Response Code | Detected |
| --- | --- | --- |<colgroup><col style="width: 45px;"><col style="width: 80px;"><col><col style="width: 120px;"><col style="width: 90px;"></colgroup>
| | 1 | Headlines-Celebrity-/Simon-Cowell-The-Wedding-Is-Back-On | 403 | 4/20/13 |
| | 2 | Gardening/Gardening-Advice-What-is-Hydroponic-Gardening/menu-id-4991 | 403 | 4/11/13 |
| | 3 | Top-Showbiz-News/Super-Injunctions-Are-Right-Says-Hugh-Grant | 403 | 4/29/13 |
| | 4 | Entertainment-Tonight/Cheryl-Cole-wants-to-spice-up-The-X-Factor | 403 | 4/11/13 |
| | 5 | Tiger-Woods-paid-10000-a-time-for-sex | 403 | 4/20/13 |
| | 6 | Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/menu-id-4448 | 403 | 4/24/13 |
| | 7 | News-Showbiz/Doctor-Who-changed-my-life-says-Matt-Smith | 403 | 4/29/13 |
| | 8 | The-Latest-Health-News/Hypnosis-Hypnotherapy-for-Relationships/menu-id-4744 | 403 | 4/11/13 |
| | 9 | news/eastenders/ | 403 | 4/11/13 |
| | 10 | entertainment-news/Prince-William-Stag-Do-To-Be-Held-in-Cape-Town | 403 | 3/24/13 ||
when i have looked at more info on this it says the following
Access denied errors
In general, Google discovers content by following links from one page to another. To crawl a page, Googlebot must be able to access it. If you’re seeing unexpected Access Denied errors, it may be for the following reasons:
- Googlebot couldn’t access a URL on your site because your site requires users to log in to view all or some of your content. (Tip: You can get around this by removing this requirement for user-agent Googlebot.)
- Your robots.txt file is blocking Google from accessing your whole site or individual URLs or directories.
- Test that your robots.txt is working as expected. The Test robots.txt tool lets you see exactly how Googlebot will interpret the contents of your robots.txt file. The Google user-agent is Googlebot. (How to verify that a user-agent really is Googlebot.)
- The Fetch as Google tool helps you understand exactly how your site appears to Googlebot. This can be very useful when troubleshooting problems with your site's content or discoverability in search results.
- Your server requires users to authenticate using a proxy, or your hosting provider may be blocking Google from accessing your site.
i have asked my hosting company about this and they say everything is fine.
any help would be great to solve this
| |
-
going to go through these today as they may have changed since the update. so you feel the sh404sef could be causing the blocking problems, i will contact them.
-
Hi Tim,
Well both those urls you give for the 301 are returning a 404, but I don't think they are the cause of your original problem which is the access denied issue. For that I am pretty sure you need to be looking at that joomla SH404SEF module.
-
hi both sides are there, for example from above,
Redirect 301 /Lingerie-Brands-expert-says-Lingerie-improves-your-sex-life /lingerie-helps-improve-your-sex-life
so i have the original page and then pointing to the destination page
i just do not understand after all the checks i have done why the error is happening
-
Hi Tim,
Your robots.txt looks ok from what I can tell. The 301s dont look odd (although what you have there is only one side of them right? I don't see the final page).
I think the clue is the message you get on the front end "Forbidden Access (flooding)". If you search for this phrase you start seeing references to the joomla SH404SEF module. See here for example: http://forum.joomla.org/viewtopic.php?p=1368937
I am not a joomla expert, but maybe it is a joomla issue instead of a server one, worth looking into.
-
they have also said my 301 redirects are causing the problems but i thought i had done this correctly
here are some of my redirects
The lines that are causing the issue are:
Redirect 301 /Jennifer-Aniston-upset-over-Brad-Pitt-Marriage /news/have-your-say/jennifer-aniston-upset-over-brad-pitt-marriage
Redirect 301 /In2town-Gossip/Liz-Hurley-Wants-Her-Husband-Back /news/have-your-say/liz-hurley-wants-her-husband-back
Redirect 301 /News-Celebrity/Take-That-and-Robbie-Williams-do-it-again /news/have-your-say/take-that-and-robbie-williams-do-it-again
Redirect 301 /Kevin-McCloud-does-not-like-the-word-Poverty /news/have-your-say/kevin-mccloud-does-not-like-the-word-poverty
Redirect 301 /Latest-Travel-News/Singapore-Tourist-Information-Singapore-a-must-for-Holidays/menu-id-4592 /news/holidays/singapore-tourist-information-singapore-a-must-for-holidays
Redirect 301 /Travel-Articles/Holiday-makers-are-rushing-to-buy-cheap-flights-to-Benidorm/menu-id-4998 /news/flight-news/holiday-makers-are-rushing-to-buy-cheap-flights-to-benidorm
Redirect 301 /The-Latest-Health-News/Stop-Biting-Your-Nails/menu-id-4744 /news/healthy-living/stop-biting-your-nails-with-hypnotherapy
Redirect 301 /Woman-celebrates-after-losing-weight-with-Weight-Loss-Hypnosis /news/gastric-band-hypnotherapy/woman-celebrates-after-losing-weight-with-gastric-band-hypnosis
Redirect 301 /Health-News-/-Stop-Smoking-Hypnosis-really-works-says-expert /news/health/stop-smoking-hypnosis-really-works-says-stop-smoking-expert
Redirect 301 /Animal-Health-News/Pet-Advice-Kennel-Cough-Advice-for-your-Pets/menu-id-4954 /news/dog-care/dog-kennel-cough
Redirect 301 /Travel-News/Flying-to-Australia-consumers-are-turning-their-back-on-Travel-Agents-over-cheap-flights/menu-id-4592 /news/flight-news/flying-to-australia-consumers-turn-their-backs-on-travel-agents-for-cheap-flights-to-australia
Redirect 301 /The-Latest-Health-News/Childbirth-Hypnotherapy/menu-id-4744 /news/health/childbirth-hypnotherapy
Redirect 301 /Travel-News/Brazil-Holidays-is-becoming-a-huge-hit-with-British-Tourist/menu-id-4592 /news/holidays/brazil-holidays-is-becoming-a-huge-hit-with-british-tourist
Redirect 301 /Celebrity-Gossip-Celebrity-News-and-latest-celebrity-gossip /Showbiz-Gossip
Redirect 301 /Latest-Travel-News/Travel-Magazine-reveals-secrets-to-finding-Cheap-Flights /news/flight-news/saving-money-on-flights
Redirect 301 /Lingerie-Brands-expert-says-Lingerie-improves-your-sex-life /lingerie-helps-improve-your-sex-life -
contacted my hosting company and they said the following robots file could be blocking google and causing the 403 errors, i thought it looked standard to me, can anyone please have a look and let me know if my robots file is causing the problem
many thanks
If the Joomla site is installed within a folder such as at
e.g. www.example.com/joomla/ the robots.txt file MUST be
moved to the site root at e.g. www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to the disallowed
path, e.g. the Disallow rule for the /administrator/ folder
MUST be changed to read Disallow: /joomla/administrator/
For more information about the robots.txt standard, see:
http://www.robotstxt.org/orig.html
For syntax checking, see:
http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/ -
thank you for this, what would be the best way to solve this, as this must be affecting my rankings
-
Hi Tim,
It looks like you have some sort of server setup that is trying to block dos attacks or similar. If you put your site into screaming frog after the first few pages it starts returning 403 errors (access denied). If you then look at a page in a browser your see the message: Forbidden Access (flooding).
Is it likely that this is happening when google is trying to spider the site also?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Weird 404 errors in Webmaster Tools
Hi, In a regular check with Webmaster Tools, I have noticed some weird 404 errors, for example, my domain URL is something like http://domainname.com/, the 404 error points to some weird URLs like http://domainname.com/james-bond&page=2/ and http://domainname.com/juegos-de&page=3/, at first I have tried to block them by robots.txt, but now I am getting these kind of 404 errors a lot, and don't think blocking them all is a perfect solution. Can anyone help me out with the issue? Thank you in advance.
Technical SEO | | nishthaj
cheers.0 -
Why google does not remove my page?
Hi everyone, last week i add "Noindex" tag into my page, but that site still appear in the organic search. what other things i can do for remove from google?
Technical SEO | | Jorge_HDI0 -
Webmaster tools Hentry showing pages that don't exist
In Webmaster Tools I have a ton of pages listed under Structured Data >> Hentry. These pages are not on my website and I don't know where they are coming from. I redid the site for someone and perhaps they are from the old site. How do I find and delete these? Thank you Rena
Technical SEO | | renalynd270 -
Google how deal with licensed content when this placed on vendor & client's website too. Will Google penalize the client's site for this ?
One of my client bought licensed content from top vendor of Health Industry. This same content is on the vendor's website & my client's site also but on my site there is a link back to vendor is placed which clearly tells to anyone that this is a licensed content & we bought from this vendor. My client bought paid top quality content for best source of industry but at this same this is placed on vendor's website also. Will Google penalize my client's website for this ? Niche is HEALTH
Technical SEO | | sourabhrana1 -
Links in Webmaster Tools that aren't really linking to us
I've noticed that there is a domain in WMT that Google says is linking to our domain from 173 different pages, but it actually isn't linking to us at all on ANY of those pages. The site is a business directory that seems to be automatically scraping business listings and adding them to hundreds of different categories. Low quality crap that I've disavowed just in case. I have hand checked a bunch of the pages that WMT is reporting with links to us by viewing source, but there's no links to us. I've also used crawlers to check for links, but they turn up nothing. The pages do, however, mention our brand name. I find this very odd that Google would report links to our site when there isn't actually links to our site. Has anyone else ever noticed something like this?
Technical SEO | | Philip-DiPatrizio0 -
Why is google webmaster tools ignoring my url parameter settings
I have set up several url parameters in webmaster tools that do things like select a specific products colour or size. I have set the parameter in google to "narrows" the page and selected to crawl no urls but in the duplicate content section each of these are still shown as being 2 pages with the same content. Is this just normal, i.e. showing me that they are the same anyway or is google deliberately ignoring my settings (which I assume it does when they are sure they know better or think I have made a mistake)?
Technical SEO | | mark_baird0 -
Disappeared from Google with in 2 hours of webmaster tools error
Hey Guys I'm trying not to panic but....we had a problem with google indexing some of our secure pages then hit those pages and browsers firing up security warning, so I asked our web dev to have at look at it he made the below changes and within 2 hours the site has drop off the face of google “in web master tools I asked it to remove any https://freestylextreme.com URLs” “I cancelled that before it was processed” “I then setup the robots.txt to respond with a disallow all if the request was for an https URL” “I've now removed robots.txt completely” “and resubmitted the main site from web master tools” I've read a couple of blog posts and all say to remain clam , test the fetch bot on webmasters tools which is all good and just wait for google to reindex do you guys have any further advice ? Ben
Technical SEO | | elbeno1 -
Google plus
With Google search plus your world, would i see results ONLY from Google plus followers ? or from someone who is my facebook friend as well.
Technical SEO | | seoug_20050
