Access denied in google webmaster tools
-
Hi I have just checked on my google webmaster tools and it is showing i 11 urls that are coming back as access denied. Now the urls are working, and they have been redirected using 301 redirect, so i have done everything right but for some reason google is not able to crawl them.
Does anyone know what i have done wrong for it to come back as access denied and how i can solve this problem. the site is www.in2town.co.uk many thanks
| | | |
| | 2 | Gardening/Gardening-Advice-What-is-Hydroponic-Gardening/menu-id-4991 | 403 | 4/11/13 |
| | 3 | Top-Showbiz-News/Super-Injunctions-Are-Right-Says-Hugh-Grant | 403 | 4/29/13 |
| | 4 | Entertainment-Tonight/Cheryl-Cole-wants-to-spice-up-The-X-Factor | 403 | 4/11/13 |
| | 5 | Tiger-Woods-paid-10000-a-time-for-sex | 403 | 4/20/13 |
| | 6 | Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/menu-id-4448 | 403 | 4/24/13 |
| | 7 | News-Showbiz/Doctor-Who-changed-my-life-says-Matt-Smith | 403 | 4/29/13 |
| | 8 | The-Latest-Health-News/Hypnosis-Hypnotherapy-for-Relationships/menu-id-4744 | 403 | 4/11/13 |
| | 9 | Soap-Gossip-Latest-News/Emmerdale-Marks-bit-on-the-side-comes-to-home-farm/menu-id-4615 | 403 | 4/11/13 |
| | 10 | news/eastenders/ | 403 | 4/11/13 |
| | 11 | entertainment-news/Prince-William-Stag-Do-To-Be-Held-in-Cape-Town | 403 | 3/24/13 || | |
-
cheers for this, i have contacted the company to see what they say about this issue and hopefully it will be resolved.
-
As I said the clue is likely in the message you get from the front end "Forbidden Access (flooding)"
If you search for this, the results all seem to mention joomla and that module. If you look through those results there are some mentions of the security features of this SEF module and how to turn them on/off. It is impossible to say if this is 100% the cause of your issue, but if your hosting company say everything is fine, and the message shown is specific to this joomla module, then it is a likely candidate. All things being equal, try turning off this security feature and see if the access denied errors in GWT go away.
-
just going into my webmaster tools and it says the following
| | Response Code | Detected |
| --- | --- | --- |<colgroup><col style="width: 45px;"><col style="width: 80px;"><col><col style="width: 120px;"><col style="width: 90px;"></colgroup>
| | 1 | Headlines-Celebrity-/Simon-Cowell-The-Wedding-Is-Back-On | 403 | 4/20/13 |
| | 2 | Gardening/Gardening-Advice-What-is-Hydroponic-Gardening/menu-id-4991 | 403 | 4/11/13 |
| | 3 | Top-Showbiz-News/Super-Injunctions-Are-Right-Says-Hugh-Grant | 403 | 4/29/13 |
| | 4 | Entertainment-Tonight/Cheryl-Cole-wants-to-spice-up-The-X-Factor | 403 | 4/11/13 |
| | 5 | Tiger-Woods-paid-10000-a-time-for-sex | 403 | 4/20/13 |
| | 6 | Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/Thousands-of-children-hurt-trying-to-stop-arguments-between-adults/menu-id-4448 | 403 | 4/24/13 |
| | 7 | News-Showbiz/Doctor-Who-changed-my-life-says-Matt-Smith | 403 | 4/29/13 |
| | 8 | The-Latest-Health-News/Hypnosis-Hypnotherapy-for-Relationships/menu-id-4744 | 403 | 4/11/13 |
| | 9 | news/eastenders/ | 403 | 4/11/13 |
| | 10 | entertainment-news/Prince-William-Stag-Do-To-Be-Held-in-Cape-Town | 403 | 3/24/13 ||
when i have looked at more info on this it says the following
Access denied errors
In general, Google discovers content by following links from one page to another. To crawl a page, Googlebot must be able to access it. If you’re seeing unexpected Access Denied errors, it may be for the following reasons:
- Googlebot couldn’t access a URL on your site because your site requires users to log in to view all or some of your content. (Tip: You can get around this by removing this requirement for user-agent Googlebot.)
- Your robots.txt file is blocking Google from accessing your whole site or individual URLs or directories.
- Test that your robots.txt is working as expected. The Test robots.txt tool lets you see exactly how Googlebot will interpret the contents of your robots.txt file. The Google user-agent is Googlebot. (How to verify that a user-agent really is Googlebot.)
- The Fetch as Google tool helps you understand exactly how your site appears to Googlebot. This can be very useful when troubleshooting problems with your site's content or discoverability in search results.
- Your server requires users to authenticate using a proxy, or your hosting provider may be blocking Google from accessing your site.
i have asked my hosting company about this and they say everything is fine.
any help would be great to solve this
| |
-
going to go through these today as they may have changed since the update. so you feel the sh404sef could be causing the blocking problems, i will contact them.
-
Hi Tim,
Well both those urls you give for the 301 are returning a 404, but I don't think they are the cause of your original problem which is the access denied issue. For that I am pretty sure you need to be looking at that joomla SH404SEF module.
-
hi both sides are there, for example from above,
Redirect 301 /Lingerie-Brands-expert-says-Lingerie-improves-your-sex-life /lingerie-helps-improve-your-sex-life
so i have the original page and then pointing to the destination page
i just do not understand after all the checks i have done why the error is happening
-
Hi Tim,
Your robots.txt looks ok from what I can tell. The 301s dont look odd (although what you have there is only one side of them right? I don't see the final page).
I think the clue is the message you get on the front end "Forbidden Access (flooding)". If you search for this phrase you start seeing references to the joomla SH404SEF module. See here for example: http://forum.joomla.org/viewtopic.php?p=1368937
I am not a joomla expert, but maybe it is a joomla issue instead of a server one, worth looking into.
-
they have also said my 301 redirects are causing the problems but i thought i had done this correctly
here are some of my redirects
The lines that are causing the issue are:
Redirect 301 /Jennifer-Aniston-upset-over-Brad-Pitt-Marriage /news/have-your-say/jennifer-aniston-upset-over-brad-pitt-marriage
Redirect 301 /In2town-Gossip/Liz-Hurley-Wants-Her-Husband-Back /news/have-your-say/liz-hurley-wants-her-husband-back
Redirect 301 /News-Celebrity/Take-That-and-Robbie-Williams-do-it-again /news/have-your-say/take-that-and-robbie-williams-do-it-again
Redirect 301 /Kevin-McCloud-does-not-like-the-word-Poverty /news/have-your-say/kevin-mccloud-does-not-like-the-word-poverty
Redirect 301 /Latest-Travel-News/Singapore-Tourist-Information-Singapore-a-must-for-Holidays/menu-id-4592 /news/holidays/singapore-tourist-information-singapore-a-must-for-holidays
Redirect 301 /Travel-Articles/Holiday-makers-are-rushing-to-buy-cheap-flights-to-Benidorm/menu-id-4998 /news/flight-news/holiday-makers-are-rushing-to-buy-cheap-flights-to-benidorm
Redirect 301 /The-Latest-Health-News/Stop-Biting-Your-Nails/menu-id-4744 /news/healthy-living/stop-biting-your-nails-with-hypnotherapy
Redirect 301 /Woman-celebrates-after-losing-weight-with-Weight-Loss-Hypnosis /news/gastric-band-hypnotherapy/woman-celebrates-after-losing-weight-with-gastric-band-hypnosis
Redirect 301 /Health-News-/-Stop-Smoking-Hypnosis-really-works-says-expert /news/health/stop-smoking-hypnosis-really-works-says-stop-smoking-expert
Redirect 301 /Animal-Health-News/Pet-Advice-Kennel-Cough-Advice-for-your-Pets/menu-id-4954 /news/dog-care/dog-kennel-cough
Redirect 301 /Travel-News/Flying-to-Australia-consumers-are-turning-their-back-on-Travel-Agents-over-cheap-flights/menu-id-4592 /news/flight-news/flying-to-australia-consumers-turn-their-backs-on-travel-agents-for-cheap-flights-to-australia
Redirect 301 /The-Latest-Health-News/Childbirth-Hypnotherapy/menu-id-4744 /news/health/childbirth-hypnotherapy
Redirect 301 /Travel-News/Brazil-Holidays-is-becoming-a-huge-hit-with-British-Tourist/menu-id-4592 /news/holidays/brazil-holidays-is-becoming-a-huge-hit-with-british-tourist
Redirect 301 /Celebrity-Gossip-Celebrity-News-and-latest-celebrity-gossip /Showbiz-Gossip
Redirect 301 /Latest-Travel-News/Travel-Magazine-reveals-secrets-to-finding-Cheap-Flights /news/flight-news/saving-money-on-flights
Redirect 301 /Lingerie-Brands-expert-says-Lingerie-improves-your-sex-life /lingerie-helps-improve-your-sex-life -
contacted my hosting company and they said the following robots file could be blocking google and causing the 403 errors, i thought it looked standard to me, can anyone please have a look and let me know if my robots file is causing the problem
many thanks
If the Joomla site is installed within a folder such as at
e.g. www.example.com/joomla/ the robots.txt file MUST be
moved to the site root at e.g. www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to the disallowed
path, e.g. the Disallow rule for the /administrator/ folder
MUST be changed to read Disallow: /joomla/administrator/
For more information about the robots.txt standard, see:
http://www.robotstxt.org/orig.html
For syntax checking, see:
http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/ -
thank you for this, what would be the best way to solve this, as this must be affecting my rankings
-
Hi Tim,
It looks like you have some sort of server setup that is trying to block dos attacks or similar. If you put your site into screaming frog after the first few pages it starts returning 403 errors (access denied). If you then look at a page in a browser your see the message: Forbidden Access (flooding).
Is it likely that this is happening when google is trying to spider the site also?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Google de-rank a website.
Hi, I was inspecting a website which is covering the topic of best wheelbarrow of 2021, it is a new website and and starts ranking on google. But, after few days it got de-rank automatically and Moz is also not showing any result to that. I was wandering why this just happened and what should I do if I made my website and will not face this kind of situation?
Technical SEO | | Moeen22330 -
Webmaster tools Hentry showing pages that don't exist
In Webmaster Tools I have a ton of pages listed under Structured Data >> Hentry. These pages are not on my website and I don't know where they are coming from. I redid the site for someone and perhaps they are from the old site. How do I find and delete these? Thank you Rena
Technical SEO | | renalynd270 -
Webmaster tools not showing links but Moz OSE is showing links. Why can't I see them in the Google Search Console
Hi, Please see attached photos. I have a website that shows external follow links when performing a search on open site explorer. However, they are not recognised or visible in search console. This is the case for both internal and external links. The internal links are 'no follow' which I am getting developer to rectify. Any ideas why I cant see the 'follow' external links? Thanks in advance to those who help me out. Jesse T7dkL5s T7dkL5s OkQmPL4 3qILHqS
Technical SEO | | jessew0 -
Google + and Google Knoladge Graph
I am trying to get things to match up for the company brand websearch and the Google + page and we have had it for years now The knowledge graph on Google is showing the map, address and name (shown in attached image), but is not linked to a G+ page, as when i click the "Are you the business owner?" its is trying to make me create a new G+ business page. Anyone have any ideas on this? Also does the wiki name have to be exact for it to show? As for phone number would that be coming from the DNS record as that is nowhere in the markup rich snippet or normal markup Thanks in advance LC9cWdG
Technical SEO | | David-McGawn0 -
Webmaster Tools Search Queries Data Drop
Hi I'm seeing a significant drop in search queries being reported for a client in GWT starting on the 7th Feb. I have seen a few articles on SERound Table etc saying that many are reporting probs like delays etc with GWT updating its data, such as these ones: https://www.seroundtable.com/google-webmaster-tools-data-stalled-19854.html https://www.seroundtable.com/google-webmaster-tools-analytics-data-19870.html However these seem to suggest the problem is simply a delay with displayed data being updated, in the case im looking at the data is up to date but showing an increasing decline. When i look at Analytics data though the data is completely different. For exmaple GWT says on the 21st Feb there were 23 impressions with zero clicks but Analytics says there were 6 clicks/sessions from organic search. I take it this means that there is a likely problem with GWT data and I shouldn't worry ? All Best Dan
Technical SEO | | Dan-Lawrence0 -
Is there a tool to find a strange link
hi, i have used screaming frog seo and it has brought up a link to another site we run but this is a strange link the link is http://www.in2town.co.uk/www.clairehegarty.co.uk/virtual-gastric-band-with-hypnotherapy i have gone through the site on the home page as this is where it is and i cannot find, so i was just wondering if there was a tool that i can use to pin point where it is. any help would be great
Technical SEO | | ClaireH-1848860 -
Google Sitelinks
Hello, Good afternoon. I am having a site issue with Sitelinks. For some reason when I search Google for the brand I represent "California Olive Ranch" Sitelinks are not being generated. When I search for "Cal Olive Ranch" our site links are being generated. Our domain is Californiaoliveranch.com. Is there a way to tell Google to to change the site links to match our domain and brand name? Is this something that can be done in Google Webmasters? Thank you very much for your help. Adam P
Technical SEO | | apost40 -
Help with google adsense
Hi i wonder if anyone can help me with google adsense. I am having trouble making money with google adsense. I have been altering my pages to try and get better results with google adsense but nothing works. my traffic at the moment is about 3000 visitors a day but this should be doubled to around 6000 a day within the next two months. here is the layout of a typical page and i would be grateful for any advice on how to alter it to make money with google adsense http://www.in2town.co.uk/showbiz-gossip/rihanna-news/rihanna-shocks-fans-over-her-sexy-body-claims
Technical SEO | | ClaireH-1848860