Google webmaster tools says access denied for 77 urls
-
Hi i am looking in google webmaster tools and i have seen a major problem which i hope people can help me sort out.
The problem is, i am being told that 77 urls are being denied access. The message when i look for more information says the below
Googlebot couldn't crawl your URL because your server either requires login to access the page, or is blocking Googlebot from accessing your site.
the responce code is 403
here is a couple of examples
http://www.in2town.co.uk/Entertainment-Magazine
http://www.in2town.co.uk/Weight-Loss-Hypnotherapy-helped-woman-lose-3-stone
i think the problem could be that i have sent them to another url in my httaccess file using the 403 re-direct but why would it bring up that google bot could not crawl them
any help would be great
-
Yup, deleted.
-
I have now deleted the old version can you check on this and make sure you can no longer see it.
-
You have a fairly complex .htaccess file (hint: I looked up your OLD .htaccess file - you should delete old htaccess files or something so people can't access them via a web browser), so I'm guessing the problem will be within your .htaccess file.
If possible, put a plain and simple .htaccess file on, test it with Google Webmaster Tools and see if the error still persists.
hi thanks for that. i will delete the old one now
-
In Webmaster Tools, you can "fetch as google bot" meaning you can enter one of those 77 URLs, and see what the Google "bot" sees when going to that URL.
You can also use:
http://www.dnsqueries.com/en/googlebot_simulator.php
For the URL: http://www.in2town.co.uk/Entertainment-Magazine
the Google Bot Simulator says:
HTTP CODE = HTTP/1.1 301 Moved Permanently
Location = http://www.in2town.co.uk/Showbiz-Gossip
and for: http://www.in2town.co.uk/Weight-Loss-Hypnotherapy-helped-woman-lose-3-stone
HTTP CODE = HTTP/1.1 301 Moved Permanently
Location = http://www.in2town.co.uk/Weight-Loss-Hypnotherapy
Interestingly, both the NEW URLs work fine although http://www.in2town.co.uk/Weight-Loss-Hypnotherapy doesn't look too good (at least in my web browser) but that's another issue.
You have a fairly complex .htaccess file (hint: I looked up your OLD .htaccess file - you should delete old htaccess files or something so people can't access them via a web browser), so I'm guessing the problem will be within your .htaccess file.
If possible, put a plain and simple .htaccess file on, test it with Google Webmaster Tools and see if the error still persists.
Adam
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
About how google works
Hey guyz,
Technical SEO | | atakala
I want to ask a basic question. If I search for Larry Page lets say.
I think google look for it's index for word larry and page distinct.
And mix it up. But the question ;
Can google show a result which only Larry exist on the page but any of the synonym or the stem of the Page not exist .
If it can happen how this page can be showned in larry page query. Thank you.0 -
Is rel=canonical needed for URLs with Google Analytics query strings?
If a page URL has Google Analytics query strings, does the page need a canonical tag? e.g., something.com/?utm_source=newsletter&utm_campaign=mar-2013-nsl I have rel=canonical on all our pages because some of them will be accessed via URLs that have non-Google strings. The strings are only for marketing purposes, not for identifying a specific page to display. e.g., something.com/?source=acme Should I only implement the canonical tag on the pages that might have non-Google marketing strings in the URL?
Technical SEO | | WayneBlankenbeckler0 -
Google webmaster tool doestn allow me to send 'URL and all linked pages"
Hello! I made a lot of optimization changes in my site ( seo urls, and a lot more ) , I always use Google Webmaster tools, fetch as Google Bot to refresh my site but now it doesnt allow me to 'Send URL and all linked pages' check the attachment Thank you
Technical SEO | | matiw0 -
How can I make Google Webmaster Tools see the robots.txt file when I am doing a .htacces redirec?
We are moving a site to a new domain. I have setup an .htaccess file and it is working fine. My problem is that Google Webmaster tools now says it cannot access the robots.txt file on the old site. How can I make it still see the robots.txt file when the .htaccess is doing a full site redirect? .htaccess currently has: Options +FollowSymLinks -MultiViews
Technical SEO | | RalphinAZ
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?michaelswilderhr.com$ [NC]
RewriteRule ^ http://www.s2esolutions.com/ [R=301,L] Google webmaster tools is reporting: Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 100.0%.0 -
Google Webmaster Sitemap *pending*
Hey guys, I've noticed that my sitemap has been "pending" for quite some time in Google Webmaster tools. This leads me to believe that Google is not indexing my site. Could someone help me and point me to what I'm doing wrong? My site is The Tech Block
Technical SEO | | ttb0 -
Bing Webmaster tool
Hi Fellas, I wanted to know once you verify the BIng Webmaster tool (via xml file) for a dev site, do you have to do the verification process again for the final site? I thought I needed to do the verification again but once I added the final website (which have almost a similar URL) into the webmaster tool account, it seemed that I didn't have to verify it. I am a bit confused. Thank you for clarifying
Technical SEO | | Ideas-Money-Art0 -
Duplicate pages, overly dynamic URL’s and long URL’s in Magento
Hi there, I’ve just completed the first crawl of my Magento site and SEOMOZ has picked up 1,000’s of duplicate pages, overly dynamic URL’s and long URL’s due to the sort function which appends URL’s with variables when sorting products (e.g. www.example.com?dir=asc&order=duration). I’m not particularly concerned that this will affect our rankings as Google has stated that they are familiar with the structure of popular CMS’s and Magento is pretty popular. However it completely dominates my crawl diagnostics so I can’t see if there are any real underlying issues. Does anyone know a way of preventing this? Cheers,
Technical SEO | | WendyWuTours
Al.1 -
Google plus
With a single Google search, you can see regular search results, along with all sorts of results that are tailored to you -- pages shared with you by your friends, Google+ posts from people you know. **Does pages shared by friends ** Does this mean pages shared by friends on Google plus ?
Technical SEO | | seoug_20050