403 Error on WMT but not on MOZ?
-
Hello,
2 days ago I found there are about 1200 of 403 errors by Google WMT when I tried to fetch my domain - Please see attached
HTTP/1.1 403 Access Forbidden Cache-Control: private Content-Type: text/html ETag: "" Server: Set-Cookie: ASPSESSIONIDSSBARTSD=BEHMJHJBKJOEJEALECNNIPFH; path=/; HttpOnly X-Powered-By: Date: Tue, 18 Feb 2014 13:54:10 GMT Content-Length: 1233
<title>403 - Forbidden: Access is denied.</title>
Server Error
<fieldset>
403 - Forbidden: Access is denied.
You do not have permission to view this directory or page using the credentials that you supplied.
</fieldset>
I ran a complete report using MOZ but I was shocked not see any 4xx , 5xx errors. Google: 246 of 404 errors
No Google, Yahoo or Bing blocking
HTTP status code: ALL 200
301 redirect: none? I have done about 2500 over 4 years.
The website is losing indexed pages. I'm not sure what's going and which numbers to trust.
Please help. Thank you.
Adam
-
Hi Adam!
Thanks for writing in to Q&A! Let's dive right into this nitty gritty!
There are a lot of reasons you could be seeing these data discrepancies. If you're seeing more GWT errors, it could be because they have 'discovered links' through different means. Moz starts with your home page and uses recursive crawling to find pages on your site. We keep crawling until we stop finding unique links, or until we hit your page crawl limit. Bottom line, when we crawl your site your homepage is the only seed. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way.
403 errors are agent specific. There's also a possibility that on a server level you're blocking WMT from crawling, but not rogerbot.
I looked into your account, and it looks like we're only crawling about 3,500 pages. Also, we generally limit our crawls to about 200 links per page. When I looked into your account, I saw that in most cases you had well beyond 200 internal links per page.
We have a few different options moving forward. To help you further I need examples of the following:
-specific URLs that show GWT errors, but don't show errors on Moz.
-a link hierarchy from that URL that goes back to your homepage.I want to respect your privacy, so if you'd like to take this conversation offline, please email us at help@moz.com!
I hope this helps! Let me know if you have any other questions!
Have a great rest of your Thursday!
Erin
-
It's difficult to say exactly what is wrong in this situation, but here are a couple of questions that might help narrow it down.
Have you tried manually going to the pages that appear to have these errors?
Did you recently change any URLs on your site?
Is the sitemap you submitted in WMT still valid?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Optimization Error
Hi, I am trying to track a page optimization feature for one of my project, https://www.writemyessay247.com for keyword: write my essay but i keep getting this below error: "Page Optimization Error There was a problem loading this page. Please make sure the page is loading properly and that our user-agent, rogerbot, is not blocked from accessing this page." I checked robots.txt file, it all looks fine. Not sure what is the problem? Is it a problem with Moz or the website?
Moz Bar | | j.rahimi19900 -
Moz Pro: Redirect Chain warning given to pages that don't have redirects
When I look up crawl errors for a page, I'm always told the page suffers from redirect chaining. However, when I do a redirect check (in this case, using the Redirect Path Chrome extension), it indicates that my page does not use a redirect. Why would Moz detect redirects, while no other redirect checker resource does? For example, this URL gets Moz's redirect chain warning: https://www.aem.org/news/january-2018/5-reasons-iot-projects-fail/ But there is no redirect associated with this URL.
Moz Bar | | jrichter0 -
Moz Crawl only crawling the top level page (1 page)
For the past few mounts my weekly site crawl has been inconsistent. One week works fine, it crawls all of my 500 or so pages. The following week it only crawls 1 page (http://mydomain.com) and nothing else. A few weekly scan go by and the crawl is back up the the 500 or so pages.I went ahead and created several campaigns with duplicate settings and crawled the site. Most times but not all the new campaign's crawl works fine crawling all pages. But within a week or two the weekly crawl will fail again. (crawling 1 page). Currently i have four campaign's all with the same settings running weekly crawls. 2 campaign's crawled the 500 pages and two crawled only the single page. Any help will be greatly appreciated
Moz Bar | | dmaude0 -
Moz Crawl - 804 : HTTPS (SSL) error encountered when requesting page.
Got an issue sending a Crawl Request to https://www.usernamebuddy.com/ " "804 : HTTPS (SSL) error encountered when requesting page." I have tried to recrawl several times now same issue keeps occurring. I cannot see an error when I access the site am I missing something, if so how can I diagnose the issue and sort the problem? I have reviewed the source and cannot use any http: resources.
Moz Bar | | GrouchyKids0 -
Moz Crawler fails on the first page
Hi guys, Can anybody shed some light, I'm running a crawl on a client's website but it's failing on the homepage and not crawling any other pages. It appears to be throwing an 804: HTTPS (SSL) error and then terminating the crawl. Now, the page in question was serving up mixed content up until about 4 days ago, but has since been fixed. I read that we should wait at least 48 hours before initiating another crawl to avoid hitting a cached version - which I did, but it still appears to be having issues. Is there anything specific I can do to get around this issue? I'm on a trial account and this feature is one I'm keen to test, so there is a bit of a time constraint. Any help is greatly appreciated! Thanks in advance!
Moz Bar | | philipdanielhayton0 -
Getting 'Sorry, but that URL is inaccessible' error msg when trying to run On-Page Grader
I just signed up for MOZ Pro for the first time today. Tried to run the 'on-page grader' tool on some of my pages but I'm getting a 'Sorry, but that URL is inaccessible' error msg. I have verified against the robot.txt file that the pages are NOT blocking any crawlers. Can anybody help?
Moz Bar | | spinoki0 -
Using Moz to find blogger with good PR.
Hi, My manager has recently informed me that you can use Moz in order to find blogger by category and their page rank. Is this possible? The reason for this is because we are looking for blogger with a good page rank and who are relevant to our sector, which is days out, outdoor activities, zipwires, adventure etc. We then want to contact them to see if we can invite them to write about our Tree Top Adventure and zip trekking activities. Any help would be much appreciated.
Moz Bar | | GoApe_20140 -
Moz Crawl Test: Referrer is sitemap.gz?
Hi,
Moz Bar | | Titan552
I'm looking at a crawl test report, and I'm seeing that most of the pages have the sitemamp.gz file listed as the referrer. As I recall in my other reports the referrer is usually the root domain - unless of course there's a redirect. Does having sitemap.gz as the referrer indicate a problem? If so, what problem does it indicate? Thanks!0