Site Crawl 1-page 301 status error but httpstatus.io says its 403
-
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301.
However when I run it in httpstatus.io it is giving me a 403 status error.
Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
-
Would this also block the Googlebot?
-
Hi John - Thanks for posting
Moz is starting at HTTP, so if you check httpstatus.io from HTTP, you will see a 301 to a 403 response. You would want to have your developer check to see if they are blocking our user-agent "rogerbot" or any AWS IP's because it doesn't look like we can crawl past the root domain.
If you need further assistance, feel free to reach out to help@moz.com.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
804 : HTTPS (SSL) error encountered when requesting page.
Hi there, I am attempting a crawl test and am getting "804 : HTTPS (SSL) error encountered when requesting page." with no further information to debug the issue. Site seems to be fine as far as SSL configuration goes. How can we get more debugging information? Thanks,
Moz Bar | | RickEH
Rick0 -
Site search solution?
Hey guys, since google site search is going away we are looking for an alternative. I saw that you guys are using a pretty nice looking site search solution. Can you tell me what it is?
Moz Bar | | snapstream0 -
Is the update site crawl feature following robot.txt rules?
I noticed that most of the errors would not be occurring if Moz's tool followed the rules implemented in sites robots.txt. Has anyone else seen this problem and do you know if Moz will fix this?
Moz Bar | | jamestown0 -
Measure the impact of SEO on a relaunch of a page
we have just re-launched one of the pages of our website with fresh content etc. Would like to know how we can use Moz to measure the changes and impact of the re-launch from an SEO standpoint.
Moz Bar | | ULaw0 -
Crawl Report Internal Links Count
We recently ran a crawl report on www.phase1tech.com. Some of the pages are coming back with a large amount of 'internal links'. These 2 pages for example are showing 800 internal links: http://www.phase1tech.com/Upcoming-Events
Moz Bar | | AISEO
http://www.phase1tech.com/Contact At best there are approximately 70 links on the page. Where is the 800 number coming from?0 -
On-page optimization
I have a list of the top 350 keywords sending volume to my site, sorted by volume. I am using your On-Page Optimization tool to look at the top 10 keywords and the grade for each of the relevant pages on the website. So for "hard wood flooring," I am searching for that term on Google and finding the first listing for my site lumberliquidators.com that comess up. Then I paste that page link into the On-Page Optimizer. Is this the best way to do this to determine performance for the most relevant page? Moz gave this keyword an F (home page) even though LL came up #2 in the organic Google rankings.
Moz Bar | | AlanJacob0 -
408 errors in crawl diagnostics
Best community, The Crawl Diagnostics Report of Moz gave our website a lot of 408 errors like below: <dl> <dt>Title</dt> <dd>408 : Error</dd> <dt>Meta Description</dt> <dd>408 Request Time-out</dd> <dt>Meta Robots</dt> <dd>Not present/empty</dd> <dt>Meta Refresh</dt> <dd>Not present/empty</dd> <dd>-----------------------------------------------------------------------</dd> <dd>The report has diagnosed a lot of these (around 320), even though we cannot reproduce the error (we cannot seem to find it ourself). </dd> <dd>2 questions relating to this: </dd> <dd>* Can you (the people of Moz) reproduce the errors manually? </dd> <dd>* Is it possible that it is a bug in the spider of Moz itself (too many spiders crawling at the same time)?</dd> </dl>
Moz Bar | | arjen.koedam0 -
Duplicate Page Content Report on MOZ
Hi, I am just wondering as to the accuracy of this report - does it pick up all the duplicate on page content? Or is there a limit? We have an ecommerce store with a lot of copied and pasted descriptions - just wondering if there is a limit on how much the moz crawler picks up? In other words, once we fix what MOZ has detected, will there be more detected because it is limited to display say up to 200?? Hope you understand what I mean. Thanks
Moz Bar | | bjs20100