Site Crawl 1-page 301 status error but httpstatus.io says its 403
-
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301.
However when I run it in httpstatus.io it is giving me a 403 status error.
Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
-
Would this also block the Googlebot?
-
Hi John - Thanks for posting
Moz is starting at HTTP, so if you check httpstatus.io from HTTP, you will see a 301 to a 403 response. You would want to have your developer check to see if they are blocking our user-agent "rogerbot" or any AWS IP's because it doesn't look like we can crawl past the root domain.
If you need further assistance, feel free to reach out to help@moz.com.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why page load time is different in google webmaster vs what is displayed in moz?
When I analyze the site through Moz tool and compare the results with google webmaster, I am not able to figure what why Moz does not report the slow pages. Fro example this page has an avrage LCP of 3.0 sec https://www.collegehippo.com/graduate-school/programs/gre-score-business-analytics-data-analytics When I see the report in moz, it does not point to any such issue. Should I be worried about what google reports and try to fix the page?
Moz Bar | | etattva0 -
Limit MOZ crawl rate on Shopify or when you don't have access to robots.txt
Hello. I'm wondering if there is a way to control the crawl rate of MOZ on our site. It is hosted on Shopify which does not allow any kind of control over the robots.txt file to add a rule like this: User-Agent: rogerbot Crawl-Delay: 5 Due to this, we get a lot of 430 error codes -mainly on our products- and this certainly would prevent MOZ from getting the full picture of our shop. Can we rely on MOZ's data when critical pages are not being crawled due to 430 errors? Is there any alternative to fix this? Thanks
Moz Bar | | AllAboutShapewear2 -
Is the Content Suggestions section under Page Optimization a TF-IDF Analysis?
If you start a campaign in Moz, go to page optimization, enter a URL and keyword, and go to the bottom where it says "Content Suggestions" is that basically do a TF-IDF analysis? I want to make sure I understand how that works. Thanks!
Moz Bar | | brettmandoes0 -
I can't seem to get Moz Crawl to run? Re-bootyourbody.com. Told its a subdomain...What do I do?
I can't seem to get Moz Crawl to run? Re-bootyourbody.com. Told its a subdomain...What do I do?
Moz Bar | | Joseph.Lusso0 -
How do you block keywords in On-Page Grader for certain URLs?
For the on-page grader, I rank A's for 9 keywords. I struggle with F's I have because it is searching keywords on pages that are not supposed to be searched. For example, I have a "bracelets" page, so I didn't optimize it for "rings", so I got an F. However, it graded me an A for the bracelets keyword, which is great. To be sure I am correct, each page should have it's own keyword, such as bracelet. So why is the grader checking my "Bracelets" page for "Rings"? Maybe there is something I am missing. Just trying to see why this happening.
Moz Bar | | tiffany11030 -
Moz Dupe content crawl anomaly
Hi Moz has completed a crawl for a site i'm working on which also has a development area (hence with lots of dupe content) on a sub domain (and this dev area hasn't been hidden from crawlers via password, robots, gwt etc etc). Moz dupe content report is not showing any of these urls though even though my campaign setting is on 'root' domain so i would have thought report should be listing the subdomain urls as dupe content (because they are dupe content). Any ideas ? Cheers Dan
Moz Bar | | Dan-Lawrence0
