Site Crawl 1-page 301 status error but httpstatus.io says its 403
-
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301.
However when I run it in httpstatus.io it is giving me a 403 status error.
Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
-
Would this also block the Googlebot?
-
Hi John - Thanks for posting
Moz is starting at HTTP, so if you check httpstatus.io from HTTP, you will see a 301 to a 403 response. You would want to have your developer check to see if they are blocking our user-agent "rogerbot" or any AWS IP's because it doesn't look like we can crawl past the root domain.
If you need further assistance, feel free to reach out to help@moz.com.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hald the keywords in my keyword list say "Gethering Metrics". How do I get that to complete?
I'vwe waited about three hours, with no change. Why does this happen — is it something I did? Will those metrics ever be gathered? Or do I need to re-do buildingthe list all over again? And how can I be sure it will be any dirfferent?
Moz Bar | | btreloar0 -
Calling all 301 htaccess Guru's - www to non www - then to https + Redirect homepage to inner page
I have tried searching, multiple opinions and multiple things that supposedly work. What I have now, seems to work from an end user perspective, but Roger tells me otherwise: Redirect Chain issue....redirect, which redirects which redirects etc..... FIRST, we need to redirect all www to non www. SECOND, we need to redirect all to https. THIRD, we need to redirect the homepage to an inner page. (Got to love BOGUS DMCA complaints! :)?) So far we have: RewriteEngine on
Moz Bar | | Jes-Extender-Australia
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
RewriteCond %{HTTP_HOST} ^mydomain.com.au$ [OR]
RewriteCond %{HTTP_HOST} ^www.mydomain.com.au$
RewriteRule ^/?$ "https://mydomain.com.au/inner-page-here" [R=301,L] Plus down the page there is the usual wordpress settings: <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]</ifmodule> So, why does it seem to work for the end user, but Roger has his knickers in a knot saying, redirect, to redirect to redirect etc? Namaste and many thank you's in advance 🙂0 -
Weird 404 in Crawl Diagnostics
I'am getting a lot of 404 errors (196 to be precise ) - but their pattern is weird.
Moz Bar | | oorbo
The page that the crawler is trying to find is (e.g):
http://www.oorbo.com/item/asufa-israeli-design-shop**/www.oorbo.com.
the linking page is** http://www.oorbo.com/item/asufa-israeli-design-shop meaning it adds to the end of the link the root URL - /www.oorbo.com. This happens in all 196 cases - trying to find a page http://www.oorbo.com/some-page/www.oorbo.com from a refferer page http://www.oorbo.com/some-page. Obviously this pages do not exist, and it's getting a 404. I've look into the pages themselves and digged into their code - It doesn't seem that the bad link is any where on the page. Did anyone came across this kind of issue? any one can point me to a solution ?0 -
On-Page Grader Url is inaccessible
Hi everybody. I'm trying to use on -page grader for https://www.upscaledinnerclub.com and get "Sorry, but that URL is inaccessible." Robots.txt are empty, another thread on MOZ was talking about DNS check - it's all good. So, I can't figure out why this is happening. Also I am trying the same for another website https://www.regexseo.com - the same story. Common thing is that they both are on Google App Engine. And at first i thought that was the problem. Bu then i checked this one : https://www.logitinc.com/ and it's working, even though this website is on GAE as well. None of these website have robots.txt or any differences in setup or settings. Any thoughts?
Moz Bar | | DmitriiK0 -
My product pages have no weight / links from root domains with the Moz tool bar
Hi, When I view my home page (http://www.arkwildlife.co.uk) with the Moz toolbar, I see a good PA and a good amount of links from root domains. As I go down the site, it seems to get worse. The category pages (http://www.arkwildlife.co.uk/Category/0/Straight_Foods.html) have a little but not much and then from this point onwards, it's nothing. The product page (http://www.arkwildlife.co.uk/Item/Straight_Foods~Sunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html) is reporting to have no root domain links but I am not sure why. Interestingly, when I click through to a review page (http://www.arkwildlife.co.uk/StockReview/0/SUNH/0/Premium_Sunflower_Hearts.html) it does have some juice. Would anyone be able help on why this is happening and what I need to be looking at in order to resolve it please? EDIT: I've been looking at the hyperlinks and notice something odd. If I review the score with the first link below, it gives a score of 1, but the second gives a PA of 13 with one root domain linked. 1:http://www.arkwildlife.co.uk/Item/Straight_Foods~Sunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html 2:http://www.arkwildlife.co.uk/Item/Straight_Foods%7ESunflower_Seeds/SUNH/Premium_Sunflower_Hearts.html Please note the "%7E" instead of the "~" in the URL. The browser appears to show the ~ character no matter what but the rank of the page changes. I don't understand what the Moz toolbar is doing with this. Note: This behaviour only happens in Mozilla Firefox, in chrome both the rankings are zero for each URL. Many Thanks.
Moz Bar | | nawgie0 -
How can I find the old ERRORS and WARNINGS report in the NEW Moz design?
I'm looking for a complete list of errors and bugs that need to be fixed within a website. I used to use the MAIN tool (at least it seemed it was the most popular) but now that its just MOZ.com I can't seem to find that great report. It had data such as: 1. List of pages with Title Tags too long 2. List of pages with Description Tags too long 3. List of RED errors and YELLOW warnings, BLUE somethings... etc... Ring a bell? I LOVED this report, where can I find this data? Thanks! Derek
Moz Bar | | DerekM42420 -
How do I get Moz to crawl my site?
If I'm not mistaken, Moz will crawl my site once a week and let me know of any errors, notices etc.. But can I ask them to crawl once I have updated all the errors? or do I have to wait for them to do this automatically?
Moz Bar | | david.smith.segarra0 -
Moz just did my weekly "crawl" and claims my meta titles and descriptions are "missing," but they aren't. What's up with this?
I can check for my titles and descriptions in the source. They are there, always have been. what in the hell?
Moz Bar | | LifeisRosie0