803 Errors, how to deal with this?
-
Hello,
During my last two MOZ crawls over couple of hundred 803 errors showed up. Thought that this might stop, but nop still creeping up. Not sure what has caused this.
My server providers are WP-Engine and they already said that all good at their end.
Pretty much all of those errors are for photos on my blog. I'm a photographer.
I have a web guy as well, but he is not sure what to do now and how to get this fixed.
Website is a-fotografy.co.uk
Thank you and if someone could shed some light. I did research here already, but nothing what cover photo side.
Regards,
Armands
-
I have screaming frog too. But it crawls only website pages. All of those are fine.
Pretty much all errors I get are for images on my website. So basically for separate files.
Also wanted to ask.
Could this type of 8xx error problem be caused by one of the plugins in WP? Around that time I installed Optimise plugin (which was suggested by WP Engine to speed up my site)
Thanks Casey.
-
I use Screaming Frog and it's bad-ass. But you have to pay for it.
A free option for you may be Xenu Link Sleuth. Also a quality crawler.
-
Another thing I noticed that when all this 803/804 issue started my crawled page amount almost doubled.
I have a photography website with loads of photos. So it treats all photos as a separate pages and brings all these 8xx errors.
Should I be doing something about this?
Thanks,
A
-
Is there another tool I could use to crawl my whole website to understand if that is only MOZ problem or could be something bigger.
-
So errors keep creeping up and I have noticed significant drop in my website rankings too. So there is definitely a problem.
Seems like here is many threads on what is the problem, but no solution how exactly to fix it.
Even MOZ guys are shaking heads and not sure what to do about it.
Sad face me.
A
-
This is the one MOZ did tests with and they said that Roger can't crawl site because of that SNI support issue. I got A- grade so I guess all should be good.
Thanks. Casey.
-
I think the recourse here is simple. IGNORE IT.
Again, this is a "Moz Crawl" prominence issue -- they just assign a higher value to it than most. I seriously doubt this is hurting you. Go over to www.ssllabs.com and run your SSL configuration through that tool there. If you returns a high score, you're good.
-
So got the response from Moz about this issue and seems like I can't do anything about and it is common problem with MOZ Roger boot.
My guess this is for both 803 and 804 errors.
" Unfortunately at this time our crawler isn't compatible with Server Name Indication (SNI). In general, SNI is a totally acceptable security configuration, but our crawler simply isn't equipped to be able to handle these settings.
If you are able to disable the SNI configuration on your site, then we should be able to crawl it normally. While it's something that we'd love to be able to accommodate for the future, we don't yet have the resources to be able to make those changes. "
Spoke with WP-Engine guys and they said it would be madness to disable this security option as it would definitely broke websites.
Sad face me and not sure what to do now.
A
-
Hi there! Kristina from Moz's Help Team here.
I just wanted to quickly interject and say that Casey is right! Unfortunately, we do not use a static IP address or range of IP addresses, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. This approach gives us the best dynamic view of the web, but it can make us incompatible with some servers or hosting providers.
The best option I'd be able to suggest would be to identify our crawler by User-agent: rogerbot
I hope this helps! As always, you can connect with our team about any issues you're seeing in the tools by sending a note to help@moz.com.
Have a great day!
-Kristina -
Unfortunately, and I could be wrong here, but I believe that Moz doesn't use anything close to a static IP address for their crawl and instead uses thousands which constantly change when they run a crawl. It's all hosted from the cloud. So there is honestly no way to get a "range" for you to pass along.
Instead, all you can do I believe is make sure you whitelist the useragent which I believe is just "roger."
-
Quick question. My server guys asking with what IP address MOZ is crawling my site. WP-Engine is saying that maybe they have blocked that IP address.
How do I find this IP address?
A
-
Thank you Casey,
I have a web guy on board now and server guys too. Weird thing is that this just started now. Web guy was fixing some issues with code recently to speed up my site and working with Optimize plugin. Could be that something was affected there maybe.
Thanks,
Regards,
Armands
-
Hey Armands,
803 errors are the result of incomplete HTTP responses being received, using as a result of incomplete/incorrect SSL implementation on a site. Per the MOZ Learn about HTTP Errors in Crawl Errors report, here's how these are defined:
"804 errors result from a site with misconfigured SSL software. If Moz's crawlers cannot correctly interpret an SSL response for a home page, the crawl ends immediately."
Basically, these "can" be bad in that they result in an inefficient crawl and can end an actual crawl, immediately. Just like you should fix 4xx level errors since they can be bad for users, so too should you probably address these 804 errors since they can be bad for crawlers (specifically Moz though).
More information on this issue can be found in these previous MOZ threads:
1. https://moz.com/community/q/moz-crawler-has-been-finding-a-lot-of-803-and-804-errors
2. https://moz.com/community/q/how-do-fix-an-803-errorIn my experience, these are usually the result of "Mixed Content" issues with your SSL implementation. However, this is also NOT what most SEOs (myself included) would classify as a HIGH PRIORITY ISSUE. But for some reason, Moz continues to classify it as such.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why my website is giving 4xx error
I was analyzing the website link, my website is giving me 4xx error. Google search console in not giving such error 27b42282-a5f6-4ad0-956a-91838633d5ad-image.png Any suggestion will be helpful. The site is on wordpress
Link Explorer | | VS-Gary0 -
Getting Page Optimization Error
Hello, I am trying to use the On-Page grader and i keep getting Page Optimisation error: The url is : https://marketsmithindia.com/mstool/evaluation.jsp The page seems to load fine in multiple browsers. I also dont have any rogerbot user-agent related errors on other pages. Please help resolve. Regards, Neeraj
Link Explorer | | Neeraj-Khanna0 -
Account Error
Hey I have Moz account free trail for 30 days whenever I try to analyze website that is based on it show error please solve my problem.
Link Explorer | | cihiloj7770 -
How to Fix Repeating 404 Error on Blog
I've been getting this same 404 Error for a ton of pages on my blog (blog.twowayradiosfor.com) out of nowhere and I can't figure out how to fix it. I have about 500 of them that are experiencing the same issue (as shown in the image I've attached/linked to). It has the correct link, then the part that gets flagged as 404 adds a /TwoWayRadiosFor.com at the end, which is apparently the issue. Is there a reason these have just now appeared even though the blog posts are from years ago? Is there an easy way to fix it? Thanks, Sawyer dF6mUJQ
Link Explorer | | AllChargedUp0 -
612 : Page banned by error response for robots.txt
Hi all,
Link Explorer | | ME5OTU
I ran a crawl on my site https://www.drbillsukala.com.au and received the following error "612 : Page banned by error response for robots.txt." Before anyone mentions it, yes, I have been through all the other threads but they did not help me resolve this issue. I am able to view my robots.txt file in a browser https://www.drbillsukala.com.au/robots.txt.
The permissions are set to 644 on the robots.txt file so it should be accessible
My Google Search Console does not show any issues with my robots.txt file
I am running my site through StackPath CDN but I'm not inclined to think that's the culprit One thing I did find odd is that even though I put in my website with https protocol (I double checked), on the Moz spreadsheet it listed my site with http protocol. I'd welcome any feedback you might have. Thanks in advance for your help.
Kind regards0 -
How Do You Deal With Duplicate Content On A Retail Site
Hi Guys We make custom portfolios and boxes and every time we have a Moz site crawl, Rogerbot always returns a number of Duplicate Content Issues relating to different products. As per the image below (which I hope is visible?!) Rogerbot has flagged duplicate content onto one product that relates to 5 other different products. For instance there is duplicate content for an A4 Leather Portfolio and an A3 Leather Portfolio and an 11"x17" Leather Portfolio. I can't redirect or canonicalise to just the A4 Portfolio as they are all individually different products. The information on each page although similar, is relevant to each of the products, so rewriting a different blurb on each product page, will not be user friendly for our customers. I could ignore the duplicate content issues, but then that isn't good practise (and also makes for a very unsatisfactory looking Dashboard!) Any ideas?? Nick wzQLgBl
Link Explorer | | nick_HandCo1 -
Error getting your data in moz ose
Why am I receiving a "There was an error getting your data" in moz ose? Everything worked fine yesterday but now I'm having trouble getting link metrics for my site.
Link Explorer | | TitanDigital1 -
I have a robots.txt error on Moz but not on Google Webmaster tools. Wondering what to do.
For the site www.patrickwerry.com, I'm getting a DA of 1 and a Error Code 612: Error response for robots.txt However, when I check webmaster tools, it's showing no errors and allowing robots.txt for the domain. Is there anything I can do to fix the issue on the Moz side so I can get better data? If you can respond in layman's terms even better. 🙂 Not an SEO. Lisa
Link Explorer | | LisaGerber0