Moz could not crawl my httpS website
-
Hi,
we have a website with HTTPS, moz could not crawl it and we get "902 : Network errors prevented crawler from contacting server for page" while in logs we see moz robot access but fail after some seconds, what could be the problem, while moz can access site when it is without httpS
| 902 : Network errors prevented crawler from contacting server for page. |
-
I check and seems that your site is OK from mine side - GCE, Amazon EC2 and in Eastern Europe. There are few issues - SSL3, weak DH and files as: https://www.pouyasazan.org/IT-news/25876/css/css/css/css/css/css/css/css/css/css/css/css/css/tooltipster.css (breaking relative and absolute path), also no 404 page because everything return "200 OK" (soft 404). But they isn't critical for your site working and it works. Server works and i just hammering it with many requests.
Please contact with Moz on mail here: help@moz.com or fill form there: https://moz.com/help/contact/pro to investigate your situation.
-
Hi,
website is httpS://www.pouyasazan.org
-
Hi,
I have read those and investigated, those are not reasons, because first server logs shows following:
<code>Line 31915: 2015-12-23 09:23:27 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET /robots.txt - 80 - 54.167.7.207 HTTP/1.1 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 200 0 0 422 293 32 Line 31915: 2015-12-23 09:23:27 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET /robots.txt - 80 - 54.167.7.207 HTTP/1.1 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 200 0 0 422 293 32 Line 31933: 2015-12-23 09:23:30 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET / - 80 - 54.167.7.207 HTTP/1.0 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 302 0 0 362 302 33 Line 31933: 2015-12-23 09:23:30 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET / - 80 - 54.167.7.207 HTTP/1.0 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 302 0 0 362 302 33 Line 31953: 2015-12-23 09:23:46 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET /sitemap.xml - 80 - 54.167.7.207 HTTP/1.0 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 200 0 0 16407 313 74 Line 31953: 2015-12-23 09:23:46 W3SVC16 NEW-WEB-POUYASA 67.227.178.29 GET /sitemap.xml - 80 - 54.167.7.207 HTTP/1.0 rogerbot/1.1+([http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com](http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help,+rogerbot-crawler+pr2-starter-01@moz.com)) - - [www.pouyasazan.org](http://www.pouyasazan.org/) 200 0 0 16407 313 74 second crawl test shows following: http://www.pouyasazan.org HTTP Status Code 302 httpS://www.pouyasazan.org HTTP Status Code 0 this shows that your crawler could connect third, when i goto http://analytics.moz.com/search/on-page/4311859.1138194 i can optimiget results for keywork analys that shows your crawler could access my site</code>
-
Did you block AWS IP range or Mozbot? Or did you applied some firewall policy?
PM me your URL so i can take a look on server and network level with different tools.
-
Hello Hamed!
Here's a little more information from our Crawl Errors page:
_ 902 Unable to contact server_
The crawler resolved an IP address from the host name but failed to connect at port 80 for that address. This error may occur when a site blocks Moz's IP address ranges. Please make sure you're not blocking AWS.
This question has come up a few times here in Q&A, so I'd recommend going to the following post for further help in getting this all diagnosed:
https://moz.com/community/q/odd-crawl-test-issuesI hope this helps. Thank you!
Kevin
Help Team
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What does Location National means in Moz Pro Campaigns?
I understand the concept of locations in Moz Pro Campaigns. What does National exactly means? Is it like globally for a United States Google Search Rank or what? Please explain.
Getting Started | | ds9.tech1 -
Multiple campaigns for one website with multiple services?
Hi. Brand new to Moz Pro today. Question about whether to set up multiple campaigns for one website that promotes multiple services. I own a law firm and we have one main location, one website, 5 attorneys, and about 8-10 practice areas we promote. Each practice area has its own page and related content in the form of guides and blogs. Website is southernmarylandlaw.com. I only have this one website to track and promote, so I initially thought I only have one campaign. But I do want to track these separate practice areas separately. Each will have separate competitors, for instance. Each will need a separate set of keywords. People tend to search for a particular type of attorney, not just for the generic "lawyers." Should I use my 10 campaigns to set up a different campaign for each practice area we promote? Am I doing something wrong by tracking my one website 10 different times in my account? If I do not do this, then how do I possibly track the various practice areas and their separate keywords and competitors? Thanks for all your help and suggestions!
Getting Started | | tuckervc0 -
Crawl test
Can anyone give me an idea how to use the MOZ crawl test results...I'm a little confused on how to read it? I have a lot of "no's"...I think this is good?
Getting Started | | sdwellers0 -
My question is, when you translate your website to another language, does moz crawl both or do i have to add another campaign to moz so that they can crawl it seperately?
Hello, i recently translated my website to spanish, keywords,meta tags, content etc. All of our URL for the translated pages start "ep", which symbolizes espanol. My question is, does moz crawl those pages along with my english pages? or do i have add another campaign for moz to crawl my spanish pages seperately?
Getting Started | | prestigeluxuryrentals.com0 -
Does Moz provide more than just the SEO Tool?
We've received a lot of good info about our website using the Moz analytic tools but I'm curious to know if Moz provides consulting services on how to implement changes based on the data. Basically, Moz gave me the "what", I need some direction on the "how" and does Moz have the in house capabilities to assist with the "who" can do the actual work.
Getting Started | | Elara1 -
Creating a daily to-do list for SEO my website
Hi all, I'm very new here, and I'm quite slow in terms of learning things. Now I've had MOZ for about one and half month, but still only just understand to use on-page grade to SEO my site. I've been spending quite a bit money (for a small business) for online marketing: facebook adverts $50 a day; google adwords $10 a day; MOZ $99 a month... However, the conversion rate isn't ideal... only less than 2% Now what I'm trying to do is, while keep doing the cash burning online adverts, do more SEO on a daily basis. Can I please ask: other than optimise 5 pages' on-page grade, what else should I include into my daily SEO to-do list? Many thanks!
Getting Started | | alexlovesmoz0 -
New to using MOZ. Familiar with Google Analytics. With MOZ is there a code snippet to include on my site?
Just taken over web marketing responsibilities at my company. Will be doing some major website upgrades soon. I'm not familiar with MOZ and don't want to overwrite anything. So when setting up MOZ, is there a code snippet that goes anyplace on the site like there is with Google analytics? Thanks.
Getting Started | | NanoLumens0 -
Can MOZ crawl our website twice in a week?
I want to generate MOZ crawl errors report twice in a week. Is it possible to do that.
Getting Started | | chandman0