Moz cannot crawl domain. Also OSE does not work properly on this specific domain?

[Guapa_zwolle]

Hi all,

Moz cannot crawl the domein http://www.hoesjescases.nl. 
When I open the crawl report I only see one line:

<colgroup><col width="229"><col width="287"><col width="420"><col width="370"><col width="141"></colgroup>
| URL | Time Crawled | Title Tag | Meta Description | HTTP Status Code |
| http://www.hoesjescases.nl | 2015-10-05T12:20:48Z | 404 : Received 404 (Not Found) error response for page. | Error attempting to request page; see title for details. | 404 |

Also when running OSE on this domain, Moz only can find 4 root domains while Majestic can find 91 domains.

Google seems not to have any problems. What can be the problem for MOZ?

Greetings!

[jorrito]

I have the same issue with my website: www.hoes.be 

[Targeras]

Thanks! I'll do that.

[ChiarynMiranda]

Hey There!

I'm sorry that you are having some trouble getting your site crawled. We are actually finding links in our index to unplag.com, because these links are found by crawling other sites rather than your actual site. However, we are unable to provide crawl diagnostics within your campaign because we are receiving a 400 bad request error from the site. You can also see that response using this header checker tool here: http://www.webconfs.com/http-header-check.php

In order to figure out why your site is returning this response, you would need to have your hosting provider investigate into your server.

I hope this helps! Please let me know if I can help you with anything else.

[Targeras]

Hello everyone.

I have the same issue with unplag.com. Both on-page grader and website crawl are not working.

Yes, I have tried different urls.

According to Charles Mazzie - moz index changes once a month so the problem can occur because a week ago I had Rogerbot disallowed in robots.txt.

Also, moz guide says that sometimes cookie policy of my site may cause RogerBot buzzing.

Unfortunately I still haven't solved the problem. So if you can - please help.

[ChiarynMiranda]

Hey There,

It looks like Donna answered your question about OSE, but I wanted to follow up about the 404 you are seeing in the crawl test for http://www.hoesjescases.nl. Unfortunately, the 404 response is what is being returned from your server to our crawler: http://www.screencast.com/t/HaQJ2zd0wiG4. I did check other HTTP checkers, like hurl.it, and I'm getting a 200 for your site, so the response does seem specific to our crawler. It may be that the webmaster intended to have the server return a 403 forbidden error or something like that (which we still wouldn't be able to crawl, but would be the correct error if we are purposefully blocked), but you would need to talk to your web host to resolve the issue.

I hope this helps! Please let me know if I can help you with anything else.

Chiaryn

[DonnaDuncan]

Yes, I see the same results.

If you're expecting to see more and wondering why not, that's a topic that frequently comes up in this forum. Here's one good explanation from member Charles Mazzie - https://moz.com/community/q/inbound-links-how-accurate-or-up-to-date-is-open-site-explorer. There are others you can find by searching Q&A.

[DonnaDuncan]

You sure you don't just have a typo in the URL? I just tried running your domain thru open site explorer and couldn't replicate the problem.