Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Moz can't crawl our site

Link Explorer

665

SigneerHFS last edited by

Moz can't crawl our site because of an error in the robots.txt, we've tried everything in the troubleshooting guide but nothing works - I believe its a server error but have no idea how to fix it pls help
1 Reply Last reply
Reply Quote 0
Natalie-Alexis Staff last edited by

Hi there!

We would be happy to look into this for you! Can you please send us an email over at help@moz.com so we can pinpoint what is going on here?

We would also need the URL of the site you are trying to crawl.

We look forward to hearing back from you soon!
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

How can i download all inbound links.

Currently i can only download 50K links, but we have around 350K links, how can i download all the inbound links.
Link Explorer | | ITWEBTEAM

0
Open Site Explorer external + follow links percentage

Hi how are you my root domain according to open site explorer has 90% of total links, external + follow. whereas my competitors have 5 - 6% how do I get this down to look more natural. Also what does this metric mean and how do I work out the percentage? Also I only have around 100 pages on my website which is a Shopify store and I have a small amount of internal followed links is this important for an ecommerce website as it is small number in comparison to my competitors. Thank you regards Adam
Link Explorer | | hourspy

1
Open Site Explorer Showing Blank Screen

Hi Moz Community, When I go to search for backlinks for any site, I am getting a blank screen after starting the search. This is happening to every search I perform for any URL or root domain. It's been happening for about a week now and I am not able to find any solutions. Any help would be greatly appreciated. Thanks! Ryan
Link Explorer | | RyanRhodesGW

0
Why doesn't Moz crawler follow robots.txt?

It is crawling the entire site, and there is stuff we do not want it to. Please advise.
Link Explorer | | Tylerj

0
Spam Score and crawling of my site

Hello, I'm trying to analyze the spam score of my site which is 9/17 Actually I have few backlinks and all of them have a low spam score (max 4/17, just one). I think there's some kind of issue with the crawler since I get strange spam factors: Large Site with Few Links (likely true, I recently deleted a lot of tags used once) Low Number of Pages Found (wasn't it a "Large Site"??) Low Number of Internal Links (I got a considerable number) No Contact Info (I have a link to my facebook in the menu and a "contacts" page) Thin Content (It's just a blog with min 300 words per post, why thin?) Site Link Diversity is Low (likely true) Ratio of Followed to Nofollowed Subdomains (likely true) Low MozTrust or MozRank Score (true) Ratio of Followed to Nofollowed Domains (likely true) Can you please help me to understand it, is it a crawling problem or similar? If needed I will post the url of the website. Thank you so much Marco
Link Explorer | | MarcoBP

0
What happened to my links in Open Site Explorer.

Hello, A while ago I had a problem with open site explorer saying that I had a problems with redirects on my home page. There was 302 redirect and it was giving me information from my www domain rather then from non www domain. With your awesome help, this issue is sorted now and when I type in my non www domain, it doesn't give me that weird message anymore. I have a different issue now. For some reason when I enter non www domain it doesn't show links from www version. I presume that this is affecting my site authority for non www version. Does it mean that link juice was not passed on to non www version from www version? Authority dropped by two points, but I'm not sure why. I'm more interested to know why it doesn't show my links from www version on non www version. And is that affecting my sites rankings. Thank you, Regards, Armands
Link Explorer | | A_Fotografy

0
Moz Spam Score

Hi! The spam score for my sites is "--" with no graph. The sites are large ecommerce sites with a ton of branded search and DAs of 50+. Simple question - are these clean spam scores, or has Moz not calculated the scores yet?
Link Explorer | | AMHC

1
Flush Open Site Explorer data

Hi folks Recently (about a month ago) we launched a new website. However, when I use Open Site Explorer and look at the "Top pages" tab, old URL's which no longer exist, appears in the list. Is there some way to get around this? Thanks.
Link Explorer | | jyskvin

0