Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Moz can't crawl our site

Link Explorer

665

SigneerHFS last edited by

Moz can't crawl our site because of an error in the robots.txt, we've tried everything in the troubleshooting guide but nothing works - I believe its a server error but have no idea how to fix it pls help
1 Reply Last reply
Reply Quote 0
Natalie-Alexis Staff last edited by

Hi there!

We would be happy to look into this for you! Can you please send us an email over at help@moz.com so we can pinpoint what is going on here?

We would also need the URL of the site you are trying to crawl.

We look forward to hearing back from you soon!
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

How can I analyze my site?

Hello Moz communication, please how can I analyze my site https://bettyrecipe.com/ with moz tool and also run a link audit? So that I can see spammy backlinks?
Link Explorer | | loadedroms

1
Open Site Explorer Showing Blank Screen

Hi Moz Community, When I go to search for backlinks for any site, I am getting a blank screen after starting the search. This is happening to every search I perform for any URL or root domain. It's been happening for about a week now and I am not able to find any solutions. Any help would be greatly appreciated. Thanks! Ryan
Link Explorer | | RyanRhodesGW

0
Fastest way to get new backlinks recognised by Moz?

Hi all, I have seen a few discussions / advice pieces around this, but wondered what the current view is on the fastest way to get new backlinks recognised by Moz? I have seen people say that the content needs to be shared (e.g. on Twitter), and advice around creating Bitly links. I am not convinced that these techniques are working. On a related note, my website is very large. I have around 2.5m pages. Most of these pages are 3-4 clicks from the home page, but I have a small amount of important content much higher up in the site (1-2 clicks from the home page). Does Moz crawl pages nearer the home page more frequently, or does it make a pass of all pages on the site before it starts at the beginning again? If it makes a complete pass of all pages, then presumably making my site smaller will help to get Moz to see new content more quickly? Is there any other value to be gained in making a large site smaller? Many thanks in advance, Neil.
Link Explorer | | NHARRIS1

0
Spam Score and crawling of my site

Hello, I'm trying to analyze the spam score of my site which is 9/17 Actually I have few backlinks and all of them have a low spam score (max 4/17, just one). I think there's some kind of issue with the crawler since I get strange spam factors: Large Site with Few Links (likely true, I recently deleted a lot of tags used once) Low Number of Pages Found (wasn't it a "Large Site"??) Low Number of Internal Links (I got a considerable number) No Contact Info (I have a link to my facebook in the menu and a "contacts" page) Thin Content (It's just a blog with min 300 words per post, why thin?) Site Link Diversity is Low (likely true) Ratio of Followed to Nofollowed Subdomains (likely true) Low MozTrust or MozRank Score (true) Ratio of Followed to Nofollowed Domains (likely true) Can you please help me to understand it, is it a crawling problem or similar? If needed I will post the url of the website. Thank you so much Marco
Link Explorer | | MarcoBP

0
Why moz is not showing My Website External Backlinks?

Google Webmaster showing my external links 41
Majestic showing my external links 28
Ahrefs showing my external links 13
&
Moz is showing my external links 0... Will anyone please explain Why moz is not showing My Website External Backlinks?
Link Explorer | | djsamrock

0
Error message coming up for Open Site Explorer

When in Open Site Explorer there seems to be an error getting the data for http://vagabondtoursofireland.ie/ or www.vagabondtoursofireland.ie I have used this with other websites and have never had a problem. Thanks.
Link Explorer | | Johnny_AppleSeed

0
I have the trial of MOZ right now but the open site explorer hasn't updated since Dec 4th

What do you do to get the lasted information?
Link Explorer | | premerevents

0
Flush Open Site Explorer data

Hi folks Recently (about a month ago) we launched a new website. However, when I use Open Site Explorer and look at the "Top pages" tab, old URL's which no longer exist, appears in the list. Is there some way to get around this? Thanks.
Link Explorer | | jyskvin

0