Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

"Our crawler was not able to access the robots.txt file on your site."

Product Support

792

K3Syspro last edited by

Hi Mozzers!

I've received an error message saying the site can't be crawled because Moz is unable to access the robots.txt. I've spoken to the webmaster and he can't understand why the robot.txt can't be accessed as this seems to be fine:

https://k3syspro.com/robots.txt

and Google isn't flagging anything up to us.

Does anyone know why this may be?

Thanks,

Matthew
1 Reply Last reply
Reply Quote 0
wholewhaler Subscriber last edited by

Hey K3,

Looks like you have bigger fish to fry, your host domain is throwing a 503 server error. No server = no content = no robots.txt will be able to be found until that is resolved. Error still happening as of this post: http://downforeveryoneorjustme.com/https://k3syspro.com
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Unsolved No replies from help@moz.com - one of our IPs is blocked by Cloudflare so we cannot access Moz Community from there

error access denied help

Hi all,
I am a bit at my wits end trying to get some acknowledgement from MOZ. Have had no replies, no ticket auto-replies, no updates on any of the messages I have sent via the Moz Help Form on the website. Literally nothing. I wanted to avoid having to post publicly, but does anyone know how to raise a "technical problem" ticket with MOZ? help@moz.com never replies and the Help Form doesn't generate any kind of ticket. From our main office we get an "Access denied" Error (via Cloudflare) specifically for the Moz Community area. This happened to us in February of this year and has been happening again all through May. After testing ourselves with our IT, we determine that MOZ's Cloudflare account has incorrectly blocked the dedicated IP address specific to the internet connection at our head office. This means that none of our Moz User accounts can access anything related to the Community area in our account when working at the studio. We can only do so when working remotely (ie. some other IP address). This is incredibly frustrating, particularly as we've been on a proper paid MOZ account for many years. And I have sent numerous email requests, messages via the Form, etc., and have never heard back from anyone at all. The problem has been on-going for some time and I guess it is my fault because I tried to politely wait a fair amount of time between each follow-up. Only to realize that, actually, I don't think anyone is monitoring help@moz.com or even the Form submissions, or are even looking into the issue for me. Am hoping this message is seen by someone at Moz so they can let me know what is going on please? Guys..... c'mon.....
Product Support | | DanielDL

0
Site DA and PA is not increasing

Hy, My site best water softener DA is Still 1. I am Publishing the content regularly, Site age is 6 months But Still DA and PA are 1. Why its not increasing. Is there any way to increase DA and PA Of my site?
Product Support | | ikzayahot

0
If I cancel, will I lose all my historical data & be able to access it?

I would like to still be able to view my historical data once I cancel. Will I still have access to it? Thanks.
Product Support | | StudioX

1
Site down: "high CPU usage is due to the large traffic generated from Moz

My client's site is down and the web host gives says that Moz is the reason why. "The fact that your site was limited is because the traffic generated by Moz. This is why I have suggested to block their IP addresses." Now we have unblocked the IP addresses and as you can see your site was limited again. And again the : Code: 54.224.139.99 - - [26/Oct/2017:16:00:43 -0500] "GET /amp/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/ HTTP/1.0" 200 58551 "-" "rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler@moz.com)"
54.224.139.99 - - [26/Oct/2017:16:01:02 -0500] "GET /amp/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/ HTTP/1.0" 200 58521 "-" "rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler@moz.com)"
54.224.139.99 - - [26/Oct/2017:16:01:16 -0500] "GET /amp/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_ HTTP/1.0" 301 - "-" "rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler@moz.com)"
54.224.139.99 - - [26/Oct/2017:16:01:30 -0500] "GET /amp/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/page/2/@Smile_Design_/@Smile_Design_/page/2/@Smile_Design_/ HTTP/1.0" 200 58528 "-" "rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler@moz.com)" "Please check with Moz if they can reduce the rate your site is crawled. Only after you confirm that the rate is decreased we will remove the limit imposed on your account." NOTE: Can you resolve this? NOTE: I have achieved the campaign at this time in an effort to keep the site live.
Product Support | | jessential

0
Keeping getting the same results for "link opportunities"

After solving my 404 error, the results in 'link opportunities' are still the same. How do i refresh the site and get updated results?
Product Support | | kevinbp

0
Giving others like employees admin access ?

Hi I have my master logins for Moz but just wandering if i can grant a degree of access to employees via their own logins (just 1 person at this stage) since i dont want to share mine (since use for other stuff too) and ideally dont want them to have full admin permission (such as being able to delete campaigns) ? Cheers Dan
Product Support | | Dan-Lawrence

0
Why hasn't my site info shown up yet?

I set up a campaign a few days ago (middle of last week) and the information still hasn't shown up. I figured that analytics were gathered weekly so I let it sit until Monday, but it still tells me to check back in 24 hours. Thanks for your help! Love the suite, by the way - lots and lots of great tools and easy enough to let my client see what's going on!
Product Support | | JamesMason

0
"We are collecting your traffic data now!" message displayed for many days

I am getting this message: "We are collecting your traffic data now! You should see your traffic metrics here within the next 24 to 48 hours." for more then 3 days now and I'm starting to worry... Is this showing only to me or is a delay on all MOZ accounts?
Product Support | | SorinaDascalu

0