Can't work out robots.txt issue.
-
Hi
I'm getting crawl errors that MOZ isn't able to access my robots.txt file but it seems completely fine to me? Any chance anyone can help me understand what might be the issue?
-
I would follow the advise from Roman, I tried to access the file via a few ways as well and it doesn't seem to be an issue to load it at all.
-
Ok, I made a quick test of your robot.txt file and looks fine, making HTTP status code test and shows me 200 code which is ok. Also, you need to make sure that your robot.txt file is accessible for the Moz crawler
- Remember to put your file is in the top-level directory of your web server.
- Also, check if your hosting provider is not blocking third-party crawlers (server level)
Here you can test the status code of your robot.txt file** https://httpstatus.io/ **If you're still having trouble this is the email of Moz help@moz.com
Also, you should check
Best of luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have a duplicate content on my Moz crawler, but google hasn't indexed those pages: do I still need to get rid of the tags?
I received an urgent error from the Moz crawler that I have duplicate content on my site due to the tags I have. For example: http://www.1forjustice.com/graves-amendment/ The real article found here: http://www.1forjustice.com/car-accident-rental-car/ I didn't think this was a big deal, because when I looked at my GWT these pages weren't indexed (picture attached). Question: should I bother fixing this from an SEO perspective? If Google isn't indexing the pages, then am I losing link juice? 6c2kxiZ
Moz Pro | | Perenich0 -
Can I view inbound links to my Facebook business page?
I changed the title of my facebook page for my business Admiral Movers. I realized that all the listings that I manually entered over the years may be lost. Is there a way to view old page and new page inbound links to our facebook page?
Moz Pro | | Smcnelley0 -
Issue with Data Updates on Campaign
I'm reviewing a campaign I am running for a client, the Keywords data has not updated yet even though it's supposed to update every monday. The competitive analysis shows a date of August 14, which is odd becuase I'm on a trial account?
Moz Pro | | amerihope0 -
My page has a 302 redirect and I don't know how to get rid of it!
I use Internet Officer tool to see the 302 redirect but I check the redirects in the CPanel and there are none. In the .htaccess there are none either. I don't know where else to look 😞 The url is http://servicioshosting.com Can you guys help me? I can't set up a campaign because Google can't crawl the website. I can't setup the Facebook OpenGraph because of the redirect. error.jpg
Moz Pro | | vanessacolina0 -
Where can I find a list of all Page Authority Metrics?
Where can I find a list of all Page Authority Metrics? I'm looking for an exhaustive list of all link metrics that calculate Page Authority? Thanks
Moz Pro | | poolguy0 -
Is the Open Site Explorer Tool working right now?
Is the Open Site Explorer Tool working right now? I keep getting directed to a sign-up page.
Moz Pro | | LarryEngel0 -
What can i use instead the juicy link finder?
I need to find authority links and i used to find them with juicy link finder, i tried to find them with open site explorer and link acquisition, but there´s no solution. Please, what can i use?
Moz Pro | | AbacoDigital0 -
Why aren't DMOZ links showing up in Open Site Explorer?
I have a page that is listed in DMOZ.org, but when I run an OSE Link Analysis, that link doesn't show up. I'm currently doing the free trial for SEOMoz Pro and I'm concerned that other links might not be showing up as well. Has anyone had any similar experiences and/or do you know why DMOZ links specifically might not appear in the results?
Moz Pro | | JoyceScott0