Can't work out robots.txt issue.
-
Hi
I'm getting crawl errors that MOZ isn't able to access my robots.txt file but it seems completely fine to me? Any chance anyone can help me understand what might be the issue?
-
I would follow the advise from Roman, I tried to access the file via a few ways as well and it doesn't seem to be an issue to load it at all.
-
Ok, I made a quick test of your robot.txt file and looks fine, making HTTP status code test and shows me 200 code which is ok. Also, you need to make sure that your robot.txt file is accessible for the Moz crawler
- Remember to put your file is in the top-level directory of your web server.
- Also, check if your hosting provider is not blocking third-party crawlers (server level)
Here you can test the status code of your robot.txt file** https://httpstatus.io/ **If you're still having trouble this is the email of Moz help@moz.com
Also, you should check
Best of luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEOMoz API not working for Scrapebox
I want to import SEOMoz data to list of URLs I have using scrapbox. I added in my credentials according to the API but am getting error 401 as the status of all my links. Any idea why and what I should be doing?
Moz Pro | | theLotter0 -
SeoMoz: Competitive domain analysis report doesn't add up?
Hi, When I check OSE against the Competitive Domain Analysis report in my account the numbers are way off. If the CDA report was 100%, OSE is showing 30% of that. I'm looking at linking root domains with no filters applied. Is the data not pulled from the same db? Thanks.
Moz Pro | | Bondara0 -
Can I see when SEO Moz has crawled my website?
I would like to know if it's possible to see (maybe in my Google Analytics) if SEO Moz has crawled my website. I'm also curious if and where I can see when the robot of Google visited my website. Thanks!
Moz Pro | | Spotler0 -
Confusion about how SEOMOZ crawler works...
So according to my SEOMOZ dashboard, I'm ranking between #3-4 for one of my keywords. My keyword is 'Boston Wedding Photographer'. My site is http://www.symbolphoto.com I show up in google places, true. But i was wanting to rank organically. Am i right in the assumption that Google Places and Google Organic are not the same thing? SEOMOZ claims 3,4th but not organically(Assuming they aren't the same thing) I get pretty good traffic right now being in Places, but i can't help but feel that organically ranking would bring more traffic. Any suggestions or advice is greatly appreciated. TIA! -Brendan
Moz Pro | | symbolphoto0 -
When's the next Webinar?
I really love the webinars! I listen to them on long walks occasionally, but I haven't seen one in May or June. When will the next one be? Thanks! I apologize if this should have gone to the help desk...
Moz Pro | | WilliamBay0 -
Why won't scheduled crawl of my site begin?
I currently have a campaign running on SEOMoz for over a month. It has been showing that a crawl was scheduled to start on 12/21. Now it's 12/23 and there has not been a new crawl, and it still says scheduled for 12/21.. Anyone know why this is happening or how to fix it? Thanks
Moz Pro | | Prime850 -
Can I see backlinks to a specific URL, not including redirects?
Does SEOMoz have a tool that allows me to see backlinks to a specific URL sans redirected URLs? So, URL is www.website.com/search. www.website.com redirects to www.website.com/search and I want to see how many people have linked to the /search, the resulting URL. Thanks for any help! Erin
Moz Pro | | ErinTM0 -
Where can I view all of the Questions that I have Subscribed to?
I recently asked a question about viewing the questions I have answered already: http://www.seomoz.org/q/where-can-i-view-all-of-the-questions-that-i-have-answered It turns out the link was right under my nose (but it appears a lot of people were having this issue). I'm now having the problem of not being able to see where I can view all the Q&As that I have subscribed to. Again, I may just be missing a link that's right in front of me but it certainly isn't obvious. Any help much appreciated!
Moz Pro | | NickPateman810