Unsolved Moz crawler not crawling on my site
-
Hi all,
im facing an issue where moz crawler is unable to crawl my site. The following error keeps showing Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag.
This is my robots.txt file : https://www.wearefutureheads.com/robots.txt
I'm not sure what else am I missing.. can anyone help
-
@teikh Hi, thanks for contacting us! Would you be able to write into help@moz.com so that our support team can take a closer look at your query please? Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How get rid of 403 crawl error?
My wordpress website has 162 crawl 403 errors. Based on what I read it means that the server is denying crawlers to access the pages. The pages itself will load so guessing it's just an issue with crawlers only. How do I go about fixing this issue?
On-Page Optimization | | emrekeserr30 -
Unsolved How do I cancel this crawl?
The latest crawl on my site was the 4th Jan with a current crawl 'in progress'. How do i cancel this crawl and start a new one? I've been getting keyword ranking etc but no new issues are coming through. Screenshot 2022-05-31 083642.jpg
Moz Tools | | ClaireU0 -
Moz keeps logging me out
Why moz keeps logging me out? It's annoying that I have to keep putting my credentials after checking the "remember me" checkbox.
Product Support | | sekamoving0 -
Site Crawl Status code 430
Hello, In the site crawl report we have a few pages that are status 430 - but that's not a valid HTTP status code. What does this mean / refer to?
Product Support | | ianatkins
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_Client_errors If I visit the URL from the report I get a 404 response code, is this a bug in the site crawl report? Thanks, Ian.0 -
Can I delete a question that I previously posted on Moz?
I posted a question with links to a website. That question now appears in google search when I search for that website, which I don't want. Can I delete the question I posted on Moz?
Product Support | | LavM1 -
Changed to HTTPS: Do I need to update Moz?
Hi Everyone! I have moved my website from HTTP to HTTPS. I believe that I've done everything correctly. Now that I've done that, do I need to update my Moz campaign (and how to do that)? Or, will Moz handle it itself? Or, do I need to create a new campaign with the HTTPS url? Thanks so much!
Product Support | | journeybeyondtravel0 -
Analytic goals in Moz
We should have the ability to see any and all conversions goals, as defined in analytics, in Moz. After all, the whole point of marketing, SEO, PPC, Social, etc. is to drive sales or leads that create sales right? So, why should we have a dashboard tool that doesn't show us the 1 thing that ultimately matters the most? I understand that Moz simply wasn't designed to do this, but I think that should change. After all, we are in this to make money I hope. My 2 Cents. Micha
Product Support | | Multiverse-Media-Group1
