Site Crawl Error
-
In moz crawling error this message is appears:
MOST COMMON ISSUES
1Search Engine Blocked by robots.txt
Error Code 612: Error response for robots.txt
i asked help staff but they crawled again and nothing changed.
there's only robots.XML (not TXT) in root of my webpage it contains:
User-agent: *
Allow: /
Allow: /sitemap.htmanyone please help me? thank you
-
Thank you,
per your instructions i have changed robots.txt exactly like:
User-agent: *
Disallow:As you can see there's no Allow: / and Disallow line is blank
is that correct?
-
Hello, this is Abe from the Moz Help team. After our reported fix in your latest support ticket, you will see correct crawl crawl results in your next weekly update
-
Would you mind sharing a link to your site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I need an interlinking report for my site, is there a report in Moz or another application that tell me how all of my pages are linked to other pages on my site?
I am in the process of doing a redesign for one of my sites. I need an interlinking report for my site. Is there a report in Moz or another application that tell me how all of my pages are linked to other pages on my site?
Moz Pro | | seoflorida0 -
How to trace 4XX errors in campaign data?
I am trying to resolve a number of 4XX (Client Error) issues noted in my campaign report, but I have no idea where they are originating. All I get is the broken link, the status code, the page authority and the number of linking root domains. How do I identify where the erroring link originates? Is it coming from within my website, or from an external website? I have gone through my site in great detail looking for these links, but can't find any evidence of them. I also use a broken link detection plugin and it reports no problems whatsoever. Is there a way I can drill down into these 404 links and get more detail about where they are coming from?
Moz Pro | | WebSupportGuy420 -
Open Site Explorer Social Media?
Open Site Explorer always reports only a few facebook likes. If I go to the business page of the site, it has 100+ likes. This seems to happen a lot. Is this a OSE issue or is there an issue with the site clearly being defined to google as a connected facebook account? If OSE doesn't see the likes, will google? How can I fix?
Moz Pro | | JML11790 -
How do I retrieve crawl and ranking data about a site from the past?
Hey. One of my main clients has asked to see the crawl data and rankings data for the past eight months. He wants to have tangible evidence of the effects of Penguin. I would like that info too. Is it possible to retrieve that information on a weekly crawl and ranking basis through SEO Moz and if so, how do you do it? I simply want to show a graph, timeline and brief explanation across several main keywords... Help me as you guys always do - You rock Best Ben
Moz Pro | | creativeguy0 -
How long does it typically take MOZBOT to crawl a site?
Our site has had "crawl in progress" for over 24 hours now without an update, we're dying for the results since our last changes :).
Moz Pro | | absoauto0 -
How do you get Mozbot to crawl your website
I trying to get the mozbot to crawl my site so I can get new crawl diagnostics info. Anyone know how this can be done?
Moz Pro | | Romancing0 -
SEOMoz Pro and Multiple Niche Sites
Currently I have 15 affiliate marketing sites and three blogs which are monetized. My budget dictates the base pro package so may I get some recommendations on how or if I can do analysis over time for all the sites. I ask this because it seems you would set your campaigns for long-term but I imagine you delete/modify/add campaigns to analyze different key words. Thanks for your help.
Moz Pro | | JavaManOne0