Site Crawl Error
-
In moz crawling error this message is appears:
MOST COMMON ISSUES
1Search Engine Blocked by robots.txt
Error Code 612: Error response for robots.txt
i asked help staff but they crawled again and nothing changed.
there's only robots.XML (not TXT) in root of my webpage it contains:
User-agent: *
Allow: /
Allow: /sitemap.htmanyone please help me? thank you
-
Thank you,
per your instructions i have changed robots.txt exactly like:
User-agent: *
Disallow:As you can see there's no Allow: / and Disallow line is blank
is that correct?
-
Hello, this is Abe from the Moz Help team. After our reported fix in your latest support ticket, you will see correct crawl crawl results in your next weekly update
-
Would you mind sharing a link to your site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
403 error but page is fine??
Hi, on my report im getting 4xx error. When i look into it it says the error is crital fo4r 403 error on this page https://gaspipes.co.uk/contact-us/ i can get to the page and see it fine but no idea why its showing a 403 error or how to fix it. This is the only page that the error is coming up on, is there anything i can check/do to get this resolved? Thanks
Moz Pro | | JU-Mark0 -
Site Not Cached By Google
Hi My name is Apoorwa and i have my own website, My website is not cached by google, Why is this problem happening with my site.can somebody help me please? its urgent....this is my site - http://www.holifestival.org/Please assist me.......
Moz Pro | | Packersmove0 -
What is error message:social_account.no_method?
When trying to add a twitter account to track, I received the following error message:social_account.no_method - has anyone else received this message?
Moz Pro | | solutionbuilt.com0 -
Site Redesign Launch - How Can I crawl for immediate review
Just redesigned my site and want to have a crawl done to check for errors or any items which need to be cleaned up. Anyone know how I can do this as SEOMoz only crawls once per week. Thanks!
Moz Pro | | creativemobseo0 -
Seomoz and site explorer completely different
seomoz and site explorer completely different results on own site and competitors on moz i am ahead in all aspects in site explorer i am way off par...why
Moz Pro | | lordmoose85450 -
SEOMoz Crawler and rel_canonical_tag Errors
This tag is showing up on category pages (that do not have a duplicate page on the site). In mid November Google cut our traffic by 30%. Could this tag be confusing the spider? According to the moz crawler - we seemed to be dinged for this on 95% of our pages. Is this hurting us? It seems to direct back to the same page.EG: From the FMI3600 Page http://www.brick-anew.com/FMI-3600-Fireplace-Doors.html: http://www.brick-anew.com/FMI-3600-Fireplace-Doors.html"> There is only one page for the FMI 3600 Fireplace Door category - however, it does have the same products on it as other FP Door Category pages,
Moz Pro | | SammyT0 -
Wild fluctuation in number of pages crawled
I am seeing huge fluctuations in the number of pages discovered the crawl each week. Some weeks the crawl discovers > 10,000 pages and other weeks I am seeing 4-500. So, this week for example I was hoping to see some changes reflected for warnings from last weeks report (which discovered > 10,000 pages). However, the entire crawl this week was 448 pages. The number of pages discovered each week seems to go back and forth between these two extremes. The more accurate count would be nearer the 10,000 mark than the 400 range. Thanks. Mark
Moz Pro | | MarkWill0 -
It won't let me print the secon or third pages of site errors.
When I place the site errors page in pdf format it won't let me print the second or any of the other webpages containing content about my site. Does any one know why?
Moz Pro | | ibex0