Thank you everyone... I'm learning! And you are helping!
Posts made by 1sixty8
-
RE: Cannot Crawl ... 612 : Page banned by error response for robots.txt.
-
RE: Cannot Crawl ... 612 : Page banned by error response for robots.txt.
I think I figured it out... For some reason, robots.txt was set at 600...I changed it to 644... I will run crawl again... Thanks.
-
RE: Cannot Crawl ... 612 : Page banned by error response for robots.txt.
Thank you for the responses. Can you give me any direction on how to correct this? I am lost
-
Cannot Crawl ... 612 : Page banned by error response for robots.txt.
I tried to crawl www.cartronix.com and I get this error:
612 : Page banned by error response for robots.txt.
I have a robots.txt file and it does not appear to be blocking anything
Also, Search Console is showing "allowed" in the robots.txt test...
I've crawled many of our other sites that are similarly set up without issue.
What could the problem be?