The issue with Moz being stuck on crawling a site for an extended period can be due to various reasons. Here are some possible causes and solutions based on the provided search results:
Crawl Troubleshooting:
The crawl may be affected if the Moz crawler was banned or blocked by a forbidden response on the site. It's essential to ensure that the robots.txt file is accessible and not blocking the Moz bot. Checking the server logs for the response to the Moz bot and performing a recrawl through Moz Pro Site Crawl can help resolve the issue
1: Site Size and Crawl Limits:
If the site is larger than the allotted crawl limit, Moz may be unable to crawl every page every week, leading to delays in the crawl completion. Adjusting the crawl limit, upgrading the subscription for higher crawl limits, or adjusting the robots.txt to instruct Moz to crawl fewer pages can be potential solutions
2: Critical Crawler Issues:
Critical crawler issues such as 4xx and 5xx errors can impact the crawl. Resolving these issues and using the recrawl feature in Moz Pro can help address the issues and ensure a successful crawl
4: Spikes in Site Crawl Issues:
Spikes in issue counts can be caused by changes to the robots.txt file, site crawl limits, or the crawler's behavior. It's important to investigate the causes of these spikes and make necessary corrections to facilitate a smooth crawl
5: In summary, to address the issue of Moz being stuck on crawling a site, it's important to ensure that the site is accessible to the Moz crawler, resolve any critical crawler issues, and investigate and address any spikes in site crawl issues. Additionally, adjusting the crawl limit, upgrading the subscription, and making necessary changes to the robots.txt file can help facilitate a successful site crawl and thank you from shareyoursaas
