After fixing Crawl Errors, how long does it take to for Moz or Google to re-crawl a website?
-
Last night I found out through Moz that my robots.txt file was blocking any crawling of my website. I fixed the issue. Now do I just sit and wait?
-
Moz will crawl the site every week so you'll need to wait a few days for that to happen.
In the meantime you could try using the Moz Crawl tool to do a manual crawl. Depending on the size of your site this can take a few days so you might find it's just worth waiting for the weekly crawl. You can find the crawl tool here:
https://moz.com/researchtools/crawl-test
If you site isn't included in Google's index because it was inaccessible due to the robots,txt, then you'll want to go into Google Webmaster Tools and use the Crawl -> Fetch as Googlebot to kickstart Google crawling/indexing your site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Ways to Use Moz to Increase Rankings
Hello, I work for an online retailer and I have been using Moz for a few weeks now but have had limited success so I wanted to ask how I could better spend my time on this platform. I have been focusing heavily on the Page Optimization tool which has allowed me to rank high on the Google Shopping's Free Listings but it seems to have had little to no impact on the Google All search itself over those weeks. Perhaps I'm not utilizing it properly? I tend to focus on relevant keywords with high volume as identified by the Keyword Explorer tool. Alternatively, is there a Moz tool that might be more helpful? I can provide additional details or specific examples if needed. Thank you for your consideration,
Getting Started | | ForestGT
Forest1 -
Got a problem in using MOZ Crawl test
Hello,
Getting Started | | turkeyanaclinic
Guys i need help as i'm getting this message "**Moz was unable to crawl your site on Dec 26, 2017. **Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster."
After i made a Campaign i'm getting this message but after i created new campaign it crawls well
can you help me to edit the old campaign ? Regards.0 -
Moz for charities and non-profits
Hi, I noticed an earlier thread on this topic that now seems dead so I decided to start a new one: https://moz.com/community/q/non-profit-rates We're a non-profit organisation based outside of the US and we were wondering if MOZ's non-profit program can be extended out? We're fully registered and audited in our home country and can provide all the necessary documentary proof if needed.
Getting Started | | OBW_Noel1 -
How can i start in moz?
I want to know what to do first, how do I start the branding, etc. Thanks!!!
Getting Started | | Gridiron2361 -
Moz Staff: History + Watch List
Hi All, I have been going crazy trying to find a thread I read last week ... I think!
Getting Started | | Mark_Ch
Several search attempts later, no closer to finding the thread in question. A genius moment or simple frustration got me thinking. How cool would it be to have the following features on MOZ Q&A Forum. Watch List
1] Check off any threads that you would like to monitor over time. Should you stumble upon a thread of value then simply add to watch list and view it's progress over time. No searching for the thread in question. History
2] Keep a history off all threads viewed over a period of time. Thanks Mark0 -
Multiple websites -- one for each brand
I am a MOZ newb, and didn't find answer to the following: We are an enterprise with four distinct brands, each acting as a (reasonably) independent subsidiary. I want to track their performance and report on keyword rankings for the enterprise. Is Moz Analytics capable of this? In Google Analytics, for instance, we have an account for each brand, and multiple properties within each account, and so on. I use 3rd-party dashboard tools to combine multiple accounts into a single report. Can I do something similar here. Thanks in advance.
Getting Started | | afhiWebMaster0 -
Campaign.crawl-seed.bad-response ???
Hello Guys I have just tried to set-up a new campaign for this site http://www.emsababies.co.uk/ HoweverI keep getting this error ?? campaign.crawl-seed.bad-response Anyone know what I am doing wrong ?? Cheers James
Getting Started | | BlueNinja1 -
Does Moz Analytics need Google Analytics installed?
I have a few websites on my Moz Analytics account but only have data for one of them. All my other accounts keepy saying come back in 24hrs for more data, although there is no data available. Do I need to connect each account with Google Analytics before Moz populates its own analytics?
Getting Started | | david.smith.segarra1