After fixing Crawl Errors, how long does it take to for Moz or Google to re-crawl a website?
-
Last night I found out through Moz that my robots.txt file was blocking any crawling of my website. I fixed the issue. Now do I just sit and wait?
-
Moz will crawl the site every week so you'll need to wait a few days for that to happen.
In the meantime you could try using the Moz Crawl tool to do a manual crawl. Depending on the size of your site this can take a few days so you might find it's just worth waiting for the weekly crawl. You can find the crawl tool here:
https://moz.com/researchtools/crawl-test
If you site isn't included in Google's index because it was inaccessible due to the robots,txt, then you'll want to go into Google Webmaster Tools and use the Crawl -> Fetch as Googlebot to kickstart Google crawling/indexing your site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I want to audit my website
Hi professionals, I am newbie in SEO field, I want to audit my company blog "https://iowatreeplanting.com/" How to check domain authority, page authority and other matrics of this blog. I want to rank this blog in google and other search engines but no one can help me to do this! Please guide me in depth i am very thankful to you Thanks!
Getting Started | | Smartlanjabdul1 -
Error code 804
URL Page Authority Linking Root Domains Status Code | Error Code 804: HTTPS (SSL) Error Encountered https://www.alleninv.com |
Getting Started | | alleninv0 -
Crawl rate
How often does Moz crawl my website ? (I have a number of issues I believe I have fixed, and wondered if there was a manual request to re-crawl ?) Thanks. Austin.
Getting Started | | FuelDump0 -
Moz campaign for a subdomain with no home page
Hello all, I'm trying to setup a campaign in Moz for one of our subdomains that we use for marketing landing pages. Unfortunately there really is no home/landing page for the subdomain so when I enter it into the campaign settings it says the site isn't accessible. Is there anyway for me to add this? I tried entering the subdomain with a star, e.g. pages.mydomain.com but it would not accept it. Any ideas? Thanks!
Getting Started | | Brando160 -
Crawl test
Can anyone give me an idea how to use the MOZ crawl test results...I'm a little confused on how to read it? I have a lot of "no's"...I think this is good?
Getting Started | | sdwellers0 -
Google+ Not Working in my Profile
I would like to add my Google+ and Facebook addresses in my profile but when I do it says my vanity names are not correct. I am copying and pasting directly from the Google+ page and Facebook page so I do not know what to do. When I do try to just add the numbers for my Google+ page that is not working either. Thanks - Travis
Getting Started | | TravG1230 -
Fixing Non-Valid URL - Oops! This does not appear to be a valid URL. Please try again.
Edit - Turns out I can't delete a question. I ended up finding out I'm supposed to email the Moz team at help@moz.com. I did a search regarding the issue, but I only found answers for people using AWS. The results say the .htaccess isn't setup correctly, but I don't know how to fix that file. Any help would be greatly appreciated. Thanks.
Getting Started | | MattHarris0 -
I don't believe moz is seeing everything that is on my webpage
I used the page key word grader and got an "F" Moz said that my keyword employee handbook was not in my title nor was it found in the body of my page. But when I look at the page and double check everything it is there all over the place. I am not blaming moz this is a wiz site and while I am a beginner and very well could be wrong could anyone just take a look and tell me if I am nuts or what. The web page is http://www.cestoday.com/#!employee-handbook/co0h I now have the font so big I will have to fix that. Thank you
Getting Started | | redsman9440