Crawl rate
-
How often does Moz crawl my website ?
(I have a number of issues I believe I have fixed, and wondered if there was a manual request to re-crawl ?)
Thanks.
Austin.
-
Many thanks
-
Perfect, thanks.
-
Thank you, Auston, I was unsure of the number of pages thanks for clearing that up
-
Hi Austin
All campaign updates occur once a week on the same day based on when the campaign was first created. The crawl tests Thomas suggested is helpful as well for instant crawls. Our crawl-test in research tools will limit to 3000 pages.
Cheers!
-
You asked "how often Moz crawls a site" I'm assuming you are asking about a campaign that would be once a week per a campaign & depending on the size/ number of pages of the site
See FAQ
https://moz.com/help/guides/moz-pro-overview/crawl-diagnostics
There are methods you could either create another campaign however my advice would be to use Moz simply crawl test it will let you crawl your site for issues immediately.
https://moz.com/researchtools/crawl-test
That should take care of it, however.
If for whatever reason that does not help some tools are free if your site has less than 500 pages you can use https://www.screamingfrog.co.uk/seo-spider/
Although to get what you want you most likely will have to spend money.
It sounds like crawl test will do what you want. If it does not still needs to be crawled.
Third-party tools like Screaming Frog SEO Spider Pro and deepcrawl.com Are instant and very helpful Additions to the fantastic tools Moz already offers.
I hope this helps,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why does Moz only seem to be crawling a snap shot of the site I am working with?
I was wondering if anyone can help? I am working using Moz to help improve the SEO on a website I am working with, the website contains thousands of pages, yet for some reason Moz only seems to be crawling a small snap shot of the website. I know there are particular pages that I had added a couple of weeks ago - about 300 in total - and none of these were showing on the first crawl, so I did another on-demand crawl and some of these showed up then. Despite this, it says it crawled 700ish pages, but there are getting close to 20-30ish thousand live pages on the site. Any thoughts and guidance as to why they crawling may be stopping?
Getting Started | | dsmith8020200 -
Moz can't crawl my site.
Moz cannot carry out the site crawl on my online shop. Not really sure what the issue is, it has no problem getting onto my site when you use www. before the address, but it needs to be able to access bluerinsevintage.co.uk Stuck as what to do, we are a shopify store. Anyone else had this problem, or know what i need to change so they can crawl the site? thjis is the page they are getting when trying to get on bluerinsevintage.co.uk but if they use www.bluerinsevintage.co.uk the site comes up. Adam
Getting Started | | bluerinsevintage0 -
Moz could not crawl my httpS website
Hi, we have a website with HTTPS, moz could not crawl it and we get "902 : Network errors prevented crawler from contacting server for page" while in logs we see moz robot access but fail after some seconds, what could be the problem, while moz can access site when it is without httpS | 902 : Network errors prevented crawler from contacting server for page. |
Getting Started | | Hamedkhorasani10 -
Crawl test
Can anyone give me an idea how to use the MOZ crawl test results...I'm a little confused on how to read it? I have a lot of "no's"...I think this is good?
Getting Started | | sdwellers0 -
I am new to MOZ, I set up one tracking campaign two weeks ago, I have tracked no keywords, I have done some keyword research for ranking difficulty and in two weeks I have already hit 50K pages crawled, I'm maxed out, is this common?
I am a startup and can't afford the higher plans yet. And even their highest plan is 600K pages crawled, which seems really low considering how lightly I used the tool and how quickly I hit 50K. Does anyone have any advice or information on how they use the tool on lower packages? Did I do something wrong to hit 50K pages crawled that fast? Does this pricing make any sense, it seems like an incredibly high price, I love the tool, any help is appreciated.
Getting Started | | Daedilus1 -
How do I interpret Duplicate Content in a Crawl Report, when it only gives me a URL? How do I know what is duplicated on that page somewhere else?
I need help interpreting the Crawl Report for Duplicate Content. It gives me the URLs of pages that have duplicate content, but how do I know what content exactly is duplicated elsewhere? And how do I figure out where it is duplicated? Also, are there Moz Analytics articles or videos teaching you how to use each component of the analytics programs? Thanks!
Getting Started | | NancyBryan0 -
Campaign.crawl-seed.bad-response
I am trying to set up a new campaign for a website, but I keep getting this error message... campaign.crawl-seed.bad-response 😞 I have no idea what the problem is. Can you tell me what I am suppose to do to fix this? The URL I am trying to set up is www.aboutplcs.com
Getting Started | | ChadC0 -
Cant download my crawl csv
When I click on the [download csv] in my crawl campaign section nothing happens.
Getting Started | | digitalmedialounge0