Why is on-demand crawl reporting numbers that are very different than the regualr site crawl by Moz?
-
When I look at the site crawl available through my dashboard, it reports 22.9K pages crawled. I just ran a on-demand crawl and it is only reporting 6K pages crawled. Why is the number so much lower?
-
Thanks! Can you tell me how long it generally takes for the Site Crawl to run? Mine has been reporting it is "In progress" for the past 8hrs??
-
Hey Charles!
The scope of the two tools are different, Site Crawl has a much higher limit of pages that can be crawled, with you plan we look for up to 20k 200ok pages, where On Demand Crawl is looking for only the first 3k pages that return 200 ok status.
Hope that helps, let me know if you need anything else.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz can't crawl my new website?
We had a new website go live at the end of April - I keep requesting crawl tests but I get this in the excel copy... URL Title Tag
Moz Bar | | RayflexGroup
http://www.pvc-strip.co.uk 602 : Page redirects to a URL outside the scope of this campaign. I always list the website as https://... but the crawl always returns the http:// version. Not sure what I can do to make sure the website can be crawled?0 -
Moz Crawl only crawling the top level page (1 page)
For the past few mounts my weekly site crawl has been inconsistent. One week works fine, it crawls all of my 500 or so pages. The following week it only crawls 1 page (http://mydomain.com) and nothing else. A few weekly scan go by and the crawl is back up the the 500 or so pages.I went ahead and created several campaigns with duplicate settings and crawled the site. Most times but not all the new campaign's crawl works fine crawling all pages. But within a week or two the weekly crawl will fail again. (crawling 1 page). Currently i have four campaign's all with the same settings running weekly crawls. 2 campaign's crawled the 500 pages and two crawled only the single page. Any help will be greatly appreciated
Moz Bar | | dmaude0 -
Tracked Ranking Keywords in Campaign and Ranking Keywords in Keyword Explorer are coming up with very different results - does anybody know why?
We have a campaign set up that is tracking keywords with varying monthly volumes and the website is ranking for them. I have gone into the new Keyword Explorer Feature (looks great!) and I have tested it by putting in the root domain address mentioned above. My problem is that none of the keywords that I am tracking and ranking for are appearing in the "ranking keywords" report in Keyword Explorer. Does anybody know why this might be happening?
Moz Bar | | RedAntSolutions0 -
Does "Disallow: /xmlrpc.php" in robots.txt affect moz tools ability to fetch DA?
Just checked a website for Domain Authority using Moz' tool, however it returned 1 for DA, which should be unlikely. I have been trying to find the problem and found "Disallow: /xmlrpc.php" in robots.txt. Could this affect Moz' tools ability to get the required data?
Moz Bar | | Foli0 -
How do you feel when Moz marks one of your questions as "answered?"
Hi everyone, This is not meant to be snarky at all, so I just want to preface my question with that. So, since the new re-branded Moz rolled out last year, I'm sure many of you have noticed that if you ask a question and it is answered by a Moz associate, your question is marked as "answered." I'm sorry, but I don't like this. Here's why, I'm the one who asked the question. I should be the one who determines if the answer was adequate for me, or if it didn't sufficiently answer my question. This is particularly true when my question doesn't have to do with a customer service issue or a Moz tool question. If I ask a question about SEO, Content, CRO, marketing or any other subject, I feel like it should be me and only me who determines whether or not I feel like my question is answered. In addition to this, Moz is actually depriving themselves of useful UGC by shutting down questions in this way. How? Because when the rest of us who frequent the Q & A see a question that's already been marked as "answered" we tend not to open it, read it and respond, because we think that person has already gotten what they needed....when in fact, it could be that a Moz associate has jumped in and marked their question as answered when it really wasn't. Consequently, we all miss out. I propose/move that Moz associates can only mark questions as "answered" when they pertain directly to Q & A about Moz tools, service and support. All other questions must be marked as "answered" only by the asker or closed as "answered" after they have been dormant for 6 months or more. Can I get a second (motion) ?
Moz Bar | | danatanseo4 -
Moz Dupe content crawl anomaly
Hi Moz has completed a crawl for a site i'm working on which also has a development area (hence with lots of dupe content) on a sub domain (and this dev area hasn't been hidden from crawlers via password, robots, gwt etc etc). Moz dupe content report is not showing any of these urls though even though my campaign setting is on 'root' domain so i would have thought report should be listing the subdomain urls as dupe content (because they are dupe content). Any ideas ? Cheers Dan
Moz Bar | | Dan-Lawrence0