Question about when new crawls start
-
Hi everyone,
I'm currently using the trial of seomoz and I absolutely love what I'm seeing. However, I have 2 different websites (one has over 10,000 pages and one has about 40 pages). I've noticed that the smaller website is crawled every few days. However, the larger site hasn't been crawled in a few days. Although both campaigns state that the sites won't be crawled until next Monday, is there any way to get the crawl to start sooner on the large site?
The reason that I've asked is that I've implemented some changes that will likely decrease the amount of pages that are crawled simply based upon the recommendations on this site. So, I'm excited to see the potential changes.
Thanks,
Brian
-
Hi Brian,
Did your site get crawled again and is everything good now?
Keri [SEOmoz associate]
-
There is no way to force a crawl, that I am aware of. I actually have 4 SEOmoz accounts that I use for various clients for crawls and their websites. Never seen a way to force a crawl, simply because there are tons of sites in the queue and they can only crawl so many sites so fast.
I'd just try to be patient and see the results on Monday or Tuesday! Good luck.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is it possible to block Moz from crawling sites?
Hi, is it possible to stop Moz from crawling a site at the server level? Not that I am looking to do this or anything, but here's why I'm asking. I have been crawling a site that is managed (currently by 2 parties), and I noticed that this week pages crawled went from 80 (last week) to 1 page!! I know, what? See my image attached... and the issues all went to zero "0"....! So is it possible that someone can't prevent Moz from crawling the site at the server level? I checked the robots.txt file on the site, but nothing there. I'm curious. dYNUwjd.jpg
Moz Pro | | co.mc0 -
Why might Google be crawling via old sitemap, when the new one has been submitted and verified?
We have recently relaunched Scoutzie.com and re-submitted our new sitemap to Google. When I look on Webmaster tools, our new sitemap has been submitted just fine, but at the same time, Google is finding a lot of 404s when crawling the site. My understanding, it is still using crawling the old links, which do not exists. How can I tell Google to refresh it's index and to stop looking at all the old links?
Moz Pro | | scoutzie0 -
Crawled pages are missing and showing just 1 page crawled
One of my campaign has got around 8500 pages crawled(seomoz) and reports are shown, but suddenly it is showing 1 page crawled. Why it is happened like this? How can i get back the previous reports?
Moz Pro | | Sulekha0 -
Is there a way to get the SEO campaign crawl to run from outside the US?
We are working on a international site that uses IP redirection. I am currently trying to crawl and monitor the Australian version of the site, however the pro-campaign crawler is running from the US and only giving me US data. Is there a way to tell the crawler to run through a local proxy so we can track international versions? Thanks!
Moz Pro | | MBASydney0 -
When do I see the new Linkscape?
Hi I have seen a Tweet and item on the latest news saying that the Linkscape has been updated. I'm still seeing a report generated at the start of May though. What do I need to do to get my grubby hands on the latest data? Chris
Moz Pro | | P4D0 -
What's with the new Q&A set up?
In the old Q&A set up your response was automatically given a thumbs up by you so that you got a mozpoint for every response you gave. Now you have to manually thumb up your responses. Why was this change made? Can we get it back to the way it was?
Moz Pro | | DanDeceuster1 -
What causes Crawl Diagnostics Processing Errors in seomoz campaign?
I'm getting the following error when seomoz tries to spider my site: First Crawl in Progress! Processing Issues for 671 pages Started: Apr. 23rd, 2011 Here is the robots.txt data from the site: Disallow ALL BOTS for image directories and JPEG files. User-agent: * Disallow: /stats/ Disallow: /images/ Disallow: /newspictures/ Disallow: /pdfs/ Disallow: /propbig/ Disallow: /propsmall/ Disallow: /*.jpg$ Any ideas on how to get around this would be appreciated 🙂
Moz Pro | | cmaddison0 -
Does it really take 3-7 days to crawl a site?
This is what you say on your website: Please note: depending on the size and speed of your site, it may take between 3 and 7 days to complete your crawl. Are you guys kidding? This is unacceptable for almost all of my deadlines, and I suspect 99% of the SEO world... you can fly to the moon faster.
Moz Pro | | rldonnell0