Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How long does a crawl take?
-
A crawl of my site started on the 8th July & is still going on - is there something wrong???
-
As the others suggested, there are too many factors involved. The major factors are the size of your site and it's navigation. Sometimes a site will offer a calendar that provides 3 years of dates. If the navigation is not set up properly a crawler can get lost in the calendar. Every day on a calendar can offer daily view, weekly view, monthly view, birthdays, etc. Calendar issues are just one example of how a crawler can get "stuck" on a website.
-
How long is a piece of string? If you website has 10 pages it will be quick if more, longer. It can take up to 7 days. Mine was started 2 days ago and is now complete for one of my campaigns. hope this helps.
-
Hi,
My crawl said it would take ~1 day in the campaign manager, but it actually took around 7 days. I think you have to be patient and wait! If you are unsure, no harm in emailing support in my opinion.
It's probably inaccurate because it's largely dependant on how much content you have as well as other factors.
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long do changes in title tags take to affect SEO?
This is kind of a loaded question. I'm completely new to SEO. I think my boss signed up for Moz Pro sometime in February and started adding data to our Ecommerce site to help with rankings. Sometime before this, I changed some of the title tags on the site (trying to help with organic search and CTR). I did not do a site wide change.... just changed maybe 10-20 (just a guess). I did it with keywords in mind but did not make note of when I did it. I didn't really know better at the time, and I did not have access to Google Analytics or Moz Pro. I was looking through the ranking data/graph for February and March. It won't let me look before February 29th (so that's why I think my boss started the Mos Pro subscription around at that time). On that day it said we ranked 12 keywords in the 1-3 spot, and then the following week (march 7) it went down to 6. I don't think or know if any major site changes were implemented, so I'm not sure why that happened and if it has anything to do with my title tag changes I did maybe a week or two before (again I am not sure when I did this unfortunately). Since then the keyword ranking numbers stayed about the same with organic traffic slowly going down (it could be because we are getting out of season for our industry though). The second week of March the site was upgraded and since then the menu has been completely changed around. Last week I did a site wide title tag change. So the minor changes I made in February are no longer in effect anyway. I added more keywords to Moz earlier this week and the number for 1-3 spot keywords went up from 6 to 20. It also says my ranking moved up 4 keywords and down 13 keywords. Anyway, I am wondering how seriously I should take these changes and if I'm damaging the site. I am new to Moz Pro also so all the data you can access is kind of confusing/overwhelming.
Moz Pro | | AliMac260 -
Woocommerce filter urls showing in crawl results, but not indexed?
I'm getting 100's of Duplicate Content warnings for a Woocommerce store I have. The urls are
Moz Pro | | JustinMurray
etc These don't seem to be indexed in google, and the canonical is for the shop base url. These seem to be simply urls generated by Woocommerce filters. Is this simply a false alarm from Moz crawl?0 -
Block Moz (or any other robot) from crawling pages with specific URLs
Hello! Moz reports that my site has around 380 duplicate page content. Most of them come from dynamic generated URLs that have some specific parameters. I have sorted this out for Google in webmaster tools (the new Google Search Console) by blocking the pages with these parameters. However, Moz is still reporting the same amount of duplicate content pages and, to stop it, I know I must use robots.txt. The trick is that, I don't want to block every page, but just the pages with specific parameters. I want to do this because among these 380 pages there are some other pages with no parameters (or different parameters) that I need to take care of. Basically, I need to clean this list to be able to use the feature properly in the future. I have read through Moz forums and found a few topics related to this, but there is no clear answer on how to block only pages with specific URLs. Therefore, I have done my research and come up with these lines for robots.txt: User-agent: dotbot
Moz Pro | | Blacktie
Disallow: /*numberOfStars=0 User-agent: rogerbot
Disallow: /*numberOfStars=0 My questions: 1. Are the above lines correct and would block Moz (dotbot and rogerbot) from crawling only pages that have numberOfStars=0 parameter in their URLs, leaving other pages intact? 2. Do I need to have an empty line between the two groups? (I mean between "Disallow: /*numberOfStars=0" and "User-agent: rogerbot")? (or does it even matter?) I think this would help many people as there is no clear answer on how to block crawling only pages with specific URLs. Moreover, this should be valid for any robot out there. Thank you for your help!0 -
Meta Tag Descriptions not being found in Moz Crawls
Hey guys, I have been managing a few websites and have input them into Moz for crawl reports, etc. For a while I have noticed that we were getting a gratuitous amount of errors when it came to the number of missing meta tags. It was numbering in the 200's. The sites were in place before I got here and a lot of the older posts no one had even attempted to include tags, links of the page or anything. As they are all Wordpress Sites and they all already had the Yoast/Wordpress SEO plug-in installed on them, I decided I would go through each post and media file one at a time and update their meta tags via the plug in. I personally did this so I know that I added and saved each one, however the Moz crawl reports continue to show that we are missing roughly 200 meta tags. I've seen a huge drop off in 404 errors and stuff since I went through and double checked everything on the sites, however the meta tag errors persist. Is this the case that Moz is not recognizing the tags when it crawls because I used the Yoast Plugin? Or would you say that the plugin is the issue and I should find another way to add meta tags to the pages and posts on the site? My main concern is that if Moz is having issues crawling the sites, is Google also seeing the same thing? The URLS include:
Moz Pro | | MOZ.info
sundancevacationsblog.com
sundancevacationsnews.com
sundancevacationscharities.com Any help would be appreciated!0 -
How long for authority to transfer form an old page to a new page via a 301 redirect? (& Moz PA score update?)
Hi How long aproximately does G take to pass authority via a 301 from an old page to its new replacement page ? Does Moz Page Authority reflect this in its score once G has passed it ? All Best
Moz Pro | | Dan-Lawrence
Dan3 -
FollowerWonk: How long have I been following someone?
Is there a way within followerwonk to find out how long you have been following someone for or indeed how long they have been following you for? I have downloaded the excel document from the "sort your followers" tab Which has sections called. "follows @name" and "@name follows" but these just give a last checked date(if they do follow/are being followed) rather than the metric I want!
Moz Pro | | BBI_Brandboost0 -
Problem crawling a website with age verification page.
Hy every1, Need your help very urgent. I need to crawl a website that first has a page where you need to put your age for verification and after that you are redirected to the website. My problem is that SEOmoz, crawls only that first page, not the whole website. How can I crawl the whole website?, do you need me to upload a link to the website? Thank you very much Catalin
Moz Pro | | catalinmoraru0 -
How to force SeoMoz to re-crawl my website?
Hi, I have done a lot of changes on my website to comply with SeoMoz advices, now I would like to see if I have better feedback from the tool, how can I force it to re-crawl a specific campaign? (waiting another week is too long :-))
Moz Pro | | oumma0