Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How long does a crawl take?
-
A crawl of my site started on the 8th July & is still going on - is there something wrong???
-
As the others suggested, there are too many factors involved. The major factors are the size of your site and it's navigation. Sometimes a site will offer a calendar that provides 3 years of dates. If the navigation is not set up properly a crawler can get lost in the calendar. Every day on a calendar can offer daily view, weekly view, monthly view, birthdays, etc. Calendar issues are just one example of how a crawler can get "stuck" on a website.
-
How long is a piece of string? If you website has 10 pages it will be quick if more, longer. It can take up to 7 days. Mine was started 2 days ago and is now complete for one of my campaigns. hope this helps.
-
Hi,
My crawl said it would take ~1 day in the campaign manager, but it actually took around 7 days. I think you have to be patient and wait! If you are unsure, no harm in emailing support in my opinion.
It's probably inaccurate because it's largely dependant on how much content you have as well as other factors.
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Moz takes to update DA and PA value?
I used to work daily on my office website https://quintdaily.com/ and follow link building strategy. however, since more than 40-50 days, there is no movement in my DA, as the PA has just lifted up. However, there is no changes in Domain Authority so far. Besides, other than this website, im working on location wise SEO projects, but, i can find those websites Domain Authority changed in less 20 days. Is there is time period in changing the Domain Authority once it touched DA 30+?
Moz Pro | | seochris172 -
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
Why Only Our Homepage Can Be Crawled Showing a Redirect Message as the Meta Title
Hello Everyone, So recently when we checked our domain using a Moz Crawl Test and Screaming Frog only the homepage comes up and the meta title says “You are being redirected to…”. We have several pages that used to come up and when submitting them to GSC no issues come up. The robots.txt file looks fine as well. We thought this might be ‘server’ related but it’s a little out of our field of expertise so we thought we would find out if anyone has any experience with this (ideas of reasons, how to check…etc.) or any potential suggestions. Any extra insight would be really appreciated. Please let us know if there is anything we could provide further details for that might. Looking forward to hearing from all of you! Thanks in advance. Best,
Moz Pro | | Ben-R0 -
Is one page with long content better than multiple pages with shorter content?
(Note, the site links are from a sandbox site and has very low DA or PA) If you look at this page, you will see at the bottom a lengthy article detailing all of the properties of the product categories in the links above. http://www.aspensecurityfasteners.com/Screws-s/432.htm My question is, is there more SEO value in having the one long article in the general product category page, or in breaking up the content and moving the sub-topics as content to the more specific sub-category pages? e.g. http://www.aspensecurityfasteners.com/Screws-Button-Head-Socket-s/1579.htm
Moz Pro | | AspenFasteners
http://www.aspensecurityfasteners.com/Screws-Cap-Screws-s/331.htm
http://www.aspensecurityfasteners.com/Screws-Captive-Panel-Scre-s/1559.htm0 -
How long do changes in title tags take to affect SEO?
This is kind of a loaded question. I'm completely new to SEO. I think my boss signed up for Moz Pro sometime in February and started adding data to our Ecommerce site to help with rankings. Sometime before this, I changed some of the title tags on the site (trying to help with organic search and CTR). I did not do a site wide change.... just changed maybe 10-20 (just a guess). I did it with keywords in mind but did not make note of when I did it. I didn't really know better at the time, and I did not have access to Google Analytics or Moz Pro. I was looking through the ranking data/graph for February and March. It won't let me look before February 29th (so that's why I think my boss started the Mos Pro subscription around at that time). On that day it said we ranked 12 keywords in the 1-3 spot, and then the following week (march 7) it went down to 6. I don't think or know if any major site changes were implemented, so I'm not sure why that happened and if it has anything to do with my title tag changes I did maybe a week or two before (again I am not sure when I did this unfortunately). Since then the keyword ranking numbers stayed about the same with organic traffic slowly going down (it could be because we are getting out of season for our industry though). The second week of March the site was upgraded and since then the menu has been completely changed around. Last week I did a site wide title tag change. So the minor changes I made in February are no longer in effect anyway. I added more keywords to Moz earlier this week and the number for 1-3 spot keywords went up from 6 to 20. It also says my ranking moved up 4 keywords and down 13 keywords. Anyway, I am wondering how seriously I should take these changes and if I'm damaging the site. I am new to Moz Pro also so all the data you can access is kind of confusing/overwhelming.
Moz Pro | | AliMac260 -
Woocommerce filter urls showing in crawl results, but not indexed?
I'm getting 100's of Duplicate Content warnings for a Woocommerce store I have. The urls are
Moz Pro | | JustinMurray
etc These don't seem to be indexed in google, and the canonical is for the shop base url. These seem to be simply urls generated by Woocommerce filters. Is this simply a false alarm from Moz crawl?0 -
How long for authority to transfer form an old page to a new page via a 301 redirect? (& Moz PA score update?)
Hi How long aproximately does G take to pass authority via a 301 from an old page to its new replacement page ? Does Moz Page Authority reflect this in its score once G has passed it ? All Best
Moz Pro | | Dan-Lawrence
Dan3 -
Crawlers crawl weird long urls
I did a crawl start for the first time and i get many errors, but the weird fact is that the crawler tracks duplicate long, not existing urls. For example (to be clear): there is a page: www.website.com/dogs/dog.html but then it is continuing crawling:
Moz Pro | | r.nijkamp
www.website.com/dogs/dog.html
www.website.com/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dogs/dogs/dog.html what can I do about this? Screaming Frog gave me the same issue, so I know it's something with my website0