Crawling Website
-
How often does Moz crawl the website? I have a lot of high priority crawl issues but my web development team says that it fixed them. However, Moz still shows me that none of the errors are fixed.
-
The crawl test will show the results in a CSV file, but not update on your dashboard. It is a way to spot-check and see if the changes have been made.
When you go to your crawl diagnostics page, it will tell you the date of the last crawl. We crawl on the same day each week.
-
So will it update automatically in my overview as well? When will it be crawled on a weekly basis, in the beginning of the week or end of the week? How do I know if my website has been crawled yet?
-
We crawl on a weekly basis. You can force a crawl for up to 3000 URLs at the crawl test at https://moz.com/researchtools/crawl-test.
-
Please see this discussion to help you out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawling issue
Hi, I have to set up a campaign for a webshop. This webshop is a subdomain itself. First question: The two subfolders I need to track are /nl_BE and /fr_BE. What is the best way to handle this? Shall I set up two different campaigns for each subfolder, or shall I just make one campaign and add tags to keywords? **Second question: **it seems like Moz can't crawl enough pages. There are no disallows in the robots.txt. Should I try putting the following at the top into my robots.txt? User-agent: rogerbot
Getting Started | | Mat_C
Disallow: Or is it because I want to crawl only a subdomain that it doesn't work? Thanks0 -
Moz Site Crawl can't index WIX sites
We've been attempting to work on some SEO for a new potential client however they are using a WIX site. We've noticed that Moz SEO tools will not index any WIX sites. e.g. https://www.sharonradisch.com/ (which is one of their case studies). Anyone seen this that can offer any advice? Thanks,
Getting Started | | monkeex
Mark2 -
How to locate page with the duplicate title? (Crawl Diagnostics - Duplicate Titles Warning)
I am looking through my crawl diagnostics and one of my errors states that a page has a duplicate title. My problem is that I do not know how to find the duplicate. Any advice here?
Getting Started | | bearpaw0 -
I am managing an existing ecommerce website and just subscribed to the MOZ tools - what is the best rout to learning how bets leverage all the tools to optimize my site?
I am managing an existing ecommerce website and just subscribed to the MOZ tools - what is the best rout to learning how bets leverage all the tools to optimize my site?
Getting Started | | DiveNSail0 -
Before making improvements, I need to collate the most relevant custom report to snapshot our current website quality.
I've just started at a company and before I start working on content and SEO improvements I need to assess 'where are we now' . The idea being after 3-6 months I can repeat the report to show improvements. Moz has an enormous range of powerful analysis tools - which should I combine into a Custom Report to create a snapshot of current website quality? Thanks
Getting Started | | ElementaryUK0 -
Moz Analytics set up: should I use www or not when I do website set up?
Hello, When doing website set up will www.mysite.com yield different results than just mysite.com ? Same about competitors: should I use www or not when I set up their sites? Or it won't matter? Thanks,
Getting Started | | dstepans
Denis0 -
Campaign.crawl-seed.bad-response ???
Hello Guys I have just tried to set-up a new campaign for this site http://www.emsababies.co.uk/ HoweverI keep getting this error ?? campaign.crawl-seed.bad-response Anyone know what I am doing wrong ?? Cheers James
Getting Started | | BlueNinja1 -
What are the solutions for Crawl Diagnostics?
Hi Mozers, I am pretty new to SEO and wanted to know what are the solutions for the various errors reported in the crawl diagnostics and if this question has been asked, please guide me in the right directions. Following are queries specific to my site just need help with these 2 only: 1. Error 404: (About 60 errors) : These are for all the PA 1 links and are no longer in the server, what do i do with these? 2. Duplicate Page Content and Title ( About 5000) : Most of these are automatic URL;s that are generated when someone fills any info on our website. What do I do with these URL;s. they are for example: _www.abc.fr/signup.php?_id=001 and then www.abc.fr/signup.php?id=002 and so on. What do I need to do and how? Plzz. Any help would be highly appreciated. I have read a lot on the forums about duplicate content but dont know how to implement this in my case, please advise. Thanks in advance. CY
Getting Started | | Abhi81870