Can't Crawl Site - but deducting crawls.
-
Why am I being deducted crawls if MOZ keeps telling me that it can't crawl my site?
-
Any time you initiate a crawl, it counts against your limits, even if something on the site is blocking us from crawling it. I would suggest that write into help@moz.com so we can help you diagnose what is blocking us from crawling your site.
-
Sometimes, when running a crawl on a site (or a section of a site), you may find that it isn’t progressing past the first level of URLs. When this happens, only the base domain, or “start URLs” are actually crawled
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved What would the exact text be for robots.txt to stop Moz crawling a subdomain?
I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this. I assume it would be: User-agent: Moz
Getting Started | | Simon-Plan
Disallow: / But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text! Many thanks.0 -
Site Crawl - Crawls only homepage?
Hi Moz Comunity! Joined Moz just 2 weeks ago and slowly trying to get used to tools available in here! Great tools and info available on this site! My concern is that Site Crawl of Moz in my Campaign seems to have crawled only my homepage and no other sub-domains, is there any reason for this? FOr some reason it seems that Moz interacts only with my homepage? Even when I tried the Keyword Exlporer set on Keyword to see if any of my pages rank for any keywords, it seems only my homepage was ranking for a few keywords. It's possible my other sub-domains don't rank for any keywords yet but still, seems suspicious... I have added a link to Site Crawl that says it has crawled only 3 pages on my site, and all are my homepage... Thanks for any help! Jacob s!AlxV7sobbcgmhJB_fXcF4EPzbPSovA
Getting Started | | Shotlife_Studio0 -
I haven't used Moz and you've been billing me. Please refund thanks
I signed on for a free trial and found out today that you've billed me in June and July. Check my history as this is only the second time i've logged in and it's just to request you refund.
Getting Started | | Completebanker0 -
901 error code showing url back to back in crawl
Hi Everyone, I'm absolutely dumbfounded about this 901 issue (showing pages with our url back to back). Our site is hosted on Big Commerce: https://www.santabarbarachocolate.com When I look for these pages being crawled I don't find them. I've called BC for help and I can't seem to find a solution or where to turn as to how to fix the issue at hand or even if it matters. Please see below what the Moz crawl shows. Could this be related to Yotpo or some app we have running? Or does this even matter and does it have any influence on rank? Do you have recommendations or ideas? Thanks so much. Pages with Crawl Attempt Error as of Mar 3 URL Page Authority Linking Root Domains Status Code | Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/100-percent-pure-cacao-unsweetened-baking-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/buy-wholesale-bulk-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/organic-chocolate-wholesale | -- | -- | 901 |
Getting Started | | santabarbarachocolate0 -
I signed up for a free trial yesterday. Today I wanted to analyze some Twitter accounts and got a message that I had reached my daily quota after only two searches! What's going on?
When I first discovered Followerwonk, I searched 15 accounts before reaching my daily quota. Now that I have signed up for a Pro account, how come I can only search two? How come there's any daily limit at all now?
Getting Started | | allpurposeguru0 -
Can't setup new campaign
Hi everyone, I'm trying to set up a new campaign for a website which has Cloudflare installed. After I enter the campaign name and URL the loading circle comes up and spins for a while, but then it just stays on the same page. No error message is given. I can't get to the next page of the setup campaign form sequence so that I can set up this campaign. Has anyone else had this problem and is there any fix? Thanks in advance
Getting Started | | _jrmo
James0 -
Cant download my crawl csv
When I click on the [download csv] in my crawl campaign section nothing happens.
Getting Started | | digitalmedialounge0 -
How do you "Moz Crawl" a website? Newbie...
Hi everyone; I've used Screaming Frog in the past and it's simple: you enter the URL to the box and click "start" and.. voila: as the button says, the crawling starts. I've had the Pro version of Moz for a while now and haven't really 'done' anything with it. I'd like to crawl a website and thought it would be as easy as it's always been with Screaming Frog... but, for some reason, I can't find the 'way' to do it. I find it really frustrating especially cause I feel like an idiot going around in circles thinking I'm missing something really obvious... Until I realised the only solution was to ask here! So... how in the world do you crawl a website using Moz tools? (Pro version) Thanks!
Getting Started | | patrihernandez1