Moz crawls
-
Hi all! I'm running two separate campaign crawls at the moment that I'm having a few issues with. The first is http://www.muchbetteradventures.com/.
I'm tracking this at root to try an pick any problems with a previous version of the site, which is on the v1.muchbetteradventures subdomain. However, I'm only seeing 100 or so pages as being crawled in Moz, compared to thousands in Google. There's no access blocked alerts in Moz either.
The second is http://www.sothebys.com/. I started this crawl at root a few days ago and no pages at all have been processed. Very odd!
Any advice would be much appreciated.
-
Thanks Reuben! Just getting my head around Screaming Frog! Can it be used to flag up duplicate content issues like Moz?
-
Hello and great question! If you are ever missing pages from your crawl, you can always email help@moz.com with the name of the campaign and a couple examples of missing pages. We can usually diagnose this issue pretty quickly. In a lot of cases, this could be due to the way some domains are forwarded or redirected. I encourage you to email us the issue with a few examples so that the Help Team, including myself, can dig in and figure out whats going on. Sorry for the delay, I hope we can help get to the bottom of your crawl issues.
-
1. Google's index may or may not be reflective of the pages you actually have on your site—likely the latter, hence the difference between the pages on your site and the pages in Google's index. Right now, the v1 subdomain times out, so no crawlers will be able to do anything with it. The www domain only has about 250 or so URLs.
2. Sotheby's website blocks all but a few crawlers, which is why Moz would not crawl any pages. http://www.sothebys.com/robots.txt
I would suggest using your own crawling software to untangle these issues. If you aren't familiar with Screaming Frog, you should check it out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to remove staging Urls from being shown in MOZ as critical crawler issues?
Hi MOZs, A few months ago we were updating our website and used a staging site to do so. Soon after we realized that it was indexed by G and shown in Search so we removed it to be indexed. The staging site has since not been visible in search except for 2 links that I just found out about today. My question is how to remove the staging site URLs being shown in MOZ as crawl issues or remove completely? I want to do this because the digital reports are connected to MOZ and it shows there are currently more than 400x 4xx errors which are all because of the staging URLs. I already marked to ignore the issues but they are still showing in. Any help would be much appreciated! Thanks
Product Support | | Strausberg0 -
Website can't be crawled
Hi there, One of our website can't be crawled. We did get the error emails from you (Moz) but we can't find the solution. Can you please help me? Thanks, Tamara
Product Support | | Yenlo0 -
I have been not using MOZ, my log in can give details,and i was charged for plan of 149$.please refund my money.never did used any tool,it s all inactive .
i have been not using MOZ, my log in can give details,and i was charged for plan of 149$.please refund my money.never did used any tool,it s all inactive .
Product Support | | rush70720 -
I canceled my free trial. Does Moz charge after expiring date although I canceled the free trial?
I canceled my free trial, will Moz charge after expiring date, unless I canceled?
Product Support | | Samantha-Amarakoon0 -
I have suddenly got a lot of duplicate page title errors in Moz, please can you advise what to do ?
my site used to be http and i have now converted to https. my site is www.paulsummerfieldphotography.co.uk, please advise how to solve the duplicate page title errors ?
Product Support | | Paul_Ward0 -
No crawl data anymore
Using moz quite some time, but I don't have any crawl data anymore. What happened? (www.kbc.be)
Product Support | | KBC
http://analytics.moz.com/settings/campaign/517920.11285160 -
Moz Netiquette
It's now happened three times in three days: I get unsolicited email (apparently harvested from my Moz profile) that invites me to try a new product or contribute to a new site. The senders are completely unknown to me. The emails are full of hype, sketchy on details and poorly written. They have all the hallmarks of phishing scams. But on further inquiry the emails seem to be legit. I nonetheless feel entitled to confess my extreme exasperation. When did sending unsolicited mass email of the "Dear Colleague" variety to perfect strangers become acceptable business practice? It's always struck me as being beyond the pale and beneath contempt. Why would anyone try to launch a business which such dubious and unsavoury tactics? Am I missing something here? Or becoming a fuddy duddy? I suppose I could remove my contact info from my MOZ profile, but I have found it useful in the past. What surprises me is that people would contact me "out of the blue" to ask for free help without making the slightest attempt to cultivate a relationship first. Am I wrong?
Product Support | | DanielFreedman1 -
Moz client structure.
I am new to MOZ and am curious if each of my clients should have their own MOZ pro account or if I, ( ME the designer & optimizer ) just needs an account and i add each of the clients website to my account? ie. do i log into 6 different clients MOZ accounts? or do i personally create a MOZ account and each site is set up as a Campaign? any advice? thanks,
Product Support | | EarGear
Chris0