Moz crawls
-
Hi all! I'm running two separate campaign crawls at the moment that I'm having a few issues with. The first is http://www.muchbetteradventures.com/.
I'm tracking this at root to try an pick any problems with a previous version of the site, which is on the v1.muchbetteradventures subdomain. However, I'm only seeing 100 or so pages as being crawled in Moz, compared to thousands in Google. There's no access blocked alerts in Moz either.
The second is http://www.sothebys.com/. I started this crawl at root a few days ago and no pages at all have been processed. Very odd!
Any advice would be much appreciated.
-
Thanks Reuben! Just getting my head around Screaming Frog! Can it be used to flag up duplicate content issues like Moz?
-
Hello and great question! If you are ever missing pages from your crawl, you can always email help@moz.com with the name of the campaign and a couple examples of missing pages. We can usually diagnose this issue pretty quickly. In a lot of cases, this could be due to the way some domains are forwarded or redirected. I encourage you to email us the issue with a few examples so that the Help Team, including myself, can dig in and figure out whats going on. Sorry for the delay, I hope we can help get to the bottom of your crawl issues.
-
1. Google's index may or may not be reflective of the pages you actually have on your site—likely the latter, hence the difference between the pages on your site and the pages in Google's index. Right now, the v1 subdomain times out, so no crawlers will be able to do anything with it. The www domain only has about 250 or so URLs.
2. Sotheby's website blocks all but a few crawlers, which is why Moz would not crawl any pages. http://www.sothebys.com/robots.txt
I would suggest using your own crawling software to untangle these issues. If you aren't familiar with Screaming Frog, you should check it out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Remove account from MOZ
I have subscribed for 1 month free Moz trial account. It was free but to subscribe this account I need to put my credit card details. Suddenly at the end of this 1 month term period, Moz tried to cut 149 Dollar from my account without notifying me. But they could not because there were insufficient funds on my credit card. I haven't used any services and would like to remove my account from Moz service? Would appreciate your assistance.
Product Support | | edgarsb0 -
Can 2 Moz profiles be merged?
I'm very excited to have received my promotion into the Moz Specialist category (yea me!) , but I've discovered that I have another Moz profile dating back to 2011. Can they be merged together?
Product Support | | julie-getonthemap0 -
Do Archived campaigns contribute to total keyword usage allowed in Moz plan?
We're debating whether to archive some campaigns or delete them entirely. One factor in consideration is if Archived campaigns count towards total keywords allowed in our Moz subscription. Thank you for your insights!
Product Support | | Bonfire0 -
Why is Moz tracking search results for UK if I have my campaign settings set for US?
One of our clients is marked in Moz Analytics as ranking 24th, but we don't see that when we search for the same key term. However, when we hover over the Universal Image Results icon, we noticed that the ranking is actually for google.co.uk/search. The campaign settings for this campaign are all set to use google.com or the US version. Does anyone know why Moz is bringing in these UK results? dKJAU2F
Product Support | | garrettkite0 -
Reset my MOZ Account?
I recently redesigned my 2 websites. Both are large 80+ page sites with unique text on every page, no keyword stuffing and generally using all the proper SEO techniques ( I hired a professional w/ 10 years experience). My pages rank for the targeted terms, but MOZ reports say "Not in the Top 50". I had friends run "Clean Searches" from the west coast and we were ranking. Prior to the redesign, I used MOZ and the reports were very accurate. Should I reset my MOZ account or what?
Product Support | | KevnJr0