Moz crawls
-
Hi all! I'm running two separate campaign crawls at the moment that I'm having a few issues with. The first is http://www.muchbetteradventures.com/.
I'm tracking this at root to try an pick any problems with a previous version of the site, which is on the v1.muchbetteradventures subdomain. However, I'm only seeing 100 or so pages as being crawled in Moz, compared to thousands in Google. There's no access blocked alerts in Moz either.
The second is http://www.sothebys.com/. I started this crawl at root a few days ago and no pages at all have been processed. Very odd!
Any advice would be much appreciated.
-
Thanks Reuben! Just getting my head around Screaming Frog! Can it be used to flag up duplicate content issues like Moz?
-
Hello and great question! If you are ever missing pages from your crawl, you can always email help@moz.com with the name of the campaign and a couple examples of missing pages. We can usually diagnose this issue pretty quickly. In a lot of cases, this could be due to the way some domains are forwarded or redirected. I encourage you to email us the issue with a few examples so that the Help Team, including myself, can dig in and figure out whats going on. Sorry for the delay, I hope we can help get to the bottom of your crawl issues.
-
1. Google's index may or may not be reflective of the pages you actually have on your site—likely the latter, hence the difference between the pages on your site and the pages in Google's index. Right now, the v1 subdomain times out, so no crawlers will be able to do anything with it. The www domain only has about 250 or so URLs.
2. Sotheby's website blocks all but a few crawlers, which is why Moz would not crawl any pages. http://www.sothebys.com/robots.txt
I would suggest using your own crawling software to untangle these issues. If you aren't familiar with Screaming Frog, you should check it out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is anyone else experiencing a lot of bugs in moz?
It seems like there have been a lot of bugs in moz over the last couple of days - is anyone else experiencing anything similar? Scheduled reports have sections coming through blank, same thing happens when trying to edit the module in the report - it's just blank. Yesterday, trying to use the KW explorer the SERP portion of the tool would not work and the metrics gathering seemed to be endless. Also been receiving various error messages across many different reports
Product Support | | rder0 -
Moz having a few issues with their site?
I've seemed to have a load of issues as of late. Just small graphical things but still enough to make me concerned. 1. https://moz.com/community/users/4207879 being noindex (however it seems to other users an external tools work) (image proof: https://i.gyazo.com/2e77c94ffd068718e9e149e8afef1e44.png ) 2. No mozpoints: https://gyazo.com/9e0777af3ebbdb6ad58a395bafe28a52 3. I'm a staff member? https://i.gyazo.com/5d7b156c13fa1d0bb6c2e0af3467423f.png 4. Trying to respond when logged out: https://i.gyazo.com/9f5fe8efd372920391a9a54178561e0b.png (not really a bug, however could be a better experience) These have all been sent via support, just wondered if it was just me having issues? Seems i'm staff on this one too: https://i.gyazo.com/6f1bc6372180f4f9f1cb1c7d6f6d941c.png
Product Support | | ThomasHarvey0 -
MOZ Profile is not getting update?
It seems like my profile (especially points) is not been updated since the start of this week. any possible reason?
Product Support | | Mustansar0 -
When will true multi-user log-in be available in Moz Pro? How about an archive?
Our team is growing and we are running into each others' research on the tool! Will our users ever be able to use their own instance of Moz? Also, how about a historical archiving function for research? Would be nice to record past research as similar requests come to us in the future. Wish list stuff - thx.
Product Support | | Jen_Floyd0 -
Changed to HTTPS: Do I need to update Moz?
Hi Everyone! I have moved my website from HTTP to HTTPS. I believe that I've done everything correctly. Now that I've done that, do I need to update my Moz campaign (and how to do that)? Or, will Moz handle it itself? Or, do I need to create a new campaign with the HTTPS url? Thanks so much!
Product Support | | journeybeyondtravel0 -
Why is Moz Crawl Diagnostics labelling pages as duplicate when they appear to be different?
Moz Crawl Diagnostics is flagging some pages on the Doorfit website as duplicate, yet the page content is completely different and not identical. Example. Page: http://www.doorfit.co.uk/locks-security/secondary-security Duplicate: http://www.doorfit.co.uk/seals-and-sealants?cat=279 Does anybody have any suggestions as to why this might be the case? Thanks
Product Support | | A_Q0 -
Is Google Analytics Connection with Moz broken?
Hi, I keep on getting this error on my moz campaigns "Our connection to your Google Analytics account was lost. Don't worry, you won't lose any data. Please reauthorize now." Everyday i click the reauthorize now and make sure everything is sync properly, but I keep on getting the same error everyday, is there something wrong with GA connection with Moz? I really need urgent help with this problem as I have more than 15 campaigns on moz and this tool is very important for my reporting. Thanks
Product Support | | Josh.B0 -
Custom Anchor text link on Moz profile section
Hi, Does anyone know if you can modify your anchor text and URL after you reach 200 Moz points on your profile section (On the web section under image)? I am close to getting my 200 pts (btw...enjoyed the entire journey) and am trying to choose my anchor text. Here is some interesting analysis too. I have a Spanish site since we are an Argentinean firm. 1. Would a spanish word as anchor text have any value since Moz.com is an English site? I am think about semantics here and how Google likes only content relevant links. My thought is that it does today but it may not be true in the future if Google is going to do away with linkbuilding and just do semantics and co-citation. Thanks Carla
Product Support | | Carla_Dawson0