Moz crawls
-
Hi all! I'm running two separate campaign crawls at the moment that I'm having a few issues with. The first is http://www.muchbetteradventures.com/.
I'm tracking this at root to try an pick any problems with a previous version of the site, which is on the v1.muchbetteradventures subdomain. However, I'm only seeing 100 or so pages as being crawled in Moz, compared to thousands in Google. There's no access blocked alerts in Moz either.
The second is http://www.sothebys.com/. I started this crawl at root a few days ago and no pages at all have been processed. Very odd!
Any advice would be much appreciated.
-
Thanks Reuben! Just getting my head around Screaming Frog! Can it be used to flag up duplicate content issues like Moz?
-
Hello and great question! If you are ever missing pages from your crawl, you can always email help@moz.com with the name of the campaign and a couple examples of missing pages. We can usually diagnose this issue pretty quickly. In a lot of cases, this could be due to the way some domains are forwarded or redirected. I encourage you to email us the issue with a few examples so that the Help Team, including myself, can dig in and figure out whats going on. Sorry for the delay, I hope we can help get to the bottom of your crawl issues.
-
1. Google's index may or may not be reflective of the pages you actually have on your site—likely the latter, hence the difference between the pages on your site and the pages in Google's index. Right now, the v1 subdomain times out, so no crawlers will be able to do anything with it. The www domain only has about 250 or so URLs.
2. Sotheby's website blocks all but a few crawlers, which is why Moz would not crawl any pages. http://www.sothebys.com/robots.txt
I would suggest using your own crawling software to untangle these issues. If you aren't familiar with Screaming Frog, you should check it out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz have multiple problems.
i cant log in my moz account, it keep me showing error. Moreover, i have seen lately that Moz works slow because it shows my ranking keywords late. Other than this my Domain authority is not increasing even though my site have alot of backlinks
Product Support | | Iuj873g0 -
How to block Rogerbot From Crawling UTM URLs
I am trying to block roger from crawling some UTM urls we have created, but having no luck. My robots.txt file looks like: User-agent: rogerbot Disallow: /?utm_source* This does not seem to be working. Any ideas?
Product Support | | Firestarter-SEO0 -
MOZ not accepting our recent changes it still showing us old Crawl Diagnostics report
Hi, 507 Temporary Redirect We made changes for 302 redirects which are listed in crawl diagnostics report. Now "Compare" and "Wishlist" links are already removed from our source code. All required changes are made but still your report listed Compare and wishlist links. We made changes on Friday (14/8/2015) and waiting for new updated report. Link: http://www.stopwobble.com/ Please let us know what is the exact issue. So that we can fix it.
Product Support | | torbett0 -
Why has my Google Analytics dropped completely from the results on this weeks crawl?
The Account code hasn't changed in either Moz or on site? Also I added 53 new keywords to my campaign and they haven't all been ranked, it's like Moz has done half a job?! S.O.S.
Product Support | | danwebman0 -
Can't add "Custom links" in my Moz Profile
Hi, I was trying to add a custom link to my profile but I got an error message: "There was a problem saving your custom URLs" Can somebody look into it for me? Kind Regards
Product Support | | Laurensvda0 -
Can you create additional users for Moz Analytics?
I have just begun setting up Moz Analytics for some of my clients. It lets me add 5 client's sites to a single account but I can't find anywhere to create an additional user/login for a specific account that way I can give access to each client only to their campaign. Is this not possible and if not something you plan on integrating in the near future?
Product Support | | Carlson3 -
Maybe a silly question but do Moz write guest blogs? If so, who do I ask?
We have refreshed our website and, to supplement the content we write ourselves, we just want some nice, simple, quality written content from our suppliers, of which Moz is one. I'm probably being stupid, but I can't (yet) see how to contact anyone at Moz directly. While my search continues, I'll post this question.
Product Support | | TheInternetWorks0 -
Why are my moz points messed up?
These are probably stupid questions, but they've been bothering me. On my profile page it says complete your profile - earn 10 more moz points! However, I completed my profile and it says it is 100% complete with 0 items left to complete. Is there any reason these points aren't registering? Also, on the header it says my username and that I have 52 moz points, but on my profile page it says I only have 46. Is there a wait time for one of these to update or sync? Also, is there a way to track how many moz points you have gotten in a particular month? I know there are limits to how many points you can get in a given period.
Product Support | | redfishking1