Moz crawls
-
Hi all! I'm running two separate campaign crawls at the moment that I'm having a few issues with. The first is http://www.muchbetteradventures.com/.
I'm tracking this at root to try an pick any problems with a previous version of the site, which is on the v1.muchbetteradventures subdomain. However, I'm only seeing 100 or so pages as being crawled in Moz, compared to thousands in Google. There's no access blocked alerts in Moz either.
The second is http://www.sothebys.com/. I started this crawl at root a few days ago and no pages at all have been processed. Very odd!
Any advice would be much appreciated.
-
Thanks Reuben! Just getting my head around Screaming Frog! Can it be used to flag up duplicate content issues like Moz?
-
Hello and great question! If you are ever missing pages from your crawl, you can always email help@moz.com with the name of the campaign and a couple examples of missing pages. We can usually diagnose this issue pretty quickly. In a lot of cases, this could be due to the way some domains are forwarded or redirected. I encourage you to email us the issue with a few examples so that the Help Team, including myself, can dig in and figure out whats going on. Sorry for the delay, I hope we can help get to the bottom of your crawl issues.
-
1. Google's index may or may not be reflective of the pages you actually have on your site—likely the latter, hence the difference between the pages on your site and the pages in Google's index. Right now, the v1 subdomain times out, so no crawlers will be able to do anything with it. The www domain only has about 250 or so URLs.
2. Sotheby's website blocks all but a few crawlers, which is why Moz would not crawl any pages. http://www.sothebys.com/robots.txt
I would suggest using your own crawling software to untangle these issues. If you aren't familiar with Screaming Frog, you should check it out.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved 403 crawl error
Hi, Moz( Also reported by GSC)have reported 403 crawl error on some of my pages. The pages are actually working fine when loaded and no visible issue at all. My web developer told me that some times error issues are reported on a working pages and there is nothing to worry about.
Product Support | | ghrisa65
My question is, will the 403 error have bad consequences on my SEO/Page ranking etc. These are some of the pages that have been reported with 403 error but loading fine: https://www.medistaff24.co.uk/hourly-home-care-in-evesham/ https://www.medistaff24.co.uk/contact-us/0 -
Is it possible to get a refund, I dont want to use MOZ
Well I entered my cc info to try MOZ out. Havent used it so I forgot to cancel my subscription. They have since billed me for a month. I wanted to know if it is possible to get my money back?
Product Support | | Bryton1 -
Moz keeps logging me out
Why moz keeps logging me out? It's annoying that I have to keep putting my credentials after checking the "remember me" checkbox.
Product Support | | sekamoving0 -
Crawl Issue
Hi, We have 3 campaigns running for our websites in different territories. All was going well until April 11th when Moz reported that our .com site (sendmode.com) could not be crawled. I get this error "Your page redirects or links to a page that is outside of the scope of your campaign settings ..." I've been through the site a number of times but have been unable to get to the root of the problem. Robots.txt and 301's look fine. Is there any way I can find out which page is causing the issue? John
Product Support | | johnmc330 -
Crawl still in process for 3 days. Not sure why the site isn't being crawled
I added a new site to the crawl, but it seems to be stalled. It was supposed to crawl Feb 19, but it is still in process Feb 22. It tried to crawl the site and there was a robots.txt issue, but that issue was resolved way before the 19th. Not sure what is going on. this is for the clear lake campaign.
Product Support | | dpsoftware0 -
Duplicate page content on Moz PRO - http://www and http:// problem
Hi Moz community, My Moz Pro writes 555 duplicate content errors, when i click on the page address to check witch page duplicate, it gives me the same page address without WWW. http://www.domain.com/page Duplicated: http://domain.com/page The first thing i checked is my .Htaccess file.. , I found out i have a www redirection that redirect everyone without the "www" to the full domain with the "www" - and it works perfectly on the browser. The code that redirect: RewriteCond %{HTTP_HOST} !^www.
Product Support | | DigitalST
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L] I'm trying to figure out why Moz Pro gives me these errors, Thanks in advanced 🙂0 -
Is the moz site down
anyone else having issues connecting to any campaign etc on Moz on Sat Oct 19? Even emails are bouncing that are sent to help@moz.com....is it just me?
Product Support | | Rye1 -
Trouble adding new campaign to Moz Analytics
Hi all, I've just tried adding a new campaign to our Moz analytics account but it's saying that it already exists. I actually removed all of the campaigns we had in Moz a while ago, so it doesn't already exist. Any one else had the same issues and if so, how did you fix it? Thanks,
Product Support | | PinpointDesigns
Lewis0