SEOmoz crawler not crawling my site
-
We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled?
We set up the campaign to track at the root domain level.
-
Thanks, Ruth. I see what you're saying about the crawl now. I'll go ahead and reach out the help team. Hopefully we can figure it out!
-
So that's really what I meant was "as long as you're seeing the full crawl," was that if your next crawl found the whole site it would probably be fine. It sounds like that's not the case, so I think the Help Team is your best bet - they're far more able to diagnose campaign-specific issues than I am
-
Thanks for the reply Ruth!
I was a little unclear as to part of your response. What were you referring to when you said "as long as you continue seeing the full crawl, you should be good to go"?
Where should I be seeing the full crawl? When I look at the Crawl Diagnostic Summary, it notes that the last crawl was on May 10, 2013 (the day we set up the campaign) and the next crawl is to start May 18, 2013. I received email notification that SEOmoz was successfully recrawled it May 11, 2013. Similar to the May 10 crawl, the May 11 crawl only went through 2 pages.
I also checked up on our robots.txt and tags. There isn't blocking anything the crawler on that front.
-
Hmm! Well, it could be a couple of things.
If your full crawl ends up crawling the whole site, it's possible that the preliminary crawl timed out on a request to your site. As long as you continue seeing the full crawl, you should be good to go.
If your next crawl continues showing just a few pages, I would check to make sure you're not blocking robots with either the meta robots tag, a robots.txt file, or nofollowed links. If you don't uncover anything out of the ordinary there, it could be an account- or campaign-specific error on our end; please email help@seomoz.org and our super-awesome Help Team can help get that fixed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does SEOmoz have a tool to find mirror sites?
I heard from a company that is trying to get my clients SEO business that they discovered multiple sites mirroring our site's content. Does SEOmoz have a tool to find these websites? Or does Google?
Moz Pro | | thomas.wittine0 -
Third crawl of my sites back to 250 pages
Hi all, I've been waiting some days for the third crawl of my sites, but SEOMOZ only crawled 277 pages. The next phrase appeared on my crawl report: Pages Crawled: 277 | Limit: 250 My last 2 crawls were of about 10K limit. Any idea? Kind regards, Simon.
Moz Pro | | Aureka0 -
Pages Crawled: 0 ?
I've been with SEO Moz for over a month and a half. Why would this weeks crawl have Pages Crawled: 0? I've made no changes since the crawl last week that had 10k pages crawled...
Moz Pro | | mr_w1 -
Archiving Campaigns in SEOmoz
First off, I love the campaign archive feature. Very useful for my purposes. My question is: Is there a limit to how many campaigns I can archive? Thanks in advance!
Moz Pro | | CollinJarman0 -
Seomoz crawling filtered pages
Hi, I just checked an seo campaign we started last week, so I opened seomoz to see the crawl diagnostics. Lot's of duplicate content & duplicate titles showing up, but that's because Rogerbot is crawling all of the filtered pages as well. How do I exclude these pages from being crawled? /product/brand-x/3969?order=brand&sortorder=ASC
Moz Pro | | nvs.nim
/product/brand-x/3969?order=popular&sortorder=ASC
/product/brand-x/3969?order=popular&sortorder=DESC&page=10
/product/brand-x/3969?order=popular&sortorder=DESC&page=110 -
Help Understanding Crawl results on this site
I'm just starting to SEO this site http://thefirmbusinessbrokerage.com/welcome and I'm having trouble with the crawl report data. First question, should I be building links to the site above or the main page http://thefirmbusinessbrokerage.com/ (which is a flash intro). If I build links to the flash page, what do I do to the forwarding URL to the welcome page to make it effective? Second question, why does the crawl data report show up almost completely blank? Is this site perfect or are there some onsite issues that I'm not seeing. Thanks for your support and guidance on this site. I'm not hosting the site, just building links and offering optimization advice onsite. JOE
Moz Pro | | KreativElement0 -
SEOmoz PRO campaign fot HTTPS site
Hi all, I'm trying to configure a PRO campagin for a https website. Butt it won't work. The software says it found a one redirect (for http to https I guess), and that's it. So now I don't have any data.... Can anybody help me? Thnx! Martijn
Moz Pro | | Men4Media0 -
HTTPS site in Open Site Explorer
I'm looking at a site for which the https URL currently ranks in Google. Using a header checker on the http URL I see that it is being 302 redirected to the https version (I have no control or input on this site). In OSE there's no option to specify an https URL as the http part is pre-populated and uneditable. My question is: does OSE treat the https and http version as the same URL? I'm guessing so as the http URL has a lot of domain authority despite not being the "default" URL.
Moz Pro | | Equatorites0