Why does Moz only seem to be crawling a snap shot of the site I am working with?
-
I was wondering if anyone can help? I am working using Moz to help improve the SEO on a website I am working with, the website contains thousands of pages, yet for some reason Moz only seems to be crawling a small snap shot of the website.
I know there are particular pages that I had added a couple of weeks ago - about 300 in total - and none of these were showing on the first crawl, so I did another on-demand crawl and some of these showed up then. Despite this, it says it crawled 700ish pages, but there are getting close to 20-30ish thousand live pages on the site.
Any thoughts and guidance as to why they crawling may be stopping?
-
Hi there,
many thanks for your message. The site is an e-commerce website which currently has just short of 8000 products (each on their own page). These pages are all presented in categories to be able to click on, therefore I would imagine that this is all set up with internal links based on that.
There are numerous products showing, but only 700 pages are coming up in Moz. its weird, because I was expecting some duplication errors on a range that I added, yet these aren't showing in the crawl - but they are definitely there on the site, and under several categories that have links to the categories.
I'm a little stumped at why it doesn't seem to be crawling the full site.
-
Hi there,
Are you internally linking to all these pages that you want Moz to discover? It is possible that those pages are orphaned and they do not have any internal links pointing to them, so Moz cannot crawl them.
Ross
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to seo my site ?
Hi, I'm owner of farsindex.com. I want to seo my site and improve page authority. What are your suggestions?
Getting Started | | amin_material0 -
Page Count per campaign - Crawl Usage 500,000 Pages
How to you find the page crawl count per campaign? I have 3 campaigns and Moz stats I have used 150,000 pages from 500,000. I want to check this. Thanks
Getting Started | | SJMDT0 -
Moz Not Pulling Monthly Data
So I set up my Moz campaigns on the 8th of this month. I can see data for the week, however, I can't see data from the previous week or for the month. So my question has two parts: a.) Is there a way to view data for the previous week, and b.) when can I expect to see data for the entire month. Seeing data for the entire month is absolutely critical for me, since this is what I'm going to use to prepare my monthly client reports. Has anyone experienced anything like this? PLEASE HELP!!!
Getting Started | | maxcarnage0 -
I am new to MOZ, I set up one tracking campaign two weeks ago, I have tracked no keywords, I have done some keyword research for ranking difficulty and in two weeks I have already hit 50K pages crawled, I'm maxed out, is this common?
I am a startup and can't afford the higher plans yet. And even their highest plan is 600K pages crawled, which seems really low considering how lightly I used the tool and how quickly I hit 50K. Does anyone have any advice or information on how they use the tool on lower packages? Did I do something wrong to hit 50K pages crawled that fast? Does this pricing make any sense, it seems like an incredibly high price, I love the tool, any help is appreciated.
Getting Started | | Daedilus1 -
Why does Moz Analytics only let you look at last month / last week?
I cant think of any other reporting product that only lets you look at the last period, am I missing a setting somewhere. I need to take a look at Feb!
Getting Started | | locusm1 -
Campaign.crawl-seed.bad-response ???
Hello Guys I have just tried to set-up a new campaign for this site http://www.emsababies.co.uk/ HoweverI keep getting this error ?? campaign.crawl-seed.bad-response Anyone know what I am doing wrong ?? Cheers James
Getting Started | | BlueNinja1 -
Why Moz Pro Campaign overview display not properly?
when i browse my campaign, i see like this on my computer http://i.imgur.com/7G9IRT8.jpg then when i see source code of http://pro.cdn.seomoz.org/stylesheets/campaigns/overview.css?1378244858 <html><head><title>404 Not Foundtitle>head><body bgcolor="<a class="attribute-value">white</a>"><center><h1>404 Not Foundh1>center><hr><center>nginxcenter>body>html> what happen??
Getting Started | | dimazm0