How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to help@seomoz.org and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Domain Authority
Has there been an update re DA since there has been signifanct drops the lat 24 hours?
Moz Pro | | NineFiftyNine.net0 -
Relation between domain age and domain authority?
what is relation between domain age and domain authority? Old registered domain help for domain authority higher or not? if so, but i am still in confused, http://www.green-lotus-trekking.com/ this is too old domain but authority is only 33?
Moz Pro | | agsln1 -
Only One page crawled..Need help
I have run a website in Seomoz which have many URLs with it. But when I saw the seomoz report that showing Pages Crawled: 1. Why this is happen my campaign limit is OK Tell me what to do for all page crawling in seomoz report. wV6fMWx
Moz Pro | | lucidsoftech0 -
How do I get our full domain link profile?
With Open Site Explorer I have exported our domain link profile but I only get 65 domains returned when our marketing software reports that we have 315 . . . I know that some domains point to www.bluelink.ca which redirects to www.bluelinkerp.com. Is Open Site Explorer missing these links?
Moz Pro | | BlueLinkERP0 -
Freaked out over domain authority ranking drop.
Should I be freaked out that website's domain authority dropped 4 points? Is it normal for websites to fluctuate a couple of points or have I really screwed something up?
Moz Pro | | bewoldt0 -
Issue in number of pages crawled
i wanted to figure out how our friend Roger Bot works. On the first crawl of one of my large sites, the number of pages crawled stopped at 10000 (due to the restriction on the pro account). However after a few weeks, the number of pages crawled went down to about 5500. This number seemed to be a more accurate count of the pages on our site. Today, it seems that Roger Bot has completed another crawl and the number is up to 10000 again. I know there has been no downtime on our site, and the items that we fixed on our site did not reduce or increase the number of pages we had. Just making sure there are no known issues with Roger Bot before I look deeper into our site to see if there is an issue. Thanks!
Moz Pro | | cchhita0 -
SEomoz slow to crawl?
Hello - I am just trying out the trial and it said the next crawl was nov 1st but I see no change in any of the errors since the initial crawl... so just waiting to find out if what I changed was fixed or not. Is this normal ?
Moz Pro | | Bethany_BabyBrowns0 -
How can I increase my domain authority score?
We've jumped up in our rankings, page authority scores, and our toolbar PR over the past year, but our domain authority score remains the same as it was this time last year. I have noticed that for most people's sites, domain authority is typically higher than page authority, and my suspicion is that our domain authority score staying static is capping our page authority potential to some degree. Our SEOmoz scores are quite low for our toolbar PR. I've looked at the SEOmoz page relating to domain authority, and it rather vaguely says 'improve SEO generally across your site'. Other than age, the only thing I can think of is that maybe we have backlinks ranging across too broad a spectrum of topics. Being a web hosting company, we get a lot of links from our customers' sites that aren't related to our industry. Anyone got any specific advice? To provide some context, it's a PR7 website that's around a decade old with links in the hundred thousands. We score just 59 for domain authority. Thanks in advance, Jenni
Moz Pro | | Jenni_HeartInternet0