How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to help@seomoz.org and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Domain Authority Goes Down After Https change
We just moved our domain from http to https. We have been at a solid 89 for domain authority for the past year, but this month we are now at 88. Does changing to https cause the domain authority to go down and is this just a temporary fluctuation?
Moz Pro | | greenjane0 -
Have a Campaign, but only states 1 page has been crawled by SEOmoz bots. What needs to be done to have all the pages crawled?
We have a campaign running for a client in SEOmoz and only 1 page has been crawled per SEOmoz' data. There are many pages in the site and a new blog with more and more articles posted each month, yet Moz is not crawling anything, aside from maybe the Home page. The odd thing is, Moz is reporting more data on all the other inner pages though for errors, duplicate content, etc... What should we do so all the pages get crawled by Moz? I don't want to delete and start over as we followed all the steps properly when setting up. Thank you for any tips here.
Moz Pro | | WhiteboardCreations0 -
Competitive Domain Analysis Update
The updates to the Competitive Domain Analysis were occuring every 7 days upto Jan 16. When will the next updates be and how often is the new schedule? R/ John
Moz Pro | | TheNorthernOffice790 -
Crawl Errors Confusing Me
The SEOMoz crawl tool is telling me that I have a slew of crawl errors on the blog of one domain. All are related to the MSNbot. And related to trackbacks (which we do want to block, right?) and attachments (makes sense to block those, too) ... any idea why these are crawl issues with MSNbot and not Google? My robots.txt is here: http://www.wevegotthekeys.com/robots.txt. Thanks, MJ
Moz Pro | | mjtaylor0 -
How to Calculate % improvement in Mozrank and Domain Authority
I understand the concept that mozrank and Domain Authority are measured on a logarithmic scale . http://www.seomoz.org/learn-seo/mozrank I'm trying to determine the percentage improvement in the Logrithmic value. For example Original Domain Authority 60 Original Mozrank 5 End Domain Authority 63 End Mozrank 6. So 5% change in DA and 20% change in MozRank. How do we convert these numbers to the real logrithmic change?
Moz Pro | | webbroi0 -
How are our competitors getting these inbound linking domains?
I'm currently managing SEO for my company's website, and I'm getting into link building for the first time. As part of the process, I'm using Open Site Explorer to see who's linking into our competitor sites, to get a better sense of what's available to us in our particular avenue of e-commerce. However, I'm finding that our competitors are getting inbound links from high-authority sites pretty far afield from selling jewelry - census.gov, parallels.com, warnerbros.com, and others. I try clicking through to these links, but each link starts a download of a file. I've seen .f4v, .7z, and .apk files listed as inbound links to our competitor. How is this happening? Again, I'm new to link building, so there may be a simple answer here, and if so I apologize for asking. However, this seems really strange to me, and a difficult situation to confront.
Moz Pro | | jozaksut0 -
How can I increase my domain authority score?
We've jumped up in our rankings, page authority scores, and our toolbar PR over the past year, but our domain authority score remains the same as it was this time last year. I have noticed that for most people's sites, domain authority is typically higher than page authority, and my suspicion is that our domain authority score staying static is capping our page authority potential to some degree. Our SEOmoz scores are quite low for our toolbar PR. I've looked at the SEOmoz page relating to domain authority, and it rather vaguely says 'improve SEO generally across your site'. Other than age, the only thing I can think of is that maybe we have backlinks ranging across too broad a spectrum of topics. Being a web hosting company, we get a lot of links from our customers' sites that aren't related to our industry. Anyone got any specific advice? To provide some context, it's a PR7 website that's around a decade old with links in the hundred thousands. We score just 59 for domain authority. Thanks in advance, Jenni
Moz Pro | | Jenni_HeartInternet0
