How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to help@seomoz.org and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to check if a domain has been penalised before I buy?
I guess the question is in the subject. We have a client that wishes to buy a domain and use it to replace the primary URL on their website. Although the domain is currently not in use, it was back in 2002 until about 2010. We're concerned that this could cause issues if we change the name over and the domain itself has had penalties in the past
Moz Pro | | AbsoluteDesign0 -
Moz crawling doesn't show all of my Backlinks
Hello, I'm trying to make an SEO backlinks report on my website When using the Link Explorer, I see only a few backlinks while I have much more backlinks on this website. Anyone has an idea about how to fix this issue. How can I check and correct this? My website is www.signsny.com.
Moz Pro | | signsny1 -
What to look for with Domain Authority?
Hi, Our site has fluctuated between 20 and 22 DA for six months, but today it jumped to 27. Is that a sign we are doing something right or is 5 points within the DA's fluctuations pretty standard? Thanks, Ruben
Moz Pro | | KempRugeLawGroup0 -
Campaign Crawl
I have a site with 8036 pages in my sitemap index. But the MozBot only Crawled 2169 pages. It's been several months and each week it crawls roughly the same number of pages. Any idea why I'm not getting fully crawled?
Moz Pro | | JMFieldMarketing0 -
Is there a way you can determine the time SEOMoz crawls your website
I'm a new user to SEOMoz Pro, and having created a number of campaigns I wanted to know if you can set / schedule when SEOMoz crawls your website? Ideally I want to prevent SEOMoz from crawling my site at certain times, but I can't seem to find a way of doing this. Thanks in advance.
Moz Pro | | Simon_Glanville0 -
Is there a easy way to see what pages are crawled?
Hello! Like the questions says... Is there a easy way to see what pages are crawled? I don't mean the ones that have issues, but just the ones that have been crawled? Regards,
Moz Pro | | MattDG0 -
Why does my crawl diagnostics show duplicate content
My crawl diagnostics show duplicate content at mysite.com and mysite.com/index.html which are essentially the same file.
Moz Pro | | MSSBConsulting0 -
Why does not the Linking Root Domains pick update?
The linking root domains is not reading the linking domains, Google shows 38, seo tool shows 180 another seo shows 92 and site explore shows 5.
Moz Pro | | 1step2heaven120