How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to help@seomoz.org and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My domain authority is low in MOZ and high in other apps
_I checked my site in other SEO apps and they are showing a higher DA for my site. However, in MOZ is very low. Can you please explain it to me? _what caused the difference?
Moz Pro | | K_Monestel0 -
Domain Authority Dropped from 13 to 1
Hi there, I know Domain Authority is not a KPI, it's a relative metric but we saw a huge dropped down in our domain authority this week and I got worried. I am trying to understand why this happened? Any idea? Our website link is here https://www.wealthmanagementcanada.com Is it something related to Moz update or with our website? Regards,
Moz Pro | | ahmetkul
Ahmet0 -
Root domain or 'www' subdomain for campaigns?
I am setting up a campaign and I am unsure whether I should specify the root domain (prohibitionhats.co.nz) or the 'www' subdomain (www.prohibitionhats.co.nz). Are there any guidelines for this or will Moz automatically aggregate the data for both irrespective of the domain you choose? Thanks Laurie
Moz Pro | | turnbullholdingsltd0 -
Root Domain Campaign track subdomains¿?
I have set the campaign for the Root domain, hw, I'm receiving issues for both subdomains, demo and www. Our site is set to redirect all non www traffic to www, except demo, who has its own rules. Is that normal? Should I contact the support team? Thanks
Moz Pro | | MilosMilcom0 -
I have corrected the Problems in Crawl Diagnostics. When would it refresh/ re-crawl my site ?
I have corrected most of the problems shown in crawl diagnostics and changed the meta desc. , titles etc. When will SEOMOZ recrawl those pages and show that Its correct now ?
Moz Pro | | VarunBansal0 -
Falling Domain Authority
What are the factors that would cause my domain authority to fall? As best I understand it, we would have to be losing links. For example, one of my clients started with a DA around 60 and now 4 months on it is sitting around 47. This has happened with a couple clients and I'm not really sure why or how to counteract it.
Moz Pro | | Dekoda_Jones0 -
Crawl Errors Confusing Me
The SEOMoz crawl tool is telling me that I have a slew of crawl errors on the blog of one domain. All are related to the MSNbot. And related to trackbacks (which we do want to block, right?) and attachments (makes sense to block those, too) ... any idea why these are crawl issues with MSNbot and not Google? My robots.txt is here: http://www.wevegotthekeys.com/robots.txt. Thanks, MJ
Moz Pro | | mjtaylor0 -
Crawl Report Technical Issue
I'm having a problem with our campaign especially the crawl report. Because the last update is Nov 4 and isn't supposed to be updated weekly? I already submit a helpdesk support ticket and even send a seperate e-mail regarding this issue but until now the report is still not updated. Anybody here can help me raise this issue/ Thanks.
Moz Pro | | shebinhassan0