My site is not being fully crawled
-
Our site has been crawled several times by RogerBot but each time only 6 pages are crawled even though we have more than 100 pages.
Do I need to submit my sitemap.xml to Moz?
-
Hey There,
Thanks for writing in. We actually don't have a way to submit an XML sitemap and we only crawl the site by following the HTML links in the source code. From what I am seeing in your crawl, we are find very few unique internal links on your site, so we are only able to find unique 6 pages using the HTML links.
Unfortunately, it looks like the majority of your site is in Javascript, which our crawler is not able to parse, so we don't see any Javascript links on your site. I'm afraid that a site that is primarily in Java would not be compatible with our crawler.
I apologize for the inconvenience this causes. Please let me know if you have any other questions.
Chiaryn
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can Moz Monitor a JS Site?
We are building a new site that, on the blog landing page, uses JS to populate the individual blog article links on the page. The links are not viewable in the page source, but do appear once the page fully renders. After running an on-demand crawl of the site (in QA...not indexed yet), it appears that Moz isn't indexing these pages, and it also isn't reading other page elements that load later (like an H1 that is rendered in JS but not in page source). Are we going to be able to use Moz to track this site? Is there some setting to help?
Getting Started | | Rodrigo-DC0 -
How to seo my site ?
Hi, I'm owner of farsindex.com. I want to seo my site and improve page authority. What are your suggestions?
Getting Started | | amin_material0 -
Moz only crawling one page of a campaign, please help
Today I set up a new campaign for a client, however the crawl has only found the home page and is saying that the URL is unavailable. The site is definitely live and the URL is correct. I have set up the campaign 3 times one with the full address (http://www.) one with www. and with just the domain name. All three of these have come page with one page crawled and "unavailable" above the URL. It is picking up the crawl issues on the page and showing domain authority but I don't know why it's not crawling other pages. Prior to setting up the campaign I did a site crawl and Moz found everything then, so I don't know why it isn't now. Please help. Thanks
Getting Started | | Wrapped0 -
Mozbot Can Not Crawl Entire Domain
I'm trying to crawl Redken.com in Moz Analytics and the Search Diagnostics is only crawling 4 pages. The domain uses a "select your country" the first time you visit, and it seems as though the bot is not getting beyond that (aka, not clicking on "USA") and is therefore not crawling the rest of the domain. There is no country specific URL other than redken.com. I've tried entering both "redken.com" and "www.redken.com" as the URL, but no luck. Any tips?
Getting Started | | LabeliumUSA0 -
How do i stop Moz from indexing my dev site?
Hey, I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google. How do i stop MOZ including it in its reports? Cheers everyone.
Getting Started | | Tholomew1
Bart0 -
How to make site pages appear higher than homepage
Hi, our site sells a mixture of clothes, for example jackets, hats, scarfs and gloves. when somebody searches for 'hats in Chicago' our main website would appear. How can we make it so that our webpage with our hats appears? Thanks
Getting Started | | danieldunn100 -
What is the "other" group in traffic to your site on the moz dashboard ?
What is the "other" group in traffic to your site on the moz analytics dashboard ? "Not provided" is in search - organic search history so it's not that.
Getting Started | | Crocodesign0 -
Can MOZ crawl our website twice in a week?
I want to generate MOZ crawl errors report twice in a week. Is it possible to do that.
Getting Started | | chandman0