My site is not being fully crawled
-
Our site has been crawled several times by RogerBot but each time only 6 pages are crawled even though we have more than 100 pages.
Do I need to submit my sitemap.xml to Moz?
-
Hey There,
Thanks for writing in. We actually don't have a way to submit an XML sitemap and we only crawl the site by following the HTML links in the source code. From what I am seeing in your crawl, we are find very few unique internal links on your site, so we are only able to find unique 6 pages using the HTML links.
Unfortunately, it looks like the majority of your site is in Javascript, which our crawler is not able to parse, so we don't see any Javascript links on your site. I'm afraid that a site that is primarily in Java would not be compatible with our crawler.
I apologize for the inconvenience this causes. Please let me know if you have any other questions.
Chiaryn
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the quickest and easiest way to run an SEO audit on a Wordpress site that at least shows all the mechanical problems?
What is the quickest and easiest way to run an SEO audit on a Wordpress site that at least shows all the mechanical problems?
Getting Started | | integratedproperty0 -
Can't Crawl Site - but deducting crawls.
Why am I being deducted crawls if MOZ keeps telling me that it can't crawl my site?
Getting Started | | BloggyMoms1 -
Scheduled update - Re-Crawl - recrawl
Can I not perform a manual update? I setup a campaign without GA as I did not have access, I got access, added the GA account to the campaign but no data is showing as I think I require an update, but have to wait 7 days? Is that right? Thanks
Getting Started | | SJMDT0 -
I have a client with a wordpress.com site.
Is it possible to manage a campaign for such a site on Moz? It looks like in order to be able to add an independent Google Analytics tracking id, he has to upgrade to a business account. Does anybody have any experience with this?
Getting Started | | chill9860 -
What Moz tool is best to find reasons google has not spidered by site
I just joined Moz and am trying to use the tools however, when I attempt to do so every link comes to a that only allows me access to post questions here. If anyone can tell me what tool is best to find reasons google has not indexed my site, I would greatly appreciate the help. Also if anyone knows why I am keep getting routed to this forum when I try to use any of the tools, I would also appreciate help with this. So far Moz is very frustrating.
Getting Started | | Johndeeray19640 -
Hi, I'm looking to find out why a google+ account that was rarely used has 10,000 views. I want to discover what sites it is linked to. I entered the page url but no joy. can anyone help?
I would like to find out where all this traffic is coming from. It is most likely from an out of date sales site etc, but it's important to find out as it could be the result of hacking etc. It appears the page is linked to another site and I would like to find out which one(s) Entering the page url is not getting results, can anyone help?
Getting Started | | cyganswenia0 -
High Number of Crawl Errors for Blog
Hello All, We have been having an issue with very high crawl errors on websites that contain blogs. Here is a screenshot of one of the sites we are dealing with: http://cl.ly/image/0i2Q2O100p2v . Looking through the links that are turning up in the crawl errors, the majority of them (roughly 90%) are auto-generated by the blog's system. This includes category/tag links, archived links, etc. A few examples being: http://www.mysite.com/2004/10/ http://www.mysite.com/2004/10/17/ http://www.mysite.com/tagname As far as I know (please correct me if I'm wrong!), search engines will not penalize you for things like this that appear on auto-generated pages. Also, even if search engines did penalize you, I do not believe we can make a unique meta tag for auto-generate pages. Regardless, our client is very concerned seeing these high number of errors in the reports, even though we have explained the situation to him. Would anyone have any suggestions on how to either 1) tell Moz to ignore these types of errors or 2) adjust the website so that these errors now longer appear in the reports? Thanks so much! Rebecca
Getting Started | | Level2Designs0 -
Is there a way to force re-crawl?
1- Can we force the system to recrawl a website without having to wait for the scheduled date? 2- Please forgive me as I am new to this, but is "Moz Analytics" supposed to replace the "Pro"? Thank you!
Getting Started | | BlackTreeIT0