Moz crawling doesn't show all of my pages
-
Hello,
I'm trying to make an SEO on-pages audit of my website.
When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ?
How can i check and correct this. My website is discoolver.com
-
Hi there,
Sam from Moz's Help Team here!
So, I took a look at your site, and unfortunately I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
-
Thanks for the answers.
I believe the issue is about a rel=canonical from the homepage which links to an external website...
Can this tag prevent a good website crawl ?
-
So just looking it appears you don't have a robots.txt file or a sitemap. At least one that I could easily find.
This Moz post might help when creating one.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why moz is not crawling my Backlinks & Keyword ?
Hello all brother & sisters, I want to know why moz bot crawler are not detected my backlinks & keyword from last 3-4 months not even single one. All other SEO crawler less popular than moz are detecting my keyword & backlinks. May I know is there any option to forcefully moz bot to crawl my all site specially.
Link Explorer | | loknath.tcms1 -
Does the Moz Pro site crawl, crawl password protected sites?
So i asked Moz Pro site crawl to crawl my page, and a lot of issues came up - but for password protected sites. Does the Moz Pro site crawl do this? A lot of the issues, are not relevant for a site that is password protected.
Link Explorer | | Minlaering.dk0 -
How can I tell what pages have links TO this page?
Hello, Mozzers! My client has a rather messy situation, with redirects for HTTP vs. HTTPS, and two different home pages, both in subdirectories. I could explain the whole setup, but I don't think it's relevant for my question, which is this: OSE is showing three links to the page in question, but doesn't tell me WHAT those links are. Am I just missing the information somewhere? Please see screenshot for details. Thanks so much! ~ Scott view?usp=sharing
Link Explorer | | measurableROI0 -
How long will it take for the changes we've made to reflect in Moz OSE spam score data?
I signed up for Moz to see the spam flags our site had triggered. As soon as I found out, we worked on it and have been trying to correct our mistakes but it's been more than a month and we've managed to neutralise zero flags. I would appreciate if someone can clarify how long the OSE data takes to refresh. Also, how do you combat the following three specific flags: Ratio of Followed to Nofollowed Subdomains Ratio of Followed to Nofollowed Domains Low Number of Pages Found Crawl only gets a valid response to a small number of pages. Thanks.
Link Explorer | | Oziti0 -
Page Authority = 1 of linking page?
I am trying to determine if any of our PR services or inbound links are hurting our SEO. To do this I am using the Open Site Explorer tool and checking some of the URL's that link to us. While doing this I found a page that has a Page authority of 1/100 most other links have a decent authority...does this score mean this page has been penalized by Google? If not what else could a 1/100 mean?
Link Explorer | | Bertrand131 -
How can I get a Moz crawl report of 404 errors on my site
I have a Moz subscription and I see dead links on my website that link externally. Is there a Moz crawl report which will show me these 404 errors and which pages on my site those 404 links are on?
Link Explorer | | Marbanasin0 -
Why won't Open Site Explorer detect inbound links from big name websites?
I understand smaller, spammy sites woudn't get indexed, but some of the websites linking to mine have high domain authorities. The website is a parked subdomain.
Link Explorer | | myers3680 -
Why does facebook, twitter, pinterest show up for links?
Hello, As I have used site explorer on competition sites, I am curious what I need to do to have my facebook, twitter, and pinterest links to shop up as a backlink? I see some other sites have it, even if its a nofollow, but curious if I am doing something wrong? I have had accounts since last year. Just curious if I am doing something wrong, Thank you Chris
Link Explorer | | Berner0