Crawl a node js page - Why can I only see my frontpage?
-
Hi
When i am trying to crawl my website ( https://www.doorot.com/ ) it can only find my frontpage. It's a node js page. Any one had the same problem or know how to crawl my site in order to see all my pages?
Kasper
-
Hi there!
Sam from Moz's Help Team here - I'm sorry about the trouble!
Unfortunately, this would be a result of our crawler's inability to work with Javascript. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real work around that I can recommend for this one, since it is a technical limitation, but I did find some good blog posts and discussions in the Q&A about this when I searched our Help Hub. While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
I'm really sorry I can't be of more help here; I'll definitely be sure to pass this along to our Product team as feedback on your behalf. Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz crawling http rather than https site
Our site is secure but when I ask moz to crawl it by giving the root domain including https moz insists on crawling the non secure version. How do i force it to crawl the secure version?
Link Explorer | | media12340 -
Crawl Errors on a Wordpress Website
I am getting a 902 error, "Network Errors Prevented Crawler from Contacting Server" when requesting a site crawl on my wordpress website, https://www.systemoneservices.com. I think the error may be related to site speed and caching, but request a second opinion and potential solutions. Thanks, Rich
Link Explorer | | rweede0 -
How can I get links indexed in Just Discovered Links, or Open site Explorer? (Because it aint Bit.ly or Feedly!)
Dear Moz & Forum, I've been on a bit of a merry-go-round with one of my clients. He knows enough about SEO to ask questions, and one thing he likes is to see the links that we are building inside Open Site Explorer. Now, of course we know that it takes a while for links to show, but that was ok, because we could just plug the links that we've built into a bit.ly link and then they'd show under Just Discovered Links in Open Site Explorer, and then boom, we're happy. It stopped working some time ago, and after the support team were a little unstuck I sent a hopeful tweet to @randfish. Here is the response: https://twitter.com/randfish/status/788412272129347584 The advice to use Bit.Ly was removed swiftly after this! Oops! The current Guidance is here: https://moz.com/help/guides/research-tools/open-site-explorer/justdiscovered It says: Want to see your new link in the Just-Discovered Links tab? An easy way to start seeing your link in the Just-Discovered section is to have it shared on Twitter and available on Feedly link. Feedly stopped supporting their own link shortner a few years ago. Also, when you share a link from Feedly on twitter it is just the same as sharing a normal tweet, because it shares the standard URL for the site not a link to view on Feedly. Or, maybe these two statements are independant of one another and they should both be done separately? I've done that too! I've also added the feed to recommended feeds inside fresh web explorer and that didn't work either! So, Moz and Help team please help me! How can I get links I've built for clients into Just Discovered Links? Thank you!
Link Explorer | | stevestretton11 -
Can you compare social profiles (FB, Twitter, Instagram, Linkedin) for sites - in a similar way you can compare sites using OSE
Can you compare social profiles (FB, Twitter, Instagram, Linkedin) for competitors sites - in a similar way to how you can compare sites stats - (DA, PA, Backlinks etc) using OSE's compare tool?
Link Explorer | | mike8780 -
None of the pages crawled contain an email address or links to a social profile...
I'm trying to reduce the amount of spam flags our website has using the Open Site Explorer tool. Currently, we have 2/17 flags based on: Large Site with Few Links - We found very few sites linking to this site, considering its size No Contact Info - None of the pages crawled contain an email address or links to a social profile The "few links" can be ignored, we're working on this. We don't have a visible email address on the website and we don't particularly want one. We prefer customers to fill out our online form or to call us. Moz say "none of the pages crawled contain an email address OR links to a social profile" - we do have social buttons on every page of the website, but these are official Facebook and Twitter buttons that are rendered with Javascript, so don't actually appear in the page source on load. If we replace these with actual links to our pages using Facebook and Twitter icons, will this flag be removed since Moz are saying "or links to a social profile" - making it sound optional. Thanks!
Link Explorer | | LiamMcArthur0 -
Facebook likes in OSE when there is no Facebook Like button on the web page
In OSE how are page specific facebook likes calculated when the page does not have a Facebook like/share button on it? I have seen many pages with multiple Facebook likes counted against it in OSE but cannot see how people could be liking the page.
Link Explorer | | TheHutGroup0 -
Why do I not see any data in Open site Explorer for my site www.mtg-sg.com
The PA and DA shows as one. The message is "It looks like we haven't discovered established link data for this URL yet." But its shows me that there are facebook shares. Therefore, it can't be the reasons stated below.
Link Explorer | | Dineshr840 -
Is there some way to tell the Moz crawler not to crawl URL's with particular dynamic tags such as "?redirect-to:http//" ?
We are encountering an issue where the crawler is finding a ton of pages from our wordpress login url that has this dynamic tag in it to kinds of different blog entries. It's madness. I can't figure out what is causing these URLs to generate to be crawled in the first place! Does this sound familiar to anyone out there, any constructive suggestions? Robots text or maybe meta robots tags that would resolve this crawl issue?
Link Explorer | | RegistrarCorp0