Moz crawling doesn't show all of my pages
-
Hello,
I'm trying to make an SEO on-pages audit of my website.
When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ?
How can i check and correct this. My website is discoolver.com
-
Hi there,
Sam from Moz's Help Team here!
So, I took a look at your site, and unfortunately I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
-
Thanks for the answers.
I believe the issue is about a rel=canonical from the homepage which links to an external website...
Can this tag prevent a good website crawl ?
-
So just looking it appears you don't have a robots.txt file or a sitemap. At least one that I could easily find.
This Moz post might help when creating one.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz was unable to crawl your site on Jun 22, 2020\. We were unable to access your site due to a page timeout on your robots.txt, which prevented us from crawling the rest of your site.
Site: www.kpmg.us Getting robots.txt timeout fail since 02/29/20. We've checked our server logs and see no errors. Went through all the steps of the "Troubleshooter". Updated robots.txt to allow rogerbot full access: User-agent: rogerbot
Link Explorer | | KPMG-Search-Social
Disallow: Any ideas how to get roger to crawl my site????1 -
OSE 'request CSV' for links blank
After doing the link check on a domain, got 40 or so links. Clicked Request CSV The file that is downloaded contains no links. The column headings are there but no content Did this twice and got same blank file. OSE is showing the 40 links
Link Explorer | | cbpayne0 -
Can you do custom date comparisons in Moz?
The only options I see in the tab is weekly and monthly. What if, for example, I wanted to compare stats over a 3 month period? Or compare stats from the same month last year? Is that possible? Or even the stats from the beginning of a campaign? Do you have to manually track them?
Link Explorer | | trypm1 -
Page Authority
MOZ will not recognize links to my webpage. I recently remodeled one of my pages, I understand that it is a new page and I currently don't have external links pointing to that page, but I do have internal links (i e. the category page before it) pointing to this page. Any thoughts?
Link Explorer | | SS990 -
What's with Moz continuously pushing back the Index Update date?
Moz seems to keep pushing back the Index update date, it was the 24th until the 24th, then it got changed to the 30th, now it is may 5th.... What is the reason for this?
Link Explorer | | mgladman1 -
Duplicated content detected with MOZ crawl with canonical applied
Hi there! I have a slight problem.
Link Explorer | | Eurasmus.com
I have a site with Joomla 3.3 that we recently migrated from 2.5. Joomla, for some reason that I don´t really get, creates hundreds of weird urls for the site like
mydomain.com/en -> joomla creates en/home/149-xxx-xxx/xxxxxx-xxxxxx that links to the first one.
The new version 3.3 knows this bug and applies a rel=canonical to the ones created "artificially", so they should not be identified as duplicated. Sample piece of code: en/home/149-all-en/xxxxxxx-xxxxxx" rel="canonical" / MOZ crawler identifies this as duplicated and like this I have thousands of pages duplicated all with titles, content etc... all the ones created by joomla. Still my site has good SEO results and I can not see any penalties but I am a bit concerned they may come in the future.... Can anyone explain me what is happening? Thank you in advance for your time,0 -
Facebook likes in OSE when there is no Facebook Like button on the web page
In OSE how are page specific facebook likes calculated when the page does not have a Facebook like/share button on it? I have seen many pages with multiple Facebook likes counted against it in OSE but cannot see how people could be liking the page.
Link Explorer | | TheHutGroup0 -
301 Redirects: How long for Google to recognize? How long for Moz/OSE to recognize?
I redirected a few pages that had funky URL issues (capitals and underscores and useless words) to the same page renamed with proper URLs. The sitemaps were changed also, and re-submitted. Like this: mydomain.com/Not_Pertinent_Words.html >>> mydomain.com/good-words.html Google seems to have found them and changed the search engine results listing in about 8 days. But it's been about a month now and Moz and OSE still have not transferred all the strength and link data from the old URLs to the new ones. Question 1: How long does it usually take Google to transfer all the link and strength data for a 301? Question 2: How long does it take Moz and OSE to do the same? Is there something I need to do to tell them about the changes?
Link Explorer | | GregB1230