Moz Crawl shows over 100 times more pages than my site has?
-
The latest crawl stats are attached. My site has just over 300 pages?
Wondering what I have done wrong?
-
total pages is higher you are right Keri but still only 581
-
I believe this image looks at what's indexed that's a subset of your sitemap that you submitted. You may want to look at Google Index -> Index Status in GWT to see what it shows there.
-
latest Moz crawl
-
latest webmaster tools crawl
-
I will definetly be paying attention to those numbers Keri. Webmaster tools is showing the right number of pages (something over 300 with 90% of those indexed)
-
It's not going to be a penalty, but it'll be good to have a bit less of a load on your server (bots no longer crawling thousands of pages) and just have your real pages in the index.
Places to look for interesting changes in site metrics would be your organic traffic in analytics and taking a look at your Google Webmaster Tools account to see your impressions, pages crawled, etc.
-
Thanks Keri, I will update asap.
could you let me know how big an issue would this be? (When you have the time of course;))
-
You're welcome! I may have opened a can of worms, however. That sitemap is generated by an automated tool (based on the footer at the bottom), so somehow it's finding that page 28 as well.
You may also want to ask the developer if you should be indexing the categories in the blog archives. There are resources on Moz about the best way to set that up in Wordpress, but I don't have them at my fingertips at the moment (I have a snuggly baby sleeping on my lap instead that's slowing me down a tad).
To answer your next question, after you figure out where the page 28 is being linked from and cure that, yes, you can do a one-time crawl from Research Tools. It won't overwrite your campaign info, but you can at least see if Moz is seeing thousands of pages or just a few hundred to see if stuff was fixed. Again, happy to provide more detail if/when you need it (and others will likely jump in with help on the thread, too).
I'd love to also see a little update a few weeks down the line of any changes you've noticed on your site metrics after getting this fixed.
-
You rock:)
-
And I found it. The sitemap at http://www.nineclouds.ca/sitemap includes a page /28, which is where the crawlers are finding the non-existent pages.
-
If you look at http://www.nineclouds.ca/blog/page/23, you'll see that there's a double arrow in the pagination at the right that goes to page 24, even though the last page is page 21. Google somehow has found the pages greater than 21 (which I'm not sure how they found), and once they found one of those, they keep seeing the link there with the double arrows to go to another page. Same happened with Rogerbot. I'm not sure where the bad originating link is (what legit page on your site is linking to something over page 21), but that's the loop that's happening and causing a ton of pages to be indexed. Get rid of those, and you'll also get rid of most of your errors.
-
Not shy about that at all thanks Keri.
any help you can provide is greatly appreciated.
-
Hi Bill,
Using my admin powers, I took a peek at your account. I'm still trying to figure out where it's coming from, but you have thousands of empty pages of your blog indexed. I'll dig around a little more and see if I can figure out what's up.
If you're comfortable with sharing your URL here in a public forum, other people can come take a look too. Otherwise, I'm happy to send you a private message with part of what's up and give your developer a place to start looking.
-
Thanks Keri. I am the owner of the site not the programmer so I am looking up the terms you are using as I write this response. If I am using pagination is there a way for the moz not to allow for this? If I understand your question about the calendar correctly I do have one as part of my blog that dates each post? Can I get the bot to not recognize this calendar?
-
My first guess would be parameters or something are being crawled. Do you have pagination? Sorting ascending and descending? A calendar that's getting crawled through the year 2525?
Your next step would be to look into what those duplicate pages are and see if something is amiss that's generating a ton of URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Fire a tag when element is loaded on page (Google Tag Manager)
I'm using an Element Visibility trigger to track a value that appears on a page. However, I want to track this value even when the user doesn't scroll to the area of the page where the element is (i.e. when the page is loaded, and the value is displayed below the fold, but the user doesn't scroll down there). Is there a way of doing this
Reporting & Analytics | | RWesley0 -
Organic reports showing a URL that isn't in Search Ask Question
In the image I've attached you can see that I have pulled a source/medium > google organic report. I've also made "landing page" my secondary dimension. The first landing page that is showing up is /v3/?slug=fnl, that is this page (https://orders.freshnlean.com/v3/?slug=fnl). You can see that the page has 230 sessions from Sep 3 - 9 and 17 transactions during that same time frame. The only thing is, that landing page is nowhere to be found in the SERPs. So how is it showing up in this report as having received google organic visitors that converted if it's not even in search? 05OclDp
Reporting & Analytics | | tdastru0 -
Google Analytic - Avg load Time - Page Timing Sometimes Graph goes suddenly up why?
Hello All, I have attached the avg load time screenshot for my ecommerce site for 1 month. Screenshot is for Mobile site. If you check the graph then only one day graph gone very high. Now again I have attached that particular day graph too. My query is why graph gone high 1) Is it my site not performed well on particular one day on particular device for single visitor? because for desktop and tablet graph showing normal 2) so here if site restart or down then it can be a problem for desktop, tablet too right? Can anyone give me any clue? Z4hX1 IanuP
Reporting & Analytics | | pragnesh96390 -
What is the most effective way of selecting a top keyword per page on a site?
We are creating fresh content for outdated sites and I need to identify the most significant keyword per page for the content developers, What is the best way to do this?
Reporting & Analytics | | Sable_Group0 -
Www and non www versions of the site: 301 redirects but I still get impressions on the wrong version
hallo, I moved from www.bastabollette.it to bastabollette.it, setting a 301 redirect. If I check google search console, I still get impressions and looks like all old www pages are stille indexed. (see attached) why? how can I fix this? thank you
Reporting & Analytics | | micvitale0 -
What is click2.scour, and why is it showing up as Referral Traffic in Google Analytics?
I've noticed that a couple of my clients in the insurance industry have been receiving a pretty large boost in Referral Traffic from a source called click2.scour.com and click2.efacts.com. What surprises me most is that the traffic has a low Bounce Rate, a high Avg. Visit Duration, and is made up of 100% New Visits. What is this? Why would they be getting so much traffic from these two sources all of a sudden? Thanks in advance for your help!
Reporting & Analytics | | copyjack0 -
Figuring Out the Source of "direct traffic" by looking at landing page parameters
I have a client who runs an e-commerce website, and I noticed that 40% of his traffic and 25% of his sales are all attributable to Direct Traffic. At first, I tried to solve this problem by tagging all of the previously untagged links in his e-newsletter, which I expect to be very helpful. However, then I looked at the landing pages for his direct traffic, and I see that it is almost entirely filled with thousands of unique URLs that begin with a question mark followed by the name of his e-newsletter or shopping cart vendor. It would be the equivalent of having a url like the following: "www.willmarlow.com/?constantcontact=keya;sldkfjsdlfkjdf;sldkjf" If we have this amount of information in the link, shouldn't there be a way to add additional parameters to the URL to move this traffic out of the Direct column? Has anyone encountered this before? Thanks.
Reporting & Analytics | | williammarlow0 -
Tool to check GA code present on every page?
Is there a tool to check if the Google Analytics code is present on every page of a website? Thanks for your help!
Reporting & Analytics | | gerardoH0