Moz Crawl shows over 100 times more pages than my site has?
-
The latest crawl stats are attached. My site has just over 300 pages?
Wondering what I have done wrong?
-
total pages is higher you are right Keri but still only 581
-
I believe this image looks at what's indexed that's a subset of your sitemap that you submitted. You may want to look at Google Index -> Index Status in GWT to see what it shows there.
-
latest Moz crawl
-
latest webmaster tools crawl
-
I will definetly be paying attention to those numbers Keri. Webmaster tools is showing the right number of pages (something over 300 with 90% of those indexed)
-
It's not going to be a penalty, but it'll be good to have a bit less of a load on your server (bots no longer crawling thousands of pages) and just have your real pages in the index.
Places to look for interesting changes in site metrics would be your organic traffic in analytics and taking a look at your Google Webmaster Tools account to see your impressions, pages crawled, etc.
-
Thanks Keri, I will update asap.
could you let me know how big an issue would this be? (When you have the time of course;))
-
You're welcome! I may have opened a can of worms, however. That sitemap is generated by an automated tool (based on the footer at the bottom), so somehow it's finding that page 28 as well.
You may also want to ask the developer if you should be indexing the categories in the blog archives. There are resources on Moz about the best way to set that up in Wordpress, but I don't have them at my fingertips at the moment (I have a snuggly baby sleeping on my lap instead that's slowing me down a tad).
To answer your next question, after you figure out where the page 28 is being linked from and cure that, yes, you can do a one-time crawl from Research Tools. It won't overwrite your campaign info, but you can at least see if Moz is seeing thousands of pages or just a few hundred to see if stuff was fixed. Again, happy to provide more detail if/when you need it (and others will likely jump in with help on the thread, too).
I'd love to also see a little update a few weeks down the line of any changes you've noticed on your site metrics after getting this fixed.
-
You rock:)
-
And I found it. The sitemap at http://www.nineclouds.ca/sitemap includes a page /28, which is where the crawlers are finding the non-existent pages.
-
If you look at http://www.nineclouds.ca/blog/page/23, you'll see that there's a double arrow in the pagination at the right that goes to page 24, even though the last page is page 21. Google somehow has found the pages greater than 21 (which I'm not sure how they found), and once they found one of those, they keep seeing the link there with the double arrows to go to another page. Same happened with Rogerbot. I'm not sure where the bad originating link is (what legit page on your site is linking to something over page 21), but that's the loop that's happening and causing a ton of pages to be indexed. Get rid of those, and you'll also get rid of most of your errors.
-
Not shy about that at all thanks Keri.
any help you can provide is greatly appreciated.
-
Hi Bill,
Using my admin powers, I took a peek at your account. I'm still trying to figure out where it's coming from, but you have thousands of empty pages of your blog indexed. I'll dig around a little more and see if I can figure out what's up.
If you're comfortable with sharing your URL here in a public forum, other people can come take a look too. Otherwise, I'm happy to send you a private message with part of what's up and give your developer a place to start looking.
-
Thanks Keri. I am the owner of the site not the programmer so I am looking up the terms you are using as I write this response. If I am using pagination is there a way for the moz not to allow for this? If I understand your question about the calendar correctly I do have one as part of my blog that dates each post? Can I get the bot to not recognize this calendar?
-
My first guess would be parameters or something are being crawled. Do you have pagination? Sorting ascending and descending? A calendar that's getting crawled through the year 2525?
Your next step would be to look into what those duplicate pages are and see if something is amiss that's generating a ton of URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to protect the site from fake traffic
How to protect the site from fake traffic On Google Analytic there are no visits, but on the front of the site there are 6000 jetpack wordpress statistique https://arabtechnologie.com/
Reporting & Analytics | | BELGHOUL0 -
Need advice on setting up primary domain and shopify site analytics to work best together
Hello, I have a client that I have been working on their primary site for the last year or so. In the last month they decided to have one of their internal employees setup a small shopify store. Now they are asking for the analytics tracking codes for it. My question for you is what would be the best way for me to set that up? variables: primary domain and shopify domain, google and bing analytics Have been looking at how cross domain tracking works (https://support.google.com/tagmanager/answer/6106951), and the instructions for setting up ecommerce in analytics for shopify (https://help.shopify.com/manual/reports-and-analytics/google-analytics/google-analytics-setup). But am still not 100% which route would be the best, any input would be greatly appreciated! thank you, Dustin
Reporting & Analytics | | pastedtoast1 -
Why my WMT is showing more clicks than my Google Analytics Organic Search Traffic?
Hello Everybody, Can somebody help me figure out this puzzle: My WMT is showing 6000 clicks, while my Organic Traffic in Google Analytics is showing only 3500 daily ... how is that possible? Best Regards
Reporting & Analytics | | Muhammad_Jabali0 -
Google Analytics shows wrong traffic source
I was reviewing out traffic sources on GA for this past week's traffic history, and noticed a couple of visits & conversions that GA cited as coming from campaigns from emails that were sent out in 2012 and early 2013. it seems odd that we would have traffic & conversions from these older campaigns - on a regular basis. it's not even from one campaign, but from multiple campaigns. is it possible that it is a cookie issue? - i wonder if these visits were really from the email campaigns - or if they were from organic searches or other sources. Thoughts? Thanks!
Reporting & Analytics | | S.S.N0 -
Google Webmaster says "0" pages indexed
Built my first Wordpress site. It launched a few months ago. Google has crawled 76 pages so far. But why are 0 indexed?
Reporting & Analytics | | cschwartzel0 -
Stats show /blog/wp-cron.php at the top. What is it?
Hi, I have worked with websites for years but have no clue when it comes to Wordpress. We have our main website and then a Wordpress blog running in a subfolder that is only about a year old. The blog has only 7 posts so you can see how small it is vs main website with 200 pages. Usually our main index page of the site is at the top of the stats with the most views and this page /blog/wp-cron.php is about 30% lower. Now suddenly over the last month this page has jumped to the top and accessed almost as much as the home page of the site. We took a big hit with the latest Google Update so we are tyring to determine if there is anything technical in our site that has caused an issue. Thanks in advance Force7
Reporting & Analytics | | Force70 -
SEOMoz crawls skewing Avg Visit Duration in Google Analytics
Hello, We are a UK based company. Our Google Analytics account is showing a rise in Avg Visit Duration for 'Direct traffic' since we started using SEOMoz. Are any other users experiencing this issue? We tracked it down to City Seattle, and when doing an Advanced filter by removing Seattle, the results are normal again. What does SEOMoz or other users recommend we do besides continuously using advanced filters? We have enquired about excluding Roger's IP address, but have been told that Roger uses the Amazon cloud, so the IP is not static. See attachment for screenshot of our Google Analytics account of Avg Visit Duration since we began with SEOMoz. Rich Talbot 718g3.gif
Reporting & Analytics | | STL1 -
No Link Data Available for this URL appears often. Are my sites too small to show up?
I am on the trial until Mar 5, 2012. I seldom get the info I want. Is it because my sites are too narrow a niche? I don't seem to be getting the data I'd like from your service. I'm trying to like it, but when I keep getting messages like this, it makes it hard to justify: "No Link Data Available for this URL appears often" Sample sites that I am unable to get data. I especially would like to know how many backlinks exist for each site. I paid someone to help me with them and I'd like to verify their work.: http://costaricadentistreview.com/ http://costaricadentistreviews.com/ http://costaricadentalimplants.org Any suggestions? Thanx Kurt Gross
Reporting & Analytics | | kurtray0