Webmaster Tools Indexed pages vs. Sitemap?
-
Looking at Google Webmaster Tools and I'm noticing a few things, most sites I look at the number of indexed pages in the sitemaps report is usually less than 100% (i.e. something like 122 indexed out of 134 submitted or something) and the number of indexed pages in the indexed status report is usually higher. So for example, one site says over 1000 pages indexed in the indexed status report but the sitemap says something like 122 indexed.
My question: Is the sitemap report always a subset of the URLs submitted in the sitemap? Will the number of pages indexed there always be lower than or equal to the URLs referenced in the sitemap?
Also, if there is a big disparity between the sitemap submitted URLs and the indexed URLs (like 10x) is that concerning to anyone else?
-
Unfortunately not, the closest you'll get is selecting a long period of time in Analytics and then exporting all the pages that received organic search traffic. If you could then cross check them with your list of URLs on your site it could provide you with a small list. But I would still check them in Google to make sure they aren't indexed. As I said it's not the best way.
-
Is there a reliable way to determine which pages have not been indexed?
-
Great answer by Tom already, but I want to add that probably images and other types of content whom are mostly not by default included in sitemaps could also be among the indexed 'pages'.
-
There's no golden rule that your sitemap > indexed pages or vice versa.
If you have more URLs in your sitemap than you have indexed pages, you want to look at the pages not indexed to see why that is the case. It could be that those pages have duplicate and/or thin content, and so Google is ignoring them. A canonical tag might be instructing Google to ignore them. Or the pages might be off the site navigation and are more than 4 links/jumps away from the homepage or another page on the site, make them hard to find.
Conversely, if you had lots more pages indexed than in your sitemap, it could be a navigation or URL duplication problem. Check to see if any of the pages are duplicate versions caused by things like dynamic URLs generated through search on the site or the site navigation, for example. If those pages are the only physical pages that you have created and you know every single one has been submitted in a sitemap - and so any other indexed URLs would be unaccounted for, that may well be cause for concern, so check nothing is being indexed multiple times.
Just a couple of scenarios, but I hope it helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help Blocking Crawlers. Huge Spike in "Direct Visits" with 96% Bounce Rate & Low Pages/Visit.
Hello, I'm hoping one of you search geniuses can help me. We have a successful client who started seeing a HUGE spike in direct visits as reported by Google Analytics. This traffic now represents approximately 70% of all website traffic. These "direct visits" have a bounce rate of 96%+ and only 1-2 pages/visit. This is skewing our analytics in a big way and rendering them pretty much useless. I suspect this is some sort of crawler activity but we have no access to the server log files to verify this or identify the culprit. The client's site is on a GoDaddy Managed WordPress hosting account. The way I see it, there are a couple of possibilities.
Reporting & Analytics | | EricFish
1.) Our client's competitors are scraping the site on a regular basis to stay on top of site modifications, keyword emphasis, etc. It seems like whenever we make meaningful changes to the site, one of their competitors does a knock-off a few days later. Hmmm. 2.) Our client's competitors have this crawler hitting the site thousands of times a day to raise bounce rates and decrease the average time on site, which could like have an negative impact on SEO. Correct me if I'm wrong but I don't believe Google is going to reward sites with 90% bounce rates, 1-2 pages/visit and an 18 second average time on site. The bottom line is that we need to identify these bogus "direct visits" and find a way to block them. I've seen several WordPress plugins that claim to help with this but I certainly don't want to block valid crawlers, especially Google, from accessing the site. If someone out there could please weigh in on this and help us resolve the issue, I'd really appreciate it. Heck, I'll even name my third-born after you. Thanks for your help. Eric0 -
Page Views for a List of URL's (Excel & Analytics)
I've been playing around with a few excel plugins trying to get the visit count for over 350 different URL's on the same website. I cant seem to find a function that can do this. Can anyone advise a work around or a way i can achieve this? Greg
Reporting & Analytics | | AndreVanKets0 -
Get a list of robots.txt blocked URL and tell Google to crawl and index it.
Some of my key pages got blocked by robots.txt file and I have made required changes in robots.txt file but how can I get the blocked URL's list. My webmaster page Health>blocked URL's shows only number not the blocked URL's.My first question is from where can I fetch these blocked URL's and how can I get them back in searches, One other interesting point I see is that blocked pages are still showing up in searches.Title is appearing fine but Description shows blocked by robots.txt file. I need urgent recommendation as I do not want to see drop in my traffic any more.
Reporting & Analytics | | csfarnsworth0 -
How to Setup New Domain in Webmaster Tools? Can Anybody HELP?
I, I have a new domain, and a new account is now created in webmatser tools. Is there anythings I should know about using or setting up webmaster tools? For example, the geographic target, I'm in Canada our web site, a local Pilates studio, is a commercial web site to provides informations about our services. Should I select Canada? One thing about this, I have another domain, http://www.furnacefilterscanada.com/ and in the setiing, I have select geographic target to: Canada. When using google.ca result are better then when using google.com! I think must Canadian user are using google.com Does this ''Geographic Target'' setting has anythings to do with search results in Goolge.ca versus Google.com? Thank you for your help, BigBlaze
Reporting & Analytics | | BigBlaze2050 -
Posting on blog comments with anchor text on high ranked pages effective?
So i've identified some blogs which have a fairly high ranking and lots of traffic. They also allow anchor text in the name field. Does it make sense for me to comment on these blogs, or does google treat these with less authority that true page links? Any advice is greatly appreciated! TIA
Reporting & Analytics | | symbolphoto0 -
Number of Visitor Entries to page via search engine
Hi, I wanted to figure out the most optimal way to track the number of visitors that comes to a specific page on my blog via search engine only. I know Google Analytics has a "top landing page" filter, but that includes all visitors that comes in directly or other channels. Is there a way to figure out how many visitors a certain page received that was generated through only search engine? Does SEOmoz have this capability?
Reporting & Analytics | | kevinyu10290 -
Keep multiple Google Analytic / Webmaster accounts seperate?
Hi, I have a paranoid suspicion that putting all my websites on one Webmaster / Analytic account will mean any links between my sites won't carry any weight, and Google might consider similar content much more readily if 2 of my sites are in the same market. So I have about 4 different accounts with my main sites split across them. Am I being over paranoid? Do any of you have any problem with all your sites being on the same account?
Reporting & Analytics | | SpecialCase0 -
Google: show all images indexed on a domain
Is there a way to display all images that google has indexed on a domain / subdomain? I'm basically looking for something like a site:-command for google image search.
Reporting & Analytics | | jmueller0