Webmaster Tools Indexed pages vs. Sitemap?
-
Looking at Google Webmaster Tools and I'm noticing a few things, most sites I look at the number of indexed pages in the sitemaps report is usually less than 100% (i.e. something like 122 indexed out of 134 submitted or something) and the number of indexed pages in the indexed status report is usually higher. So for example, one site says over 1000 pages indexed in the indexed status report but the sitemap says something like 122 indexed.
My question: Is the sitemap report always a subset of the URLs submitted in the sitemap? Will the number of pages indexed there always be lower than or equal to the URLs referenced in the sitemap?
Also, if there is a big disparity between the sitemap submitted URLs and the indexed URLs (like 10x) is that concerning to anyone else?
-
Unfortunately not, the closest you'll get is selecting a long period of time in Analytics and then exporting all the pages that received organic search traffic. If you could then cross check them with your list of URLs on your site it could provide you with a small list. But I would still check them in Google to make sure they aren't indexed. As I said it's not the best way.
-
Is there a reliable way to determine which pages have not been indexed?
-
Great answer by Tom already, but I want to add that probably images and other types of content whom are mostly not by default included in sitemaps could also be among the indexed 'pages'.
-
There's no golden rule that your sitemap > indexed pages or vice versa.
If you have more URLs in your sitemap than you have indexed pages, you want to look at the pages not indexed to see why that is the case. It could be that those pages have duplicate and/or thin content, and so Google is ignoring them. A canonical tag might be instructing Google to ignore them. Or the pages might be off the site navigation and are more than 4 links/jumps away from the homepage or another page on the site, make them hard to find.
Conversely, if you had lots more pages indexed than in your sitemap, it could be a navigation or URL duplication problem. Check to see if any of the pages are duplicate versions caused by things like dynamic URLs generated through search on the site or the site navigation, for example. If those pages are the only physical pages that you have created and you know every single one has been submitted in a sitemap - and so any other indexed URLs would be unaccounted for, that may well be cause for concern, so check nothing is being indexed multiple times.
Just a couple of scenarios, but I hope it helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
PPC ads: landing page vs website page ... other metrics to consider?
I run 3 PPC ads in Chinese language on Baidu, of which one landing page consistently ranks around 5th amongst 80+ other English website pages. So, when our site was recently developed for Chinese language, I redirected the ads to the relevant Chinese website pages for a month to see which would attract more visitors (Chinese landing pages vs Chinese website). I haven't fully analysed the results yet, but what other metrics should I consider than just the volume of visitors? Landing page
Reporting & Analytics | | SteveMauldin
https://www.mogas.com/en-us/ppc-ads/mogas-球阀
https://www.mogas.com/en-us/ball-valves (in English) Website page
https://www.mogas.com/zh-cn/产品
https://www.mogas.com/en-us/products (in English)0 -
Webmaster Tools vs. Google Trends data doesn't add up
I am investigating a two-month 25% drop in organic traffic from Google to a client's site. When I turned to the Webmaster Tools data for the site, there is a clear, gradual drop over the course of a couple months both in impressions and clicks. In general, the drop occurred across many pages and for a large number of queries; there wasn't a core group of keywords or pages that saw the drop...it was more sitewide. Yet, the average rankings reported by WMT were, for the top 100 or so landing pages, not significantly different. The site hosts information about medical conditions, and I wouldn't expect any time-related variations in search volume, and this was confirmed by looking at Google Trends data for a number of the top keywords. I started to look at the data by query for all the top keywords (all ranked in the top 10), and saw the following general trend: impressions were down, rankings stayed in the top 10, and Google Trends showed either flat or rising volumes. So I am trying to make sense of that. If the search volume trend did not decline and rankings held inside the top 10, then how could the number of impressions drop significantly? Am I trusting the WMT data too much? But the reality is that the volume of traffic measured by Google Analytics from Google organic did indeed drop the way Webmaster Tools show it.
Reporting & Analytics | | WillW0 -
How to safely exclude search result pages from Google's index?
Hello everyone,
Reporting & Analytics | | llamb
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blah I wanted to block everything of that sort, but how do I do it without blocking /index.php ? Thanks in advance and have a great day everyone!0 -
Switch to www from non www preference negatively hit # pages indexed
I have a client whose site did not use the www preference but rather the non www form of the url. We were having trouble seeing some high quality inlinks and I wondered if the redirect to the non www site from the links was making it hard for us to track. After some reading, it seemed we should be using the www version for better SEO anyway so I made a change on Monday but had a major hit to the number of pages being indexed by Thursday. Freaking me out mildly. What are people's thoughts? I think I should roll back the www change asap - or am I jumping the gun?
Reporting & Analytics | | BrigitteMN0 -
Landing page URL appearing as keyword
Hi Mozers, I've recently experienced the URLs of my key landing pages coming up as keywords. This has been on the rise since early July (when it was relatively insignificant) to the current position (see image below) where they make up the majority of my top keywords. Drilling down into a bit more detail, this seems to be almost exclusively Desktop traffic but in terms of Technology there are no clear standouts (seems to be mostly Windows OS and Chrome). Has anyone else been experiencing this?
Reporting & Analytics | | mopland0 -
Since the 6th June there has been a serious drop in search impressions and clickthroughs in my GA + webmaster accounts.
Since the 6th June there has been a serious drop in organic search impressions and click throughs in my GA + webmaster accounts for my website. This leads me to believe my site has been punished for something? Where would be the best place to start diagnosing the issue causing the drop in search traffic? 2T9hz.jpg
Reporting & Analytics | | Scurrypoint0 -
Please Give This Page a Good Ass Kicking
This page on my site has a high bounce rate (around 90%) despite being right on point for the search queries that lead visitors to it (i.e. keyword data shows visitors are searching for this information exactly). Also, Google keeps giving the page good placement and it receives a good bit of traffic. Anyone have thoughts as to why the bounce rate is so high? Feel free to offer candid criticism.
Reporting & Analytics | | JSOC0 -
Why did I loose all my product page rankings (e-commerce site)
This friday I noticed that I'd lost pretty much all my product pages in the SERP and also their rankings for the product names. These are products I both have introduced to the market (sweden) and also some that I've been the only one selling. I've analyzed a couple of different ranking-faults. Examples: **"super mario väggdekaler" should rank **http://www.roligaprylar.se/Super-Mario-Vaeggdekaler.html as #1 and has done for several years. Instead this search in my internal search engine ranks #10-#15 with no relevance. www.roligaprylar.se/?q=mario%20v%E4g "jedi morgonrock" should rank www.roligaprylar.se/Jedi-Morgonrock.html as #1 or #2 but instead this url ranks as #12 www.roligaprylar.se/product_detail.php?pid=Jedi-Morgonrock "Charlie sheen bobblehead" (in the swedish serp this should be the most simple term to rank on. previously #1) my internal search engine ranks for #8 with this url <cite>www.roligaprylar.se/?q=Charlie%20Sheen%20Bobblehead</cite>J So I've drawn these conclusions and actions Products that don't rank well longer but still ranks with their alternative non-rewritten url has gotten deep links from affilliates (i track affilliate ids and stuff via this link) and have replaced the original url which is rewritten. Action: Canonical urls for these non-rewritten products to the rewritten version. For example on this product page www.roligaprylar.se/product_detail.php?pid=Jedi-Morgonrock I've placed a canonical for this url www.roligaprylar.se/Jedi-morgonrock.html With the products not ranking at all or when searches in my search engine shows up I suspect some kind of dup content punishment where Google thinks the search result is more important than the product page. Action: All search-pages are now noindex,follow I also increased product name density in terms of keywords on the product page. But I'm still owned and losing tons of money during the holidays (buying adwords at obscene amounts instead hehe). So just wanted to hear with you guys. Are my conclusions and actions correct? What have I missed, what more could I do to reverse this? Thanks Dan
Reporting & Analytics | | nuttinalle0