Indexed URLs in Webmaster Tools
-
Hi everybody! I've been looking at my Webmaster Tools stats, and it looks like not all the URLs in the sitemap tree have been indexed, according to WMT at least. Is this reliable, and if so, is it worth investigating further?
|
|
|
|
|
|
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
1,309
|
| || |
Index
|
Dec 15, 2011
|
8,695
|
4,127
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
998
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
819
|
| || |
Sitemap
|
Dec 15, 2011
|
2,000
|
719
|
| || |
Sitemap
|
Dec 15, 2011
|
695
|
282
|
Thanks!
-
Hi,
Your example doesn't look out of the ordinary.
Submitting a sitemap doesn't automatically result in indexing of all pages. There are a number of factors that will influence which pages are indexed. These include predetermined allocations of crawl resources by the search engine, relevance/quality of content on the pages, accessibility etc.
As a site gains in popularity, adds and improves content etc then allocations and indexed pages can increase over time. If you see a sudden or significant decrease in the number of pages indexed (without any action by you to remove them) then you would need to start investigating what might be causing it.
If there are specific "money pages" that you are particularly concerned about you can manually check for them in search engines. If they are not returned in the result pages, that would indicate that they are not indexed. In this situation you can use manual indexing features to add individual URLs to both Google & Bing through WMT & Webmaster Central. To do this in Google, use "fetch as Googlebot" and once the URL is successfully fetched, you will be offered the option to add the URL to the index. There are monthly & daily limits on the number of URL's you can add this way.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Do custom tracking codes affect indexing?
Hello, My company uses a tracking system that allows our employees to apply a short code snippet to the end of our URLs for marketing attribution. An example of such a code would be: https://www.schoolofmotion.com/?ref_id=moz-test However, in Google Analytics we are seeing duplicates of our content, where the pages with the individual tracking codes are counted separately from the pages without. From a reporting perspective, this is annoying and definitely worth a fix. However, I'm curious if this problem is affecting our search potential as well. Could this tracking system be splitting traffic in Google's eyes? From an SEO perspective, how should we approach this? Would canonical tags fix this duplication issue in Google Analytics? Is there something else that we should use? Thanks in advance. The Moz community is incredible.
Reporting & Analytics | | CalebWardSoM1 -
Webmaster Tools Suddenly Asking For Verification of Site Registered for 5 Years
Google Webmaster Tools has been successfully installed on my website, (www.nyc-officespace-leader.com) for more than five years. Suddenly, today I have received a request to Verify this Site". This makes no sense. The only possibility I can think of is that this is somehow tied to the following events in the last month: 1. Launch of new version of website on June 4th
Reporting & Analytics | | Kingalan1
2. Installation of Google of Tag Manager
3. Sudden Increase in number of pages indexed by Google. Unexplained indexing of an additional 175 pages. About 625 pages should be indexed, while 800 are now indexed. In the last month ranking and traffic have fallen sharply. Could it be tat these issues are all linked? But the strangest issue is the request to verify the site. Does anyone have any ideas? Thanks,
Alan0 -
Webmaster Tools Records
Hey, I'm trying to find a way of getting data from Googles Webmaster Tools and recording it in either a database or spread sheet. Is there any software out there that I can use to do this or would i have to look at developing it my self? Thanks, Luke.
Reporting & Analytics | | NoisyLittleMonkey0 -
Google Webmaster Tools - When will the links go away!?
About 9 months back we thought having an extremely reputable company build our client some local citations would be a good idea. You definitely know this citation company, but I'll leave names out. Regardless, it's our mistake to cut corners. Google Webmaster Tools quickly picked up these new citations and added them to the links section. One of these citation spawned a complete mess of about 60K+ links on their network of sites through ridiculous subdomains of every state in the country and so many other domain variations. We immediately went into remove mode and had the site's webmaster take down the bad links from their site. This process took about a month for outreach. The bad links (60K+) have not been on the spam site for well over 6 months but GWT still shows them in the "links to your site" section. Majestic, Bing, and OSE only displayed the bad links for a brief time. Why is webmaster tools still showing these links after 6+ months? We typically see GWT update about every 2 weeks, a month tops. Any ideas? Could a changed robots.txt on the bad site prevent Google from updating the links displayed in GWT? We have submitted to disavow, but Google replied with "no manual penalty". We even blasted the bad site with Fiverr links, in hopes that Google would re-crawl them. No luck with anything we do. We have patiently waited for way too long. The rankings for this site got crushed on Google after these citations. How do we fix this? Should we worry about this? Any advice would really help. Thanks so much in advance.
Reporting & Analytics | | zadro0 -
Google Webmaster Tools - spike in 'not selected' under Index Status
Hi fellow mozzers Has anyone seen a huge shift in the number of pages 'Not Selected' under Index Status in Google WMT, and been able to identify what the problem has been? My new client recently moved their site to wordpress - and in doing so the number of pages 'not selected' rose from ~200 to ~1100, It was high before but is ridiculous now. I am thinking there must be a new duplicate content issue which should be cleaned up in my quest to improve their SEO. Could it be the good old WP tag/category issue? In which case I won't worry as Joost is doing its job of keeping stuff out of the index. There are loads of image pages which could well appear as dupe as have no content on them (i do need to fix this), but Google is already indexing these so doesn't explain the ones 'not selected'. I've tried checking dupe title tags but there are very few of them so that doesn't help Any other ideas of how to identify what these problem pages maybe? Thanks very much! Wendy
Reporting & Analytics | | Chammy0 -
What tools are people using to analyse clicked links
Hi, What tools do you use/recommend to analyse what/where links are being clicked on a page. I have seen a few mentions about CrazyEgg but are there any free (but good) tools around worth using?
Reporting & Analytics | | NeilD0 -
Tool to check pervious rankings for a keyword ?
I am looking a tool/site to check ranking history of specifc keyword . Is this possible ? Thanks
Reporting & Analytics | | bfletc40 -
Subdomains and SEO - Analytics & Webmaster Tools Setup Help
Any advice on the following greatly appreciated: How to get multiple subdomain data into 1 Google Analytics profile? Can we get multiple subdomain data into Google Webmaster Tools (and if so how?) or do we need to set GWT up per subdomain?
Reporting & Analytics | | AndyMacLean0