Sitemap Indexed vs. Submitted
-
My sitemap has been submitted to Google for well over 6 months and is updated frequently, a total of 979 URLs have been submitted by only 145 indexed. What can I do to get Google to index them all?
-
SF finding 'useless' links is actually part of its purpose, if you believe they're useless you should be asking why they're there. Your XML sitemap should have nothing but clean URLs; 200 response codes and not canonicalized to another URL. The problem isn't that you have category URLs, it's that those (like the one in my previous example) have a canonical tag that points elsewhere. Anytime this is the case, the URL is considered un-indexable. You can see the proof of this by doing a Google search for "https://www.interstellarstore.com/meteorite-jewelry/meteorite-necklaces", I just checked and this URL isn't in the index.
You mentioned the age in your original comment that your XML sitemap had been submitted for well over 6 months, that's where I got the age from, maybe I misunderstood?
You have no reason to not trust SF, it's one of the most valuable tools in an SEO's toolbox. I've used it for 5+ years to create hundreds of sitemaps and countless other SEO tasks with no problem in providing reliable, accurate data points.
-
Hi Logan,
I tried using Screaming Frog but it kept finding useless links, so I wrote the sitemap myself and I update it manually, I updated it only this morning. What makes you think it is over 6 months since an update?
I was told on Moz in an earlier post that having all of the category links, not just the canonical ones, wasn't a problem, is this not the case?
Every link in the sitemap should work fine, I wrote it by copy and pasting the links directly from my site. I have no trust in Screaming Frog.
-
Hi,
I poked around a bit on your sitemap and noticed a couple things:
- You've got URLs on there that have canonicals to another page. For example:This page https://www.interstellarstore.com/meteorite-jewelry/meteorite-necklaces has a canonical tag that points here https://www.interstellarstore.com/meteorite-necklaces.
- A bunch of the URLs in your sitemap redirect elsewhere or have no response - I got 13% through crawling your XML sitemap with Screaming Frog and there were zero 200 response code URLs, not good.
Both of these things combined are causing a discrepancy in the amount of submitted URLs vs. indexed URLs. If you use Screaming Frog to create your XML sitemap it's quite easy to have only clean URLs in there. You can easily remove all URLs that are not 200 status and by default Screaming Frog will exclude any URL that canonicalizes to another URL.
Also, as a side note, you should be updating your XML sitemap more frequently, a 6 month old sitemap for an ecommerce site is far too old with new products being added and products dropping off.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page with metatag noindex is STILL being indexed?!
Hi Mozers, There are over 200 pages from our site that have a meta tag "noindex" but are STILL being indexed. What else can I do to remove them from the Index?
Intermediate & Advanced SEO | | yaelslater0 -
Getting Google to index our sitemap
Hi, We have a sitemap on AWS that is retrievable via a url that looks like ours http://sitemap.shipindex.org/sitemap.xml. We have notified Google it exists and it found our 700k urls (we are a database of ship citations with unique urls). However, it will not index them. It has been weeks and nothing. The weird part is that it did do some of them before, it said so, about 26k. Then it said 0. Now that I have redone the sitemap, I can't get google to look at it and I have no idea why. This is really important to us, as we want not just general keywords to find our front page, but we also want specific ship names to show links to us in results. Does anyone have any clues as to how to get Google's attention and index our sitemap? Or even just crawl more of our site? It has done 35k pages crawling, but stopped.
Intermediate & Advanced SEO | | shipindex0 -
Trouble Indexing one of our sitemaps
Hi everyone thanks for your help. Any feedback is appreciated. We have three separate sitemaps: blog/sitemap.xml events.xml sitemap.xml Unfortunately we keep trying to get our events sitemap to pickup and it just isn't happening for us. Any input on what could be going on?
Intermediate & Advanced SEO | | TicketCity0 -
Microsites: Subdomain vs own domains
I am working on a travel site about a specific region, which includes information about lots of different topics, such as weddings, surfing etc. I was wondering whether its a good idea to register domains for each topic since it would enable me to build backlinks. I would basically keep the design more or less the same and implement a nofollow navigation bar to each microsite. e.g.
Intermediate & Advanced SEO | | kinimod
weddingsbarcelona.com
surfingbarcelona.com or should I rather go with one domain and subfolders: barcelona.com/weddings
barcelona.com/surfing I guess the second option is how I would usually do it but I just wanted to see what are the pros/cons of both options. Many thanks!0 -
Sitemaps on the fly
Has anyone submitted pages that generate sitemaps on the fly as opposed to only submitting static XML files to Bing? For instance, sitemap.php vs sitemap.xml, video sitemap.php vs videositemap.xml?
Intermediate & Advanced SEO | | alhallinan0 -
Sitemap for SmartPhone site
Hello I have a smartphone site (e.g.m.abc.com). To my understanding we do not need a mobile sitemap as its not a traditional mobile site. Shall I add those mobile site links in my regular www XML sitemap or not bother to add the links as we already have rel = canonical (on m.abc.com ) and rel= alternate in place (on www site) to respective pages. Please suggests a solution. I really look forward to an answer as I haven't found the "official" answer to this question anywhere.
Intermediate & Advanced SEO | | AdobeVAS0 -
Sitemap or Sitemaps for Magento and Wordpress?
I'm trying to figure out what to do with our sitemap situation. We have a magento install for our shopping cart
Intermediate & Advanced SEO | | chrishansen
sdhydroponics.com
and a wordpress install on
sdhydroponics.com/resources In Magento we get the XML sitemap manually by going to Catalog => Google Sitemap => Add Sitemap In wordpress we use Google XML sitemaps plugin. My questions are: Do I need both of these sitemaps? Or can I use one or the other? If I use both, do I make one sitemap1.xml and the other sitemap2.xml and drop them in the root? How do I make sure google knows I have 2 sitemaps? Anything else I should know? Thank You0 -
How do I index these parameter generated pages?
Hey guys, I've got an issue with a site I'm working on. A big chunk of the content (roughly 500 pages) is delivered using parameters on a dynamically generated page. For example: www.domain.com/specs/product?=example - where "example' is the product name Currently there is no way to get to these pages unless you enter the product name into the search box and access it from there. Correct me if I'm wrong, but unless we find some other way to link to these pages they're basically invisible to search engines, right? What I'm struggling with is a method to get them indexed without doing something like creating a directory map type page of all of the links on it, which I guess wouldn't be a terrible idea as long as it was done well. I've not encountered a situation like this before. Does anyone have any recommendations?
Intermediate & Advanced SEO | | CodyWheeler0