To include in Sitemap or not to include?
-
Hello all,
A bit of a confusing one but please bear with me...
On our website we have a Used Cars section where each morning a feed is loaded onto our site with any changes to the stock. Some cars may have been sold and removed, some new cars may be added, some prices may be changed, every day every morning this very large section of our website is updated.
The question I have is, should I be including these urls in my sitemap? The Used Cars section is a huge portion of our website content and is our most important area, the Used Cars overview is our most frequently visited page.
The reason I ask is because of course Google might crawl and see car X, but tomorrow car X could be gone and be replaced with car Y. Should I be even mentioning these pages to Google if by tomorrow some of those urls could be gone? It's always changing and it's something we don't have control of.
Thanks!
-
Thank you for your suggestions!
Essentially, yes we want to tell Google this page is there, but to not 'trick it' and then that page disappears just as Google finds it resulting in a 404.
We're going to get a sitemap generated each day as so much content changes on the site anyway, but No Indexing and 301'ing is something we'll then give a go.
I'll keep you posted on progress!
Thanks!
-
I would put the content in the sitemap, and once the cars are gone I would remove the items from the sitemap and put up 301 redirects to either the car category page or a similar page. The article below does a pretty good job explaining.
-
Hi Michele,
First of all, I would try to include it in sitemap. Than I'd do the following:
- For all the cars that you have sold, I'd try to implement a noindex, follow (and keep the page alive for the following few days, giving time to Google to crawl these URLs again).
- Every day a sitemap should be generated, then pinged (check Google's "best practices for sitemaps" article: http://googlewebmastercentral.blogspot.ro/2014/10/best-practices-for-xml-sitemaps-rssatom.html and the ping protocol: http://www.sitemaps.org/protocol.html#submit_ping)
That's my personal approach. But might be that others come up with better ideas.
I hope it helped.
Gr., Keszi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemaps: Best Practice
What should and what shouldn't go in the sitemap? In particular, pages like subscribe to our newsletter/ unsubscribe to our newsletter? Is there really any benefit in highlighting those pages to the SEs? Thanks for any advice/ anecdotes 🙂
Intermediate & Advanced SEO | | Fubra0 -
Should we be showing last modified date for each page in our sitemap?
We're not currently displaying the last modified date in our sitemap, e.g.: <url><loc>http://www.soschildrensvillages.org.uk/about-our-charity</loc></url> Are their any advantages to including this data? One benefit that occurred to us is that it will enable Google to determine which pages have fresh content and which are therefore worth crawling, helping Google index beneficial changes quicker. Thanks!
Intermediate & Advanced SEO | | SOS_Children1 -
Mobile Search Results Include Pages Meant Only for Desktops/Laptops
When I put in site:www.qjamba.com on a mobile device it comes back with some of my mobile-friendly pages for that site(same url for mobile and desktop-just different formatting), and that's great. HOWEVER, it also shows a whole bunch of the pages (not identified by Google as mobile-friendly) that are fine for desktop users but are not supposed to exist for the mobile users, because they are too slow. Until a few days ago those pages were being redirected for mobile users to the home page. I since have changed that to 404 not founds. Do we know that Google keeps a mobile index separate from the desktop index? If so, I would think that 404 should work.. How can I test whether the 404 not founds will remove a url so they DON'T appear on a mobile device when I put in site:www.qjamba.com (or a user searches) but DO appear on a desktop for the same command.
Intermediate & Advanced SEO | | friendoffood0 -
Custom sitemap or sitemap generator tool
I have recently launched a website which is using a free sitemap generator (http://web-site-map.com/). It's a large travel agency site (www.yougoadventure.com) with predominantly dynamically generated content - users can add their products as and when and be listed automatically. The guy doing the programming for the site says the sitemap generator is not up to the job and that I should be ranking far better for certain search terms than the site is now. He reckons it doesn't provide lastmod info and the sitemap should be submitted every time a new directory is added or change made. He seems to think that I need to spend £400-£500 for him to custom build a site map. Surely there's a cheaper option out there for a sitemap that can be generated daily or 'ping' google every-time an addition to the site is made or product added? Sorry for the non tech speak - Ive got my web designer telling one thing and the programmer another so im just left trawling through Q&As. Thanks
Intermediate & Advanced SEO | | Curran0 -
Best server-side sitemap generators
I've been looking into sitemap generators recently and have got a good knowledge of what creating a sitemap for a small website of below 500 URLs involves. I have successfully generated a sitemap for a very small site, but I’m trying to work out the best way of crawling a large site with millions of URLs. I’ve decided that the best way to crawl such a large number of URLs is to use a server side sitemap, but this is an area that doesn’t seem to be covered in detail on SEO blogs / forums. Could anyone recommend a good server side sitemap generator? What do you think of the automated offerings from Google and Bing? I’ve found a list of server side sitemap generators from Google, but I can’t see any way to choose between them. I realise that a lot will depend on the type of technologies we use server side, but I'm afraid that I don't know them at this time.
Intermediate & Advanced SEO | | RG_SEO0 -
Sitemaps
I am working with a site that has sitemaps broken down very specifically. By page type: article, page etc and also broken down by Category. Unfortunately, this is not done hierarchically. Category and page type are separate maps, they are not nested. My question here is: Is is detrimental to have two separate sitemaps that point to the same pages? Should we eliminate one of these taxonomies, or maybe just try to make them hierarchical? IE item type -> category -> pagetitle Is there an issue with having a sitemap index that points to a nested sitemap index? (I dont think so, but might as well be sure. Thanks Moz Community! Can't delete my question, but turns out that isn't how they are structured. Food for thought anyway I suppose.
Intermediate & Advanced SEO | | MarloSchneider0 -
Domain and Sitemap Question
Hi - I am hoping you can help me with this issue we are currently trying to solve. We are hosting our mobile site's content on a different domain than what the URL of the site is, though owned by same company. In Google Webmasters tool we have the mobile sitemap under "sitemaps.xyz.com", however the URL of the site is "m.xyz.com". We have submitted 60MM pages in the mobile sitemap, but only 1MM pages have been indexed. Do you think this set up causes confusion with the bots? Does this affect the crawlability of the site? Any thoughts would be greatly appreciated. Thank you!
Intermediate & Advanced SEO | | ladylana
Eva0 -
Sitemap not indexing pages
My website has about 5000 pages submitted in the sitemap but only 900 being indexed. When I checked Google Webmaster Tools about a week ago 4500 pages were being indexed. Any suggestions about what happened or how to fix it? Thanks!
Intermediate & Advanced SEO | | theLotter0