Sitemap include all site links or just ones we want indexed?
-
Got a quick sitemap question. We have a clients site built in opencart and are getting ready to submit the sitmap. The default sitemap setting generates urls right off of the root. For example site.com/product. These urls are also accessible through the site itself. We prefer to give the site some depth and have structured the products so the urls are site.com/category/product. All of the product pages have canonicals including the category so we should not have to worry about duplicate content on the /product page vs the /category/product page. My question is both types of product pages are included in the sitemap at the moment. Since we don't want google to index the /product urls should we leave them off of the sitemap even though they are readily accessible from the frontend(though not linked)? Or just leave them and let the canonical tag be used in directing google as to which urls to index. Thanks in advance.
-
Hi again JS,
I think it's great that you continue to evaluate your platform from all perspectives and evaluate its strengths/weaknesses. Many times, a platform can do a lot of the basics well, but fall short on the details that differentiate us from our competition. For example, opencart may do the basic SEO requirements well, but not include ecommerce microdata (schema.org) which have a high impact on our search listings.
You can do a lot of harm/good with the robots.txt file - like deindex entire website (probably not a good thing) or block certain directories (your /product issue). I would gain some deeper knowledge about what you can do with the robots.txt file and how you need it to perform for your business.
-
Hey Raymond,
Thanks for the response, feel like I'm over thinking this a bit, as usually we just leave our opencart setups as is, other then a few minor tweaks. Lately I've really been scrutinizing opencart's SEO setup and how to improve it, since it seems there are a lot of gaps in he way it handles this.
I thought the robots.txt would have been a good way to block the pages, but the issue is I would need to block every single product page as opencart automatically creates a page for every product that is site.com/product and since we are adding lots of products there should be a better way to handle this. After I posted I came across this tidbit from a 6 year old google webmaster central blog post. Basically it states that 'While we can't guarantee that our algorithms will display that particular URL in search results, it's still helpful for you to indicate your preference by including that URL in your Sitemap. '. I think going this route along with the canonical should do the trick.
-
Hi JStrong,
Great question to be asking and an important topic to be doing your due diligence on, especially when dealing with an eCommerce related website.
Google uses a sitemap as a guideline for crawling your site. So, just because you put a URL in your sitemap, doesn't mean that they URL will actually be indexed. You can see those stats in your Google Webmaster Tools account, under the Sitemap area. It will display how many URLs are in the sitemap and how many out of those URLs are indexed.
If you do not want certain pages to be indexed by Google, then you would need to adjust your robots.txt file to give Google those instructions.
As long as you have the correct Canonical configurations, you should avoid any duplicate content issues from the URLs you've described above.
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hello, I've heard that the outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about or they aren't really worth including. Is this true?
Hello, I've heard that relevancy of the content between the source page and the target page of outbound links in my content matters greatly. The outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about, or they aren't really worth including. Example: Don't just link to the homepage of an organization mentioned in the article, link to a page on their site that is related to the topic you are writing about. Is this true? Would including less relevant links negatively impact SEO in any way?
On-Page Optimization | | DJBKBU0 -
Include Site Name in Page Titles or not
i would like to ask if it is a good practice or not to Include Site Name in Page Titles. My page is not selling products it is about plagiarism checker tool. i will give one example in one page we are writing about the plagiarism types so the page title is plagiarism types and then is the site name. what is the better practice? Keep it or not? thanks in advance
On-Page Optimization | | anavasis3 -
On-Site Optimization Issue!
Hello, I have some confusion about how to structure my site to better in on-site optimization. I am using WordPress. Therefore, there are many things that I need to consider as following: Static Page for homepage OR Latest posts? Archive, Category, Author, Attachment and Tag pages - To put meta robots (no index, follow) or not to prevent duplication? If I use Static Page for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? If I use Latest Posts for homepage, do I need to add meta robots (no index, follow) to POINT 2 above or not? To have breadcrumb or not? To have recent posts, comment, tag clouds or popular posts/comments widget or not? To have social sharing icons and related posts in single post or not? If you don't mind adding more tips that I don't know it would be very great! Thanks!
On-Page Optimization | | dinabrokoth0 -
Two sites, one with a ccTLD domain, the other with TLD domain, same content
Hi there! I have a site which can be accessed with two different domains: one ccTLD for Spain: www.piensapiensa.es one TLD www.piensapiensa.com Should I take care of something regarding SEO? I have also a redirection from www.piensapiensa.com to piensapiensa.com. I have set up them in webmasters tools individually, with the same sitemap obviously. Thanks in advanced.
On-Page Optimization | | juanmiguelcr0 -
Too Many On-Page Links
Hi, I did a SEOmoz campaign and got results today, One of the results is Too "Many On-Page Links" when i am drilling down, i see that that's include inside links. for example, i sale food, i have my main department window - inside i have 30 products - each product is linked to a detailed page about the product. so automatically i have 30 links - not including all the others in this page, and i easily get over 100 and even sometimes 200 is this a big issue? does it damages my SEO? If yes, is there a way to write the HTML in a way that internal links like that wont be counted? Thank you SEOWiseUs
On-Page Optimization | | iivgi0 -
Moving content from one site to another
I have a couple established, content rich sites with some content that I would like to move over to a new site. My question is what steps I need to take to ensure that neither my older sites nor newer sites are penalized for duplicate content. The purpose for moving the content is to add some depth to the new site for users, as well as possibly optimize it all for SEO. There is a fair amount of content involved, about 50 posts and pages per site, so I'd like to know if the potential problem with duplicate content might be serious enough that I should think twice. What do you recommend?
On-Page Optimization | | LeeAbrahamson0 -
Why does Google no longer like our site?
Hey guys, I'm trying to figure out why the traffic and rankings have been plummeting on www.readprint.com. It's a collection of both public domain books and books on Amazon's store. If anyone can offer any pointers as to if it's duplicate content or ??? It used to get 300K visits/mo but has slowly been dropping over the last year. I appreciate anyone's expertise!
On-Page Optimization | | CoBraJones0 -
Does anyone know of a Domain auction site that includes MOZ rankings?
Trying to locate a site that has this as part of the search criteria
On-Page Optimization | | hooopdream0