Does a sitemap override Google parameter handling?
-
This question might seem silly, but I'll ask anyway.
We have an eCommerce site with a ton of duplicate content, mostly caused by faceted navigation. In researching ways to reduce the clutter, I've decided to use Google parameter handling to stop Googlebot from crawling pages with certain parameters, like: sort order, page #, etc...
Now my question:
If I set all of these parameters so that Googlebot doesn't crawl the grids, how will they ever find the individual product pages? We do upload a sitemap with all of the product pages. Does this solve my issue? Or, should I handle the duplicate content with noindex, follow tag?
Or, is there an even better way?
Thanks
-
Hello John,
This is a very good question, and something people don't often think about when blocking the navigational paths on their site from being crawled.
Depending on how fast your category pages load and how many products are on each of them, you may consider a View All Canonical page: http://googlewebmastercentral.blogspot.com/2011/09/view-all-in-search-results.html
There are many different ways to handle faceted navigation problems, including javascrpt, GWT parameter handling, robots meta, robots.txt, rel canonical... and combinations of these. The right approach should be customized for your specific needs. When possible, I prefer to allow Google to crawl and index down to a certain level of faceting, similar to allowing them into sub-categories (though it depends entirely on your taxonomy) but not tertiary (i.e. sub-sub) categories. For the next couple of levels I might allow them to crawl, but not index. And once it gets down to 4 or 5 levels deep (e.g. /?category=1&size=5&color=blue&price=low&this=that&so-on=so-forth...) I just block them from being both indexed and crawled (i.e. Meta NOINDEX,NOFOLLOW or robots.txt block) to save crawl budget by avoiding spider traps.
With all of that said, if you are giving Google an XML sitemap that contains the indexable URLs to all of your products they should have no problem indexing them, regardless of whether or not they can crawl all the way through your faceted navigation.
-
I would recommend you to use 'Canonical Link'
You can find more here:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Fetch as Google
I have odd scenario I don't know if anyone can help? I've done some serious speed optimisation on a website, amongst other things CDN and caching. However when I do a Search Console Fetch As Google It is still showing 1.7 seconds download time even though the cached content seems to be delivered in less than 200 ms. The site is using SSL which obviously creams off a bit of speed, but I still don't understand the huge discrepancy. Could it be that Google somehow is forcing the server to deliver fresh content despite settings to deliver cache? Thanks in advance
Intermediate & Advanced SEO | | seoman100 -
Google serving wrong page...
Hi, When you Google: "Los Angeles divorce attorney", you will see this site on the 5th page of the SERPS: www.berenjifamilylaw.com/blog/. For some reason, Google is serving the BLOG page as opposed to the homepage. This has been going on now for several weeks. Any tips on how to fix this? Obviously, the Homepage is more relevant and has more links going to it, so not sure why it's happening. Would you just leave it alone? Would you use robots.txt to block Google from crawling the BLOG post page? Thanks.
Intermediate & Advanced SEO | | mrodriguez14400 -
How to handle a blog subdomain on the main sitemap and robots file?
Hi, I have some confusion about how our blog subdomain is handled in our sitemap. We have our main website, example.com, and our blog, blog.example.com. Should we list the blog subdomain URL in our main sitemap? In other words, is listing a subdomain allowed in the root sitemap? What does the final structure look like in terms of the sitemap and robots file? Specifically: **example.com/sitemap.xml ** would I include a link to our blog subdomain (blog.example.com)? example.com/robots.xml would I include a link to BOTH our main sitemap and blog sitemap? blog.example.com/sitemap.xml would I include a link to our main website URL (even though it's not a subdomain)? blog.example.com/robots.xml does a subdomain need its own robots file? I'm a technical SEO and understand the mechanics of much of on-page SEO.... but for some reason I never found an answer to this specific question and I am wondering how the pros do it. I appreciate your help with this.
Intermediate & Advanced SEO | | seo.owl0 -
Is Sitemap Issue Causing Duplicate Content & Unindexed Pages on Google?
On July 10th my site was migrated from Drupal to Google. The site contains approximately 400 pages. 301 permanent redirects were used. The site contains maybe 50 pages of new content. Many of the new pages have not been indexed and many pages show as duplicate content. Is it possible that there is a site map issue that is causing this problem? My developer believes the map is formatted correctly, but I am not convinced. The sitemap address is http://www.nyc-officespace-leader.com/page-sitemap.xml [^] I am completely non technical so if anyone could take a brief look I would appreciate it immensely. Thanks,
Intermediate & Advanced SEO | | Kingalan1
Alan | |0 -
Mobile Sitemaps
We are planning on creating a mobile site using a different URL. Our current sitemap creator won't dynamically detect mobile pages using the rel="alternate" tag but can can create a Project for that domain in Sitemap Creator and use the "mobile" option when you export it. The Sitemap Creator will then insert the mobile:mobilecontent tag for all the URLs in that sitemap. </mobile:mobile> Is this okay or will it cause problems?
Intermediate & Advanced SEO | | theLotter0 -
Google Phone Numbers
What process is performed to get a company's phone number to show as "A" on google maps. Google displays the phone number for the company on the map as "A" first. It would be beneficial to get that position. Is there a sub-category of seo that does this? Thanks in advance!
Intermediate & Advanced SEO | | JML11790 -
Getting Google Authorship to Work
Hi I set up authorship about 10 days ago at the suggestion of this great forum but haven't seen anything happen yet. Could someone take a quick look and check I've done it right? You can see a typical post here: http://www.touristisrael.com/neve-tzedek-tel-aviv/354/ Thanks!
Intermediate & Advanced SEO | | ben10000 -
Check Google ban on domainname
Hello all, If I wanted to know if a domainname has a google ban on it would the following be a good idea to test it. Place an article on the domain page with unique content and then link to the page so its gets indexed and then link to the article from a well indexed page. If it doesn't get indexed there might be a ban on the page, if it does get indexed there is no ban on the page... Or are there other points I should keep in mind while doing this. All help is very welcome. Cheers, Arnout
Intermediate & Advanced SEO | | hellemans0