Can I safely block my product listing from search? Does it even make sense?
-
Hi,
I've an ecommerce website with more than 50k urls and only 10% or so are getting crawled regularly by Google.
Product listing pages represent roughly 80% of these 50k pages.Trying to improve this, I was thinking to remove altogether all (most?) of my product listing from search (via Robot.txt) to keep only the product pages themselves and the product categories.
My organic situation since Jan 2019:
Users: 2,300,000 (of which 9% are visiting product listing pages)
Page views: 8,000,000 (of which 5% are product listing pages).Am I about to unleash armageddon (or more like harakiri) on my website by doing so or actually get Google to crawl much more relevant resources (product pages, product categories, blog content and so on)?
Thanks,
G -
Have had a lot of success with that kind of deeper logic in the past, you can usually quite easily create such rules using robots.txt wildcards
-
Thanks for your answer and the link, that's actually very useful.
What I mainly struggle with is to understand what I can prune/not and based on what criterias. And the article you've linked is helping a fair bit on that aspect.I'm thinking to start on multiple filter pages with a rule such as "block product listing pages from being indexed if at least 2 filters have been selected". And then see how it impacts the site and my crawl budget.
-
Hello GhillC,
I think we need to agree on terminology first, but it sounds like you can safely limit some of Google's access. Some people call your "product listing pages" either "refinements", "facets", or "filters". When I read "product listing pages" I typically think of what is also called a "category" page, which is a page listing multiple products. A single product page is often referred to as a product detail page (PDP).
Now that we're on the same page (pun intended), let me know if this article answers your question. It is very dated (2011) but gets the point across, which is that you need to be strategic about which facets/refinements/filters you allow to be crawled and/or indexed: https://moz.com/blog/building-faceted-navigation-that-doesnt-suck .
One more thing: 5% - 9% of traffic going directly from organic search into a page type would be considered significant for most businesses. When I look at pruning out page types, they're typically responsible for less than 0.5% of traffic from organic search.
-
A product page is a page speaking about a selected product.
Example: a page about the smartphone Samguns Supernova 30A product category is a page speaking about the category, or the type of product it falls into.
Example: smartphone or, if it is too broad, the Supernova lineA product listing is a page listing all the product refined with available criterias such as a product line.
Example: a page listing all the Samguns Supernova phones
Or a page listing all the Samguns Supernova phones with more than 128GB of HD.
Etc.Does it make more sense?
-
Can you explain the difference between the 'products listings' and the 'actual products themselves'?
You say you still want products and product categories to rank, but not product listings. But to most readers, a product listing is usually a product category or product page (so the info seems to contradict itself, which actually it may not do - just needs more explaining)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Indexed, but not shown in search result
Hi all We face this problem for www.residentiebosrand.be, which is well programmed, added to Google Search Console and indexed. Web pages are shown in Google for site:www.residentiebosrand.be. Website has been online for 7 weeks, but still no search results. Could you guys look at the update below? Thanks!
Technical SEO | | conversal0 -
PDF in search results?
Hello community! I am not an SEO professional, though I am a practitioner, I would say. I am seeking a solution on behalf of a friend. If you search the term "Peter Blatt" you will discover a "black eye" on the first page, towards the bottom of SERPs. It's a PDF published on the Florida Department of Financial Services website regarding the final order for a settlement he and his company ("Blatt Financial Group") reached with the state as it related to professional conduct allegations. Does anyone have any advice on how to address this? I don't want "game" the search engines, but at the same time, this document looks really scary and much worse than it actually is to people, and I would love for it do drop below page one. Any advice or suggestions from the community? Thanks! Tom
Technical SEO | | 800GoldLaw0 -
What can I do with 8000+ 302 temporary redirects?
Hi I'm working on a clients site that has an odd structure with 8000+ 302 temporary redirects. They are related to actions on the site (they have to be there and work this way for the site to function) but they also have a stupid number of perameters. Would it be ok to block them all in the robots.txt file? Would that make any difference?
Technical SEO | | TomVolpe0 -
Making URLs automatically clickable
Hi all, I have a PHP function which i use to make all links clickable. Problem is, if some one writes a link in a a-tag, the URL inside the href value is made clickable. Not good. Can someone perhaps help me with the issue? Function: function makeClickableLinks($text)
Technical SEO | | rasmusbang
{
$text = preg_replace('/<//', ' $text = preg_replace('(
)', '
', $text);
$text = preg_replace('!((https?://www.|https?://|www.)(([a-z0-9-]+.)+[a-z]{2,6})(/\S+|/)*)!ie', '"[".shortenurl("\1")."]("".(strtolower('$2'))"', $text);
$text = str_replace('( <a href',="" '<a="" $text);<br="">$text = str_replace(')" target', '" target', $text);
$text = str_replace('):" target', '" target', $text);
$text = str_replace(')..." target', '" target', $text);
$text = str_replace(').." target', '" target', $text);
$text = str_replace(')." target', '" target', $text);
return $text;
}</a> <a href',="" '<a="" $text);<br="">Pleeeeease heeelp 🙂 Can't fix it on my own - been at it for ages. -Rasmus</a>0 -
Should we include ANY generic product descriptions on our site?
Until recently all of our product pages (we run an eCommerce site) had product descriptions copied directly off the manufacturer's sites. Obviously this isn't good from an SEO perspective, so we were planning to write our own original copy. My question is: should we keep the manufacturer's copy too? This is often good, useful content from a customer's perspective, but obviously it's not original. Any help appreciated! Alex
Technical SEO | | reddogmusic0 -
When keywords are on the top of the google search engine then what to do ?
My two keywords are on the top of my desired market place that means google.co.uk . So now what should I do to sustain this position???
Technical SEO | | JohnDooley0 -
Summarize your question.Sitemap blocking or not blocking that is the question?
Hi from wet & overcast wetherby UK 😞 Ones question is this... " Is the sitemap plus boxes blocking bots ie they cant pass on this page http://www.langleys.com/Site-Map.aspx " Its just the + boxes that concern me, i remeber reading somewherte javascript nav can be toxic. Is there a way to test javascript nav set ups and see if they block bots or not? Thanks in advance 🙂
Technical SEO | | Nightwing0 -
Can hidden backlinks ever be ok?
Hi all, I'm very new to SEO and still learning a lot. Is it considered a black hat tactic to wrap a link in a DIV tag, with display set to none (hidden div), and what can the repercussions be? From what I've learnt so far, is that this is a very unethical thing to be doing, and that the site hosting these links can end up being removed from Google/Bing/etc indexes completely. Is this true? The site hosting these links is a group/parent site for a brand, and each hidden link points to one of the child sites (similar sites, but different companies in different areas). Thanks in advance!
Technical SEO | | gemcomp1230