Noindex large productpages on webshop to counter Panda
-
A Dutch webshop with 10.000 productpages is experiencing lower rankings and indexation. Problems started last october, a little while after the panda and penguin update.
One of the problems diagnosed is the lack of unique content. Many of the productpages lack a description and some are variants of eachother. (color, size, etc). So a solution could be to write unique descriptions and use rel canonical to concentrate color/size variations to one productpage.
There is however no capacity to do this on short notice. So now I'm wondering if the following is effective.
Exclude all productpages via noindex, robots.txt. IN the same way as you can do with search pages. The only pages left for indexation are homepage and 200-300 categorypages. We then write unique content and work on the ranking of the categorypages. When this works the product pages are rewritten and slowly reincluded, category by category.
My worry is the loss of ranking for productpages. ALthoug the ranking is minimal currently. My second worry is the high amount of links on category pages that lead to produtpages that will be excluded rom google. Thirdly, I am wondering if this works at all. using noindex on 10.000 productpages consumes crawl budget and dillutes the internal link structure.
What do you think?
-
I see. There's a pretty thorough discussion on a very similar situation here: http://moz.com/community/q/can-i-use-nofollow-tag-on-product-page-duplicated-content. Everett endorsed Monica's answer with, "... you might consider putting a Robots Noindex,Follow meta tag on the product pages. You'll need to rely on category pages for rankings in that case, which makes sense for a site like this." Monica's long term solution was to also work on getting specific user-generated content on as many product pages as possible. Cheers!
-
@Ryan, thx for your answer. The pagerank flow is indeed one of the things I worry about when deindexing large parts of the site. Especcialy since the category pages will be full of internal links to productpages that are excluded from indexation by robots.txt or robots meta.
The problem I am trying to solve however has nothing to do with pagerank sculpting. I suspect an algorithmic drop due to thin, duplicate and syndicated content. The drop is sitewide. Assuming that the drop is due to panda I suspect the percentage of low quality pages should be optimized. Would outlinking and better DA really be sufficient to counter a suspected Panda problem? Or is it required to make the 10.000 product pages of better quality, I would think the latter. Since there is no budget to do so I wonder if it is possible to drop these low quality pages from the index (but keep them in the website). Would this strenghten the remaining pages to bounce back up, assuming these remaining pages are of good quality offcourse.
Since SEO is not the only factor to be taken into account I'd rather not delete these pages from the website.
-
Matt Cutts speaks to part of what you're thinking about doing here: https://www.mattcutts.com/blog/pagerank-sculpting/ and it's important to note that it's not nearly as effective. The thing I would focus more on is the DA and quality of referrals to your site. Secondly, linking out from pages is actually a positive strength indicator when done in the right way, per Cutts in the same article, "In the same way that Google trusts sites less when they link to spammy sites or bad neighborhoods, parts of our system encourage links to good sites." Perhaps your product pages could be strengthened further by this as well.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Using NoIndex Tag instead of 410 Gone Code on Discontinued products?
Hello everyone, I am very new to SEO and I wanted to get some input & second opinions on a workaround I am planning to implement on our Shopify store. Any suggestions, thoughts, or insight you have are welcome & appreciated! For those who aren't aware, Shopify as a platform doesn't allow us to send a 410 Gone Code/Error under any circumstance. When you delete or archive a product/page, it becomes unavailable on the storefront. Unfortunately, the only thing Shopify natively allows me to do is set up a 301 redirect. So when we are forced to discontinue a product, customers currently get a 404 error when trying to go to that old URL. My planned workaround is to automatically detect when a product has been discontinued and add the NoIndex meta tag to the product page. The product page will stay up but be unavailable for purchase. I am also adjusting the LD+JSON to list the products availability as Discontinued instead of InStock/OutOfStock.
Technical SEO | | BakeryTech
Then I let the page sit for a few months so that crawlers have a chance to recrawl and remove the page from their indexes. I think that is how that works?
Once 3 or 6 months have passed, I plan on archiving the product followed by setting up a 301 redirect pointing to our internal search results page. The redirect will send the to search with a query aimed towards similar products. That should prevent people with open tabs, bookmarks and direct links to that page from receiving a 404 error. I do have Google Search Console setup and integrated with our site, but manually telling google to remove a page obviously only impacts their index. Will this work the way I think it will?
Will search engines remove the page from their indexes if I add the NoIndex meta tag after they have already been index?
Is there a better way I should implement this? P.S. For those wondering why I am not disallowing the page URL to the Robots.txt, Shopify won't allow me to call collection or product data from within the template that assembles the Robots.txt. So I can't automatically add product URLs to the list.0 -
Should I nofollow/noindex the outgoing links in a news aggregator website?
We have a news aggregator site that has 2 types of pages: First Type:
Technical SEO | | undaranfahujakia
Category pages like economic, sports or political news and we intend to do SEO on these category pages to get organic traffic. These pages have pagination and show the latest and most viewed news on the corresponding category. Second Type:
News headlines from other sites are displayed on the category pages. The user will be directed to that news page on the main site by clicking on a link. These links are outgoing links and we redirect them by JavaScript (not 301).
In fact these are our websites articles that just have titles (linked to destination) and meta descriptions (reads from news RSS). Question:
Should we have to nofollow/noindex the second type of links? In fact, since the crawl budget of websites is limited, isn't it better to spend this budget on the pages we have invested in (first type)?0 -
No: 'noindex' detected in 'robots' meta tag
I'm getting an error in Search Console that pages on my site show No: 'noindex' detected in 'robots' meta tag. However, when I inspect the pages html, it does not show noindex. In fact, it shows index, follow. Majority of pages show the error and are not indexed by Google...Not sure why this is happening. Unfortunately I can't post images on here but I've linked some url's below. The page below in search console shows the error above... https://mixeddigitaleduconsulting.com/ As does this one. https://mixeddigitaleduconsulting.com/independent-school-marketing-communications/ However, this page does not have the error and is indexed by Google. The meta robots tag looks identical. https://mixeddigitaleduconsulting.com/blog/leadership-team/jill-goodman/ Any and all help is appreciated.
Technical SEO | | Sean_White_Consult0 -
Can panda penalize News publisher sites?
Hey Guys,I was wondering how Panda behaves with news publisher sites.A site with +-1M visits a day that publishes +-300 news articles a day and the life of each article is one week top, given the nature of a news articles -->only relevant now.After one week the the news articles have virtually no page views. This results on a site with thousands of quality content pages that has no page views for years.Is it possible that the site gets penalized by panda for having thousands of pages with no visits?
Technical SEO | | Mr.bfz0 -
Exclude Noindex, Followed pages from sitemap?
Hello Everyone! This is a question about my site, which is running on WordPress. Currently, I have category page to have the noindex, follow attributes, as they have little unique content. I do have them currently in my sitemap.xml file, however. Should I remove them from the sitemap since Google technically shouldn't index them? Thanks for your help!
Technical SEO | | Zachary_Russell0 -
Panda and unnatural links caused ranking drop
Hi I have been approached to do some SEO work for a site that has been hit badly by the latest panda update 3.3, they have also had a warning in their Google webmaster tools account saying they had unnatural looking links to their site, they received this in 26 Feb and that prompted them to stop working with their excising seo company and look for a new one. Apparently their rankings for the keywords they were targeting have dropped dramatically, but it looks like just those they were actively building back links for, other phrases do not look affected. Before I take them on I want to be clear that it is possible to help them reclaim their rankings? I have checked the site and the on-page seo is good, the site build is good, just a few errors to fix but the links that have been built by the seo company are low quality with a lot of spun articles and the same anchor text so I see what the Google webmaster tools message is refuring to. I do not think these links can be removed as there is no contact details on the sites I checked I have not checked all of them but a random sample does not show promise, they are from low authority domains. So if I am to take them on as a client and help them to regain their previous rankings what is the best strategy? Obviously they want results yesterday and from our phone call they would rather someone else did the work than them, so my initial response of add some better quality content that others in your industry would link to as a reference did not go down well, to be fair I think it is a time issue there are only 3 people in the company and they are not technical at all. Thanks for your help Sean
Technical SEO | | ske110 -
Seek help correcting large number of 404 errors generated, 95% traffic halt
Hi, The following GWT screen tells a bit of the story: site: http://bit.ly/mrgdD0 http://www.diigo.com/item/image/1dbpl/wrbp On about Feb 8 I decided to fix a large number of 'duplicate title' warnings being reported in GWT "HTML Suggestions" -- these were for URLs which differed only in parameter case, and which had Canonical tags, but were still reported as dups in GWT. My traffic had been steady at about 1000 clicks/day. At midnight on 2/10, google traffic completely halted, down to 11 clicks/day. I submitted a recon request and was told 'no manual penalty' Also, the 'sitemap' indexes in GWT showed 'pending' for 24x7 starting then. By about the 18th, the 'duplicate titles' count dropped to about 600 or so... the next day traffic hopped right back to about 800 clicks/day - for a week - then stopped again, down to 10/day, a week later, on the 26th. I then noticed that GWT was reporting 20K page-not found errors - this has now grown to 35K such errors! I realized that bogus internal links were being generated as I failed to disable the PHP warning messages.... so I disabled PHP warnings and fixed what I thought was the source of the errors. However, the not-found count continues to climb -- and I don't know where these bad internal links are coming from, because the GWT report lists these link sources as 'unavailable'. I'v been through a similar problem last year and it took months (4) for google to digest all the bogus pages ad recover. If I have to wait that long again I will lose much $$. Assuming that the large number of 404 internal errors is the reason for the sudden shutoff... How can I a) verify the source of these internal links, given that google says the source pages are 'unavailable'.. Most critically, how can I do a 'RESET" and have google re-spider my site -- or block the signature of these URLs in order to get rid of these errors ASAP?? thanks
Technical SEO | | mantucket0 -
Would moving a large part of our website onto a separate website be SEO suicide?
Hello, Our website currently has what I would call educational and sales pages - which sells our services and also a techy section for the developer community. The developer pages on the website have some of the highest authority pages that we have and equates for about 50% of the content. It has been proposed to move the developer pages onto their own domain - away from the main website. Now, would this crush a lot of the SEO benefit that we have on our main site? Does anyone know of a workable solution that would help retain the SEO. Would linking to our main site from the developer site help? It would be great to hear what people think, Thanks,
Technical SEO | | esendex0