Index or No Index (Panda Issue)
-
Hi,
I believe our website has been penalized by the panda update. We have over 9000 pages and we are currently indexing around 4,000 of those pages. I believe that more than half of the pages indexes have either thin content. Should we stop indexing those pages until we have quality page content? That will leave us with very few pages being indexed by Google (Roughly 1,000 of our 9,000 pages have quality content). I am worried that we would hurt our organic traffic more by not indexing the pages than by indexing the pages for google to read. Any help would be greatly appreciated.
Thanks,
Jim Rodriguez
-
Firstly, please don't assume that you've been hit by Panda. Find out. Indexation count is generally not a good basis for assuming a penalty.
- Was there a traffic drop around the date of a known Panda update? Check this list. https://moz.com/google-algorithm-change . If the date of traffic drop lines up, you might have a problem. Otherwise it could easily be something else.
- How many links does your site have? Google indexes and crawls based on your authority. It's one area where it doesn't really matter where the links go: just having more links seems to increase the amount your site is crawled. Obviously the links should be non-spammy.
- Do you have a site map? Are you linking to all of these pages? It could be an architecture issue unrelated to penalty.
If it is a Panda issue: generally I think people take the wrong approach to Panda. It's NOT a matter of page count. I run sites with hundreds of thousands of URLs indexed, useful pages with relatively few links and no problems. It's a matter of usefulness. So you can decrease your Panda risk by cutting out useless pages - or you can increase the usefulness of those pages.
When consulting I had good luck helping people recover from penalties, and with Panda I'd go through a whole process of figuring out what the user wanted (surveys, interviews, user testing, click maps, etc.), looking at what the competition was doing through that lens, and then re-ordering pages, adjusting layout, adding content, and improving functionality toward that end.
Hope that helps.
-
Every case is different, what might work for someone else may not work for you. This depends on the content you are saying is thin - unless it has caused a penalty, I would leave it indexed and focus on writing more quality content.
-
I think it is a critical issue - you have thin content on your most of the pages; If google bot can access your thin content pages, you may not recover from panda until you add quality content on your all the pages and that pages indexed by google (it may take a very long time)
If you have added noindex (just you told Google that do not index pages), still Google can access your pages so, google can still read your thin content and you can not recover any how.
so as per my advice you need to either remove all thin content from your pages and add quality content as fast as you can and tell google to indexed your new content (using fetch option in Google search console) (recommended) or add nofollow and noindex both to the thin content pages (not recommended) because you may lose huge number of traffic and (may you can't recover from panda - i am not sure for this statement).
-
Hi Jim,
From my own experience with Panda-impacted sites, I've seen good results from applying meta robots "noindex" to URLs with thin content. The trick is finding the right pages to noindex. Be diligent in your analytics up front!
We had a large site (~800K URLs), with a large amount of content we suspected would look "thin" to Panda (~30%). We applied the noindex to pages that didn't meet our threshold value for content, and watched the traffic slowly drop as Google re-crawled the pages and honored the noindex.
It turned out that our analytics on the front end hadn't recognized just how much long-tail traffic the noindexed URLs were getting. We lost too much traffic. After about 3 weeks, we essentially reset the noindex threshold to get some of those pages back earning some traffic, which had a meaningful impact on our monetization.
So my recommendation is to do rigorous web analytics up front, decide how much traffic you can afford to lose (you will lose some) and begin the process of setting your thresholds for noindex. It takes a few tries.
Especially if you value the earning potential of your site over the long term, I would be much more inclined to noindex deeply up front. As long as your business can survive on the traffic generated by those 1000 pages, noindex the rest, and begin a long-term plan for improving content on the other 8000 pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap include all site links or just ones we want indexed?
Got a quick sitemap question. We have a clients site built in opencart and are getting ready to submit the sitmap. The default sitemap setting generates urls right off of the root. For example site.com/product. These urls are also accessible through the site itself. We prefer to give the site some depth and have structured the products so the urls are site.com/category/product. All of the product pages have canonicals including the category so we should not have to worry about duplicate content on the /product page vs the /category/product page. My question is both types of product pages are included in the sitemap at the moment. Since we don't want google to index the /product urls should we leave them off of the sitemap even though they are readily accessible from the frontend(though not linked)? Or just leave them and let the canonical tag be used in directing google as to which urls to index. Thanks in advance.
On-Page Optimization | | Whebb0 -
Google Panda This Past Weekend Impact
I understood that Google was implementing a major Panda refresh this past weekend. Did it happen? Anyone notice any impacts? What changed?
On-Page Optimization | | lbohen0 -
Sitemap error is reported when using a sitemap-index generated by Yoast
I've installed the Yoast SEO Plugin for wordpress and I've setup the sitemaps using it. I saw the tool has generated the Sitemap index file http://www.phraseexpander.com/sitemap_index.xml with different indexes for posts and pages I've submitted that to google and it's indexed. When I use Seoquake to check my website, I see that it says that the sitemap is missing (in fact http://www.phraseexpander.com/sitemap.xml) is returning 404. Shall I fix that? Shall I do a 301 redirect in my .htaccess file to http://www.phraseexpander.com/sitemap_index.xml Thanks.
On-Page Optimization | | nagar0 -
Duplicate Content Issues with Forum
Hi Everyone, I just signed up last night and received the crawl stats for my site (ShapeFit.com). Since April of 2011, my site has been severely impacted by Google's Panda and Penguin algorithm updates and we have lost about 80% of our traffic during that time. I have been trying to follow the guidelines provided by Google to fix the issues and help recover but nothing seems to be working. The majority of my time has been invested in trying to add content to "thin" pages on the site and filing DMCA notices for copyright infringement issues. Since this work has not produced any noticeable recovery, I decided to focus my attention on removing bad backlinks and this is how I found SEOmoz. My question is about duplicate content. The crawl diagnostics showed 6,000 errors for duplicate page content and the same for duplicate page title. After reviewing the details, it looks like almost every page is from the forum (shapefit.com/forum). What's the best way to resolve these issues? Should I completely block the "forum" folder from being indexed by Google or is there something I can do within the forum software to fix this (I use phpBB)? I really appreciate any feedback that would help fix these issues so the site can hopefully start recovering from Panda/Penguin. Thank you, Kris
On-Page Optimization | | shapefit0 -
What are the best practices for google indexing ajax response urls?
We just did a build to our site and our server erros went up to over 9,500. After looking into it, it seems like google is crawling the ajax urls and coming back with the errors. Here is one example http://www.rockymountainatvmc.com/productDetail.do?navType=type&vehicleId=1423&webTypeId=68&navTitle=Drive&webCatId=9&prodFamilyId=29660 If you know of any good articles on this please send them my way.
On-Page Optimization | | DoRM0 -
Why are some of page indexed and others not
I have created a site structure like this: domain/for-sale/brand domain/for-sale/brand-model domain/for-sale/brand-model/pg1 domain/for-sale/brand-model/pg2 domain/for-sale/brand-model/pg3 etc.... I cannot understand why the domain/for-sale/brand-model does not seem to be indexed, yet the domain/for-sale/brand-model/pg6 is? This is a new site, but I cannot understand why this URL would be indexed without the others... Any ideas? My home pages has links to the domain/for-sale/brand, this page has links to domain/for-sale/brand-model1, domain/for-sale/brand-model2 etc, each of these pages have links to domain/for-sale/brand-model/pg1, domain/for-sale/brand-model/pg2 etc...
On-Page Optimization | | MirandaP0 -
How do you create a 301 redirect for www.mysite..com/index.html in htaccess.
I understand that it is possible to create a 301 redirect for www.mysite..com/index.html to www.mysite.com. (as well as subdirectories.) How is this accomplished? My hosting company says that setting this up in htaccess will cause "Apache to geti into an infinite loop and the page won’t load." I have read on the forum that this is possible. Any help would be greatly appreaciated. THanks. Perri
On-Page Optimization | | PerriCline0 -
Getting pages indexed by Google
Hi SEOMoz, I relaunched a site back in February of this year (www.uniquip.com) with about 1 million URL's. Right now I'm seeing that Google is not going past 110k indexed URL's (based on sitemaps). Do you have any tips on what I can do to make the site more likeable by Google and get more indexed URL's? All the the part pages can be browsed to by going to: http://www.uniquip.com/product-line-card/suppliers/sw-a/p-1 I've tried to make the content as unique as possible by adding random testimonials and random "related part numbers" see here: http://www.uniquip.com/id/246172/electronic-components/infineon/microcontrollers-mcu/sabc161pilfca Do I need to wait more time and be more patient with Google? It just seems like I'm only getting a few thousand URL's per day at the most. Would it help me if I implemented a breadcrumb on all part pages? Thanks, -Carlos
On-Page Optimization | | caneja0