Why did Moz remove thumbs down from blog posts?

[Trevor-Klein]

You may have already noticed one of the decisions we made when we redesigned the Moz Blog:
We removed thumbs down from the posts. And it was largely in the name of transparency.

Wait, HUH? You took away a method of critique, and you're calling that transparent?

Yes. Here's the scoop: Thumbs down are one of the most cryptic, uninformative, and often passive-aggressive forms of feedback on the Internet today. By removing the mud from the water, we make the entire picture clearer. It's so easy to see a handful of thumbs down on a post (we would almost always get 1-2), and begin hypothesizing what went wrong. We shouldn't have published that one. The topic was too tangentially relevant; it was too long or too hard to follow. There wasn't enough evidence to support the claims. We could dive into analytics, attempting to glean clues about what happened, but in reality, any one of the following are reasons someone might thumb a post down:

• The title is confusing
• The topic is one that I'd like to deny exists (algo update, e.g.)
• The milk I poured on my cereal this morning had gone bad, and I need to take out this frustration somehow
• I once had a falling-out with the author of this post
• I still have a bad taste in my mouth about yesterday's post, which is skewing my thoughts about this one
• I found one of the comments offensive
• My finger slipped on my phone while I was trying to thumb this post up (we've confirmed this happens)
• I didn't like the author's self-promotion in this post
• I saw the new Star Wars trailer, and am terrified that Disney might think including Jar Jar's long-lost brother in the new film is a good idea. I hate everything right now.

Okay, the last one might be a stretch. But you get the idea.

Sometimes a post would receive a disproportionate amount of thumbs down simply because the author was proposing an idea that wasn't popular, no matter its importance. One great example: Carson Ward wrote a fabulous post in 2012 titled "Guest Blogging – Enough is Enough," divining what Matt Cutts would write about nearly 17 months later. The response? 45 thumbs down – one of the most maligned posts in the history of the Moz Blog.

Authors have emailed us in a tizzy, asking if their thumbs down meant they weren't quite right for the Moz audience, and in replying to them we came to this overarching realization: We didn't know why they got thumbs down, and we couldn't find out with any certainty, but more often than not it just didn't really matter. We were confident in their points and their presentation, and real criticism would nearly always show up in the comments.

All that said, we love it when people offer up constructive criticism. We always take it to heart, and hearing directly from you all is the best way we can improve. For that reason among many others, we'll always have the comments below the post. If you feel like a post wasn't up to snuff, please take a moment and tell us why in those threads (please keep it TAGFEE).

One last note: Thumbs down remain available on comments, though that's a temporary stop-gap while we work on a more informative system for flagging comments that are offensive, or facepalm-worthy attempts at links (they're nofollowed anyway!), or otherwise inappropriate for our community.

We'd love your questions or comments on this change, and hope you're enjoying the new look of the Moz and YouMoz blogs!

[Trevor-Klein]

Joe, thanks so much for this idea. It's one I've been playing around with for a while -- we've certainly noticed the same pattern (with some noteworthy exceptions), and we frequently discuss whether or not it's cause for concern. We've got the data, and would certainly benefit from the experience of putting such a post together, so I might just treat your reply here as the straw that broke the camel's back.

With regard to well-known SEO leaders, I have a theory about that, too -- I like to draw an analogy to classes at a university. In the early days, everyone is a freshman, getting their bearings and eager to try things out and participate. The sophomore year brings wisdom that wasn't there before. By junior year, you could throw the term "expert" around, but time starts to become a factor -- reading and commenting on blog posts isn't given quite the same priority it once was -- and by senior year, you're thinking about what comes next. Then that happens.

I think, for the most part, the industry legends that made this blog what it is today have largely moved on to other phases of their lives. Some are CEOs trying to keep their own companies in order, some have even left the industry altogether. In their place, though, we have a new cohort of freshmen and sophomores that'll take the Moz Blog to the next level. I've seen plenty of folks contributing incredibly thoughtful comments and participating regularly, and I think it won't be long before they're seen at the same level as those original legends.

My two cents. Now to get to work on that post...

[Joe.Robison]

@Trevor - I've noticed an anecdotal trend since around 2011/2012 as the average thumbs up on any Moz post has decreased. Seems that after the switch from SEOmoz to Moz there was a dip, and then a second dip after this most recent redesign. Am I totally off here, or is this something you guys are seeing.

Would be a cool blog post to compare the trends of page views and thumbs up over time and any big meta trends that go along with that, could be very informative. I'm sure there's some UX and brand engagement metrics to go along with that.

Have also noticed less comments from well known SEO/other leaders since around that time. Could very well be a correlation between those who were leaders in 2012 that have since just received so much business that they can't keep up with commenting and/or they've shifted attention to their own websites and content. I'm thinking of people like Wil Reynolds, Ross Hudgens, Dan Shure, even Cyrus Shepard and Dr. Pete, among others!

Again, would make an excellent blog post!

[EGOL]

Whoops!    I thought they were gone and came back.

Thanks for letting me know.

[Trevor-Klein]

Hi EGOL,

No reversal -- they're still gone from blog posts. They were never removed from blog comments (see the little note at the bottom of my OP above), nor were they removed from Q&A. Are you seeing something different? Please let me know if so...

[EGOL]

I see that thumbs down are back.

Nice.

Although they can be hard to interpret, I think that it is good to have them.

---  It allows you to give thumbs down to stuff that you think is rubbish

---  It allows you to receive thumbs down and be proud about your contrarian opinion

---  and... it allow you to ponder behavior that you find to be totally irrational   

Care to comment on the reversal ?

[EGOL]

Thanks for your perspective, Trevor.

[Trevor-Klein]

Thanks for the thoughtful response, EGOL. While I wholeheartedly agree that we should find ways to increase engagement, I actually disagree that our posts show relatively low engagement. I think the Moz Blog's comment threads are some of the most extensive, insightful threads you can find, and it's maybe my favorite thing about our blog. A quick look at SEL's most popular recent posts reveals around 20 comments on average. The latest six posts from Content Marketing Institute have a total of 9 comments, and one of HubSpot's blog managers recently confided to me that she wished they could get the engagement we do. Even you subtly referenced a "big tribe of fans" in your comment above. We're incredibly lucky to get as many great comments as we do, and while the "Nice post!" versions are indeed prevalent, we also see some really, really insightful comments, and I don't want to take that for granted for one single minute.

That said, more quality engagement is always a good goal, and I hear you about many folks not having time. I think, though, that I'd suggest folks go ahead and leave that quick feedback in the form of a comment. We do indeed look at every comment on every post, so even if it's only a sentence or two and it doesn't spark a great debate, we hear the feedback... and I really don't think anonymizing it or encouraging drive-by feedback is productive. We want to encourage and nurture a site where people feel as comfortable as possible offering up their opinions, and a potentially fantastic idea might get left unsaid because the person who wanted to leave it saw a bunch of "trolls, haters, and idiots" who'd already stunk up the threads.

It's a delicate balance, but given the choice, I'll take fewer enthusiastic discussions over more quick feedback any day of the week -- it benefits those who comment and those who simply peruse the threads.

[EricaMcGillivray]

We are currently in the process of revising our community etiquette, which will hopefully more explicitly cover some undesired behaviors that it doesn't. Though the current one does cover spammy thumbs.

[EGOL]

Do this, go to the homepage and look at the number of comments that posts in the Moz Blog receive.  What do you see?  40?  60?  80?  Something with a lot of activity might get over 100?

Now, go into the post and examine those comments.  Many of them are fan posts.  Some of them are comments by people who "don't get it" (sometimes I am in that group).

The genuine engagement is often 1/2 or less of the total comments at most, by my count.

When I compare that to what I believe the stature of Moz is in the search industry and the number of people who visit the blog, those are low numbers.  Really low numbers.  In my opinion.

In an industry where there is so much uncertainty and so much data, I would think that the engagement and debate would be a lot higher, and the amount of diverse opinion would be a lot higher.

Why is that?  We could list a hundred reasons or more.   Competitive industry and don't want clients or boss seeing secrets being revealed.  Lack of time.  Earning money is more important and don't want boss or clients seeing time being spent.  Hesitant to disagree with the poster because of Moz fans.  Hesitant to disagree with poster because he/she is world famous.  They think TAGFEE means you gotta love everything.  Not motivated to do the research and writing required to present a good argument that contradicts world famous expert and a big tribe of fans. Don't want to come back several times to engage in protracted debate.   (And many people spend an hour typing a big, researched, passionate comment, only to see it disappear when the "post response" button is pressed.)  I could go on.  I understand why the engagement isn't higher.

So, I think that in addition to comments, the  Moz Blog needs a way that people can quickly give feedback.  Make it quick and you will get more.  Expect people to write and you will find that people hate writing.  They really hate writing.  Everybody knows that you gotta pull teeth to get content, right?

Many of us are online merchants and we daily receive one to five star feedback from our customers along with one or two sentences.  Lots of customers do that because it is quick and easy.  As merchants we  often make business decisions based upon their very brief input.  I think that can happen at Moz.   Yes, you will have trolls, haters, idiots and people who "didn't get it" responding.  If those comments are attributed to member names then the reader and the poster will have some ability to place value on them.

So I would encourage Moz to find a way to get this quick feedback from the many busy people who visit and I think that will help the participation rate and give feedback that is more meaningful than a simple up/down vote.

[DonnaDuncan]

Which I think falls in line with what others, including myself, were saying earlier.

• Guidelines need to be published. e.g. A thumbs down is not a bad thing, it's a sign of disagreement.
• Comments need to be encouraged. We all don't have to agree, but we do need to explain ourselves so others can decide which side of the fence they fall upon and contribute to the conversation. Disagreements are healthy so long as they're civil, open and honest. That's how a community grows and evolves.
• Guess you'd also have to reflect whatever policy you come up with in the attribution of Mozpoints as well.

Glad you came back. I think you've made a lot meaningful contributions to the site.

[MattAntonino]

I'm happy for this change because I think one thumbs down would lead to more. For instance, let's say I make what I think is an important (but possibly controversial) comment.  It gets 1 or 2 thumbs down then another then another then another just because people like to pile on. Then, because I got 7 thumbs down, maybe I'm angry. And I take it out on the rest of the comment stream by downing every other comment just so we're "even."  It happens - I am 100% sure of that.

Now, let's say you want to encourage discussion. What does thumbs down do? It makes everyone yes-men & women. Let's just agree, maybe someone will like my comment. Big whoop. lol  I'd rather make the honest critique rather than tell someone what they want to hear.  When I first came back to Moz after a few months off, I almost immediately got 5 very quick thumbs down. It was a reminder that the system in place suggested that I was supposed to just agree with everything everyone says.

Let's put it this way - one of the most discussed, necessary posts on Moz last year had more thumbs down than any I've ever seen on Moz. Yet, it was a necessary conversation no matter which side of the debate you fall on. Negative commentary doesn't have to be a bad thing. I think Moz should encourage more posts like Josh B's, not fewer. If people are less "afraid" to post controversial things for fear of being knocked back, that is good for Moz.

[Trevor-Klein]

I agree 100%, Paul -- thanks for the idea. The Q&A section is due for an update before too long, and I think that's a great change for us to keep in mind.

[paulissai]

I have no problem whatsoever with the removal of the thumbs down and think it's a good move for all the aforementioned reasons.

Another thing that I think could be improved is the star rating system attributed to users. I have always assumed star ratings denote a scoring system that follows something similar to 1 star=poor, 2 stars=below average, 3 stars=average, and so forth. I think, when applied to a users contribution and experience, the star rating system is confusing and misleading. If I see someone who is, say, 2 stars out of 5, I automatically assume their ability to respond to a question as "below average", when in reality it may simply mean they are new to the forum or only post sporadically.

If I could suggest an alternative system, I would instead like to see one similar to the Whirlpool forums where you are given a user title that correlates to your participation and experience levels.

Just a thought

[renehansen]

I don't think that removing the thumbdown is a bad idea. A negative thumb should have some sort of comment associated, in order to be critique.
I was worried that the thumbs down was removed all together, but thankfully only on blog posts!

[Trevor-Klein]

Bingo. So glad you agree.

[MatShepSEO]

Totally agree with this move. Thumbs down is too easy, not specific, and too anonymous. If a reader really has constructive feedback then they should be prepared to post a comment.

[Trevor-Klein]

Haha! So meta...

[DonnaDuncan]

I gave you a thumbs up

[Trevor-Klein]

Entirely agreed. Thanks!

[grobro]

I like the idea! Very often I was asking myself why a particular blog post got 6 thumbs down when I thought it was brilliant. Criticism should always be constructive.

[Trevor-Klein]

Interesting idea. Maybe we can have a little tooltip for the thumbs somewhere. Will have to noodle on it.

Thanks again!

[DonnaDuncan]

I'd just say publish " If something resonates with you, give it a thumbs-up. If a comment evokes a negative reaction, go ahead and thumb it down." where it's visible and influential. You want as many visitors as possible to see those guidelines and they won't if they're just published here.

[Trevor-Klein]

Thanks for the kind words, Donna!

The guidelines would be quite simple, I think. If something resonates with you, give it a thumbs-up. If a comment evokes a negative reaction, go ahead and thumb it down. If we get more prescriptive than that, I think we risk losing the usefulness of the system.

One thing we've done on the blog with the design change is added the thumb/comment buttons to the sticky module on the right side, so you can thumb a post up without scrolling back to the top or down to the bottom. If you've got any other suggestions for the future, we'd love to hear 'em -- I'm right with you, often blown away by the quality of comment threads. It's one of my favorite things about the blog!

[Trevor-Klein]

Yes.

If something seems like it'd be a clue to a spammy account, we look at it. Irrelevant comments (especially those with links to irrelevant sites) are an easy sign. An irrelevant comment with a link from an account that was created that same day (and has the bare minimum profile fields filled) is a pretty good sign that account isn't going to be a productive member of the community. And yep -- if we're on the fence about intent, we'll definitely look at whether the account has been used in recent history (as well as other things), too. We usually opt for the benefit of the doubt... lots of people either misunderstand the point, or make an honest mistake.

[DonnaDuncan]

This is good to know Trevor; thanks for sharing. I think it would be great of you guys published some guidelines for how to use the thumbs-up / thumbs-down feature, and just to encourage more folks using it. I'm sometimes amazed at the great stuff I learn from comments and wonder why I'm the only one that seems to appreciate it.

PS - I checked to see what guidelines are published before stating this. Your "answering questions" help link is broken.

[PatrickDelehanty]

Thats awesome to know Trevor! That's exactly what I was referencing above - good to know you guys are on top of it!

Without giving too much away, what are other elements to a Moz profile do you monitor? I'd imagine comment spam is a major one. Does activity or dormancy come into play at all?

[Trevor-Klein]

Little-known fact: We actually do have a way of monitoring the thumb activity of accounts, and regularly send emails to folks who are abusing the system. I'd say we ban 5-10 accounts every day that are blatantly violating our community guidelines. It's a bit laborious to manually review things like that, and we're super-careful not to go any farther than addressing spam/abuse, but it's one of the ways we're able to keep the comment threads productive.

[KevinBudzynski]

Good point Patrick. Adding another layer, typically these rogue accounts may have just a few MozPoints and are newly created. If a user has been w/moz for let's say one year and have a couple hundred points, they may typically be more prone for spammy comments.

[PatrickDelehanty]

I agree with Kevin here.

If there was some way to track rogue accounts that continually thumbs down content, that would be great, although that's highly unlikely. I like that it's a manual process for Moz - assessing what went wrong and what you can do to improve it.

Like with TAGFEE, I think it will force users to write constructive comments on they didn't like the post, or if the post didn't speak to them. It will make Moz better. Thumbs down are extremely easy to be passive aggressive with.

My thoughts!

[Trevor-Klein]

Thanks, Kevin! Appreciate the support.

[KevinBudzynski]

I think it's a good move. I have seen thumbs down for some of the best replies and could not figure out why other than the points your brought up. Sure, thumbs down is good for spammy-self promoting-irrelevant threads. However, if the reply is informative/useful and keeps the discussion progressing in an useful matter, why do we need them? The tricky part is implementing a mechanism to filter these two items out. Good luck.