On plugin/theme "ratings"

#1

With ClassicPress using Semantic Versioning numbering for updates, I thought we could discuss ratings in a similar vein. Wordpress.org uses a misleading ratings system - to rate a plugin a user is asked to give it a rating of between 1 and 5 stars. This gives the misleading impression that a mediocre plugin with equal satisfaction to dissatisfaction would be rated 2.5 since that is half of 5, however since the minimum rating is 1 not 0 the mediocre score is actually 3.

We can see the results of this with Gutenberg. Its rating is displayed as 1.9 out of 5, however if people were instead asked to rate the plugin between 0 and 5 the rating would be closer to 1.2 out of 5. If asked to rate between 0 and 4 the rating would be closer to 0.9 out of 4 (it would be exactly this assuming everyone just dropped their rating by one star, but the subjective psychological effect of the pseudo 5-star system could be skewing results).

Part of the problem is in designing the feedback interface. It’s very easy, and visually pleasing, to just have empty stars and let users click them to indicate their rating - however such a system fails to incorporate the 0-star rating. Either 0-star ratings are not permitted at all (like wordpress.org), or it simply isn’t clear to the user that it is an option. A sensible way around such a problem is by using a drop-down menu instead, or ye olde radio buttons to select between options (example).

5 Likes
#2

Moved to the plugin directory subforum, as this is something we can consider for our directory.

I like 1-5 stars for simplicity. Still, another option would be something like (Awful, Bad, OK, Good, Excellent), where these would map to (-2, -1, 0, +1, +2) internally.

3 Likes
#3

-2 … +2 sounds better than 1 … 5, because in the case of 1.9 star rating it would be worse than “bad”, near “awful”, compared to today’s perception of “almost neutral”.

1 Like
#4

Right now there’s only one rating for a plugin or theme - it’s a bit of a blunt instrument.

What about rating several aspects of a plugin? Obviously we can’t go crazy with that, but if we picked 3 things?

“Terrible, Bad, Neutral, Good, Excellent” would be my choice.

1 Like
#5

I like this idea: negative numbers make it clear if it’s an unfavorable rating, rather than a lukewarm rating.
I could also see some value in rating plugin performance and separately, plugin support.

1 Like
#6

Yes, world is moving in that direction…
if we make the minus/plus my suggestion is to delete the “neutral” 0, it just equalise a vote around nothing. so either you like it or not.

#7

I think people understand 0-4 or 0-5 just fine, it’s just 1-5 or 1 to anything that is perceived incorrectly. As I mentioned I think the decision to use 1-5 is more a UI decision than anything else.

2 Likes
#8

I’m not sure I agree with this, 1 is still a pretty natural base for ratings. A couple of things that would help with the interpretation issues:

  • Show words instead of numbers. Sticking to words like (Terrible, Bad, OK, Good, Excellent) wherever possible means it doesn’t really matter what the numbers are behind the scenes.
  • Don’t make or publish incorrect interpretations ourselves. 1.9 out of 5 stars is not 40%.
1 Like