View source for Dungeons and Dragons Wiki talk:Rating Articles/Archive

==RC Replacement==

I consider the Rating Committee a failure in practice, even if it had potential as a concept. The members were never particularly active in their duties, are even less so now. As it was our vehicle for getting favored articles, and I would actually like to include those on the main page, I want to replace it.

I'm proposing the following:
*Anyone can rate an article (except their own). Seriously. Rating is still giving 0, 1, or 2 points, and you still add the points and the rater's name to the author block for tracking purposes.
*Bronze, silver, and gold ratings are removed. If an article has been rated, even if that rating is a 0, it gets a note that you should look at the talk page to see what people like or dislike about it. When an article gets 10 points, or five Great ratings, it gains Featured Article status (category). 
*Featured articles get minor write-ups and are added to the front page in a random rotation (code for this is already setup, mostly). This step may need to be completed by an admin, depending on page protections.
*If an article gains this status and the community comes back and says that was a bad call, we remove it from rotation and the Featured category.

I understand there are potential problems with this setup, but none so bad that they outweigh the potential benefits of getting featured articles in rotation IMO. I currently plan on implementing this change in 2 weeks, but it's not fixed in stone and I'd appreciate thoughts and comments. - [[User:Tarkisflux|Tarkisflux]] 18:26, 26 May 2011 (UTC)

:Looks tasty.  Can't hurt to try at the least. -- [[User:Eiji-kun|Eiji-kun]] 18:43, 26 May 2011 (UTC)

:: The only way I see it hurting is if retards rate stupid things highly. Also I'd add a caveat that you can't rate an article you created. Though that still carries a potential problem with sock-puppetry... --[[User:Ghostwheel|Ghostwheel]] 19:57, 26 May 2011 (UTC)

:::Hence the community veto. And yeah, you still couldn't vote for your own stuff, should have made that clear. It's not like you rate something up and it automatically gets added though, someone would still need to write a blurb and make some changes to a partially/fully protected page. This would just open up the process of getting articles sufficiently rated to qualify for featured status. ''[Note - minor edits made to process above for clarity].'' - [[User:Tarkisflux|Tarkisflux]] 20:22, 26 May 2011 (UTC)

:::Bump. Clock is still ticking on this. Get your concerns in now, and marshal whatever community support for or against you feel appropriate :-) - [[User:Tarkisflux|Tarkisflux]] 07:21, 31 May 2011 (UTC)

::::Actually do we need points?  It could just be "Dislike" or "Like", and then the rating being based on the amount of each.  Besides that, I don't have any issue. -- [[User:Eiji-kun|Eiji-kun]] 07:24, 31 May 2011 (UTC)

::::: I don't actually care too much as long as there's a way to veto obviously retarded votes and I especially like the fact that it points people directly to the talk page so they can see the pros/cons of what's being debated. --[[User:Ghostwheel|Ghostwheel]] 07:33, 31 May 2011 (UTC)

::::::Sounds good to me. Especially if raters are encouraged/required to explain their reasoning for like/dislike. --Genowhirl 08:23, 31 May 2011 (UTC)

::::::: Is this like a template thing people use?  Like <nowiki>{{Name: Bob, Pros: It's a pie., Cons: I'm not a little girl.  Score: Approve.}}</nowiki> or something?  I suppose we'd put the code for it in the preload above talk pages.  I think we can do that. -- [[User:Eiji-kun|Eiji-kun]] 08:28, 31 May 2011 (UTC)

(Reverting indent)I'm all for this idea, honestly. - [[User:MisterSinister|MisterSinister]] 09:23, 31 May 2011 (UTC)

:Eiji, are you suggesting we assign a rating of like, dislike, or mixed, and then score the article as the sum of the likes minus the sum of the dislikes? That sounds workable, and we could totally do that. It has the added benefit of allowing people to vote down things that they feel got voted up inappropriately. It doesn't block attempts at sockpuppetry, but those can be pretty easily discovered by checking the ip of a user and sockpuppet votes will just be deleted. I'd set featured status at 4 overall likes in that case, because that seems a reasonable hump to get over if people get to vote you down.

:And yeah, I'd re-purpose the RC Favor template to suit this purpose. So people voting would put a template up. Something like <nowiki>{{Favor |name=Me |rating=Like |reason=Stuff}}</nowiki>, and then people would have to pop over to the article proper and make minor adjustments to the author template. It would all be spelled out on whatever this page turns into post change. - [[User:Tarkisflux|Tarkisflux]] 16:42, 31 May 2011 (UTC)

::Glee, I suggested something good!  :) Rock rock on bro! -- [[User:Eiji-kun|Eiji-kun]] 17:17, 31 May 2011 (UTC)

:::I would have issues with a system where my only recourse to a vote I disagree with is a vote in the opposite direction. Quality control of proffered analyses is my primary concern, but letting anyone write reviews is a step forward. -- [[User:Jota|Jota]] 21:20, 31 May 2011 (UTC)

::::I figured it was both.  Veto and +1/-1 votes.  That way you don't end up with a favored article with 10 likes... and 47 dislikes. -- [[User:Eiji-kun|Eiji-kun]] 21:39, 31 May 2011 (UTC)

:::::I actually suggested (likes - dislikes) >= 4, so your example would be a -37 Eiji. And not featured. 

:::::I've previously said that I'd be fine with community discussion being sufficient to pull an article that somehow got enough votes (and got a writeup, and got into rotation, and so on...) from featured status, and getting that level of support behind pulling it seems like it would come hand in hand with sufficient down votes to get it pulled anyway. I strongly suspect that any articles that get voted up will be community affairs and not abuse of policy (especially given the number of regular users I could ask in chat ''right now'' to review bomb crap if it comes up), so the times when you disagree with a decision and can't get it changed are likely limited to when it's a bunch of regular users who happen to disagree with you. And I don't think we need to worry about those cases too much.

:::::But if you've got any suggestions, they should get put up. I'm trying to get something workable, not dictate how things are going to go. I just don't see a way to let anyone write reviews without also letting the occasional bit of crap through as a result. We could try to reduce it by limiting votes to active users (where active means something specificish) or users who have contributed original articles, but that requires more policing and doesn't even necessarily eliminate it. - [[User:Tarkisflux|Tarkisflux]] 21:43, 31 May 2011 (UTC)

::::::Been thinking about Eiji's mention of a veto. We could add in an admin veto on top of being able to vote stuff off the list as a last resort. Or someone could add the winter cleaning template to it, which would disqualify it until the issues were dealt with. Would either of those options address your concerns Jota? - [[User:Tarkisflux|Tarkisflux]] 00:04, 1 June 2011 (UTC)

:::::::I just don't see a reason for a dislike option, or at least there needs to be a clear distinction between disliking a review as opposed to disliking an article. If you disagree with a review, that's one thing, which would seemingly be handled by the option to veto (assuming that's an option available to everyone) in my mind. If it (the veto power) was just available to admins that changes things a bit. The veto, in my mind, is about the factuality of the analysis, appropriate assessment of balance, and so forth, whereas dislike is "I don't like this because of XYZ," with XYZ being reasons apart from balance, reasons that could just as easily be cited by another as reasons they like the article. For that reason, it seems to me that as long as the review is deemed creditable, there's little reason for a dislike counter. -- [[User:Jota|Jota]] 17:52, 1 June 2011 (UTC)

::::::::I can see that so basically nothing, Like, and Veto then?  Or nothing, Like, and disqualifying it via Winter Cleaning.  I... I can see that working. -- [[User:Eiji-kun|Eiji-kun]] 18:00, 1 June 2011 (UTC)
{{ri}}

I'm only talking about voting for or against an article, and the veto would only apply to an article gaining featured status. If a legitimate user puts up a vote, I don't see any reason to give someone the power to take it down or call it invalid or wrong. I'm trying to get articles actually featured, and putting each review up for individual veto just makes that process harder on top of alienating users with differing opinions. If there is a reason why we should want that, I don't see it.

The proposal that you can only 'vote for', 'block entirely', or 'stay out of the way' of an article doesn't leave you with many options if you disagree with an article gaining featured status. I want an option between 'stay out of the way' and 'block entirely', so that you have options other than 'choke down' and 'be a dick' when an article you dislike gets sufficient votes to become featured. The reason for a dislike vote against an article is to allow people who feel one way or the other to be outvoted and generate featured articles via community consensus, rather than community allowance (i.e. not invoking a block). There's a difference in feel there, and a difference in ease of disqualification that I think is important. I don't mind if contesting an article makes it harder for that article to get featured, but I don't want it to be impossible. If someone put up MS's skill point assignment variant I would vote against it because I disagree with that call, but I wouldn't want to just block it. If enough other people here think it's a great article and should be featured and I get substantially outvoted I'm not going to stand in the way of that, and I don't want anyone else to either (barring legitimate block reasons, hence the suggested admin limitation of the veto). - [[User:Tarkisflux|Tarkisflux]] 18:58, 1 June 2011 (UTC)

:When you put it that way I understand where you're coming from, but I fear petty reasons will be much more prevalent than simple differences in design philosophy or what have you. Still, you're point is made well enough, and I wouldn't have any issues with a dislike button, I suppose. -- [[User:Jota|Jota]] 19:14, 1 June 2011 (UTC)

::I fear pettiness more with the block option actually since pettiness goes farther there, but I think you're right and we'll see some of it in this setup too. I just think it will be manageable. - [[User:Tarkisflux|Tarkisflux]] 22:27, 1 June 2011 (UTC)

==Updated Proposal==

Original proposal above for historic purposes, new one here for clarity.

*Any registered user can rate any article other than their own. Seriously. Rating can be either "Dislike" or "Like", and must include a list of reasons for the stand. A third, neutral rating could be added if there's desire for official mixed feelings rating, but that's not on the table at present.
:*Users can edit or remove their rating at any time.
:*Ratings will not be removed unless they were placed inappropriately. Ratings added by the same user under a different name (sock-puppeting) and ratings added by contributors who have only that contribution are considered inappropriate and subject to removal.
*An article has a score equal to the number of likes minus the number of dislikes. This score, as well as the list of users who have rated the article, will still be listed in the author block for tracking purposes.
*Bronze, silver, and gold ratings are removed. If an article has been rated, even if that rating is a 0, it gets a note that you should look at the talk page to see what people like or dislike about it. When an article gets 4 points, or four more likes than dislikes, it gains Featured Article status (category). 
*Featured articles get minor write-ups and are added to the front page in a random rotation (code for this is already setup, mostly). This step may need to be completed by an admin, depending on page protections. Articles with the Winter Cleaning, Incomplete, or Delete templates are not eligible for listing regardless of the number of likes on them, as we want quality, finished articles listed on the main page.
*An admin may veto an article for cause. This will also be tagged in the author block, and must be explained on the talk page.

And that's where we are right now. - [[User:Tarkisflux|Tarkisflux]] 22:27, 1 June 2011 (UTC)

:That seems good to me. The only thing I would add is that the Bronze, Silver and Gold categories are probably not a bad thing to keep. Basically, having Bronze be +4, Silver +6 and Gold +8 or whatever could work, for example.

:Additionally, will there be a way to de-rate something once you've rated it? I don't see it happening often, but given that content on the wiki can change, sometimes dramatically, this may be a point to consider. - [[User:MisterSinister|MisterSinister]] 23:16, 1 June 2011 (UTC)

::I figure it'd look like this, example of a bad one going to good...
::||Rating Good.
::Reason-Your article is nothing but the word "Bwa" repeated forever.  Also you ate a dog.  -- Eiji, Date 2011
::(Edit: Now it's better.  You gave plushie power as a capstone, and it works.  Changed rating. -- Eiji, Date 2012)
::Blah blah blah.||

::Something like that? -- [[User:Eiji-kun|Eiji-kun]] 23:50, 1 June 2011 (UTC)

:::With featured at 4, there's not a range for the other levels to be meaningful, so I was just going to drop them. I don't think they're a very good indicator at this point anyway, since even if you set silver at 3 that just means 3 more likes than dislikes, and the actual number of reviews could be quite large (indicating a contentious rating). ''Edit'' - unless you're talking about rankings of featured articles... which I guess we could do and make it show up somewhere. Not sure it would be particularly helpful though.

:::Yes, you should be able to edit or remove your own review at any time. I thought that was implied, and have added that above for clarity. - [[User:Tarkisflux|Tarkisflux]] 23:55, 1 June 2011 (UTC)

::::I've been reading the discussion for a while now as it's been going and figured I should probably weigh in my thoughts. The new proposal appears worth giving a shot in practice. While the old RC had potential, it missed the mark in a few places (for me, making rating feel like work rather than fun, and really, we all do this for fun). We, of course, can always adapt and modify things as we go. Which brings me to updating RC Ratings &rarr; Community Rating System. If we want to do this rather than have all the old raters re-rate stuff or simply null out those ratings. RC members should be encouraged to go through their rated articles list I would imagine. Also, any chance of calling it something other than a Featured Article? Dang, I had a suggestion, and now I have forgotten it. I guess that's what I get for not writing it down sooner. --[[User:Ganteka Future|Ganteka Future]] 17:15, 2 June 2011 (UTC)

:::::I was just going to map all of the old ratings to the new setup actually, with a 0 counting as a dislike, a 1 not displaying, and a 2 counting as a like. That's up for debate of course, but I don't see any reason not to just take the old stuff as a starting point. - [[User:Tarkisflux|Tarkisflux]] 20:30, 2 June 2011 (UTC)

::::::Quality Works instead of Featured Articles?  Since we aren't even using actual articles (see:  Publication & SRD namespaces) calling them articles is a bit ingenious.  --[[User:Havvy bot|Havvy bot]] 00:59, 3 June 2011 (UTC)

:::::::"Quality Work" sounds fine. "Community Favorite", "Noteworthy Homebrew", or "Stuff That Should Be In Your Games" might also work. - [[User:Tarkisflux|Tarkisflux]] 03:57, 8 June 2011 (UTC)

==RC Favor Transition==

It looks like this is going through, so we need to talk about migrating old RC reviews over. I think keeping the old template is fine for now, since I can edit the template to just call the new one. And since most have reviews in them already (mine are the exception, cause I got really lazy at the end), it's just a matter of changing the scores around to match the new setup. And that can be handled on the back end, by doing something like:
*"favor=0" would call the new template with a "dislike" vote, and keep the same review.
*"favor=2" would call the new template with a "like" vote, and keep the same review.
*"favor=1" is harder. I'd prefer to have it not call the new template, but flag the page for rater review instead so that the rater can decide if they like it enough to actually give it that vote. Either nothing would display for these, or it would display as a review in need of updating. I could also just turn all of the "favor=1" ratings into "like" votes if the RC would rather have that.

I'll also bot edit all of the existing scores down to half of their current values, rounding down. It's not entirely accurate, but it's close enough and it saves us from going through the pages one at a time to update them. Which should make the whole change over pretty transparent and effortless for old RC members.

Looking for approval on that plan though, since it's your reviews and such that would get migrated. Let me know, preferably soon. - [[User:Tarkisflux|Tarkisflux]] 03:57, 8 June 2011 (UTC)

: I'd rather have 2 ranks of "like", since some things are decent and good-ish that I approve of the idea of a whole, and others are amazing and are fitting for a much stronger approval rating. Perhaps the same with dislike? So it could go anywhere from +2 to -2. --[[User:Ghostwheel|Ghostwheel]] 07:41, 8 June 2011 (UTC)

::I don't think the extra granularity here would be particularly helpful. If you only sorta like an article, are you really going to put time into helping it get community favored status? IMO, people either like something enough to put effort into getting it recognized, dislike it enough to try to block any recognition, or don't have a strong enough opinion to put work into it. There's no nomination process anymore, no people who are supposed to do this stuff, so a partial good rating sounds like the sort that would not get used very much because it's work towards something they only feel sorta good about recommending. Doing this would make migration easier, in that I don't have to change scores really, but that looks like about it to me. 

::It's also not what we've been talking about for the past week, so it looks a bit late to the party to me. Still, I'll write it up and put the two options up for a vote and extend the clock a bit if you want me to. - [[User:Tarkisflux|Tarkisflux]] 04:15, 9 June 2011 (UTC)

::[Edit] Just to be clear, I do actually want you to tell me again that you will spend your time half-heartedly voting stuff up or down without any sort of mandate on you to do it. I want to hear someone say that they'll actually use the extra granularity in rating stuff before I spend any time writing it up and putting the two plans up for a vote. Otherwise I'm inclined to just go with like/dislike and the previously discussed migration plan. - [[User:Tarkisflux|Tarkisflux]] 22:10, 9 June 2011 (UTC)

:::Ghost said in chat that he would actually use this, but didn't want me to put it up for a vote. If someone else says they would use it, it should still go up though. So if anyone would like the extra granularity, let me know please. - [[User:Tarkisflux|Tarkisflux]] 05:52, 12 June 2011 (UTC)

::::Replying just so you know it's there, but I vote against it.  I don't need a reason to add extra granularity to the system, as a "ok" vote is the 1 result, "I see your article, I rate it, but because of X Y Z it is not the cream of the crop, though it is still quite good." -- [[User:Eiji-kun|Eiji-kun]] 06:13, 12 June 2011 (UTC)

:::::Seconding Eiji.  I don't want to have more maintenance work than is necessary. --[[User:Havvy|Havvy]] 06:36, 12 June 2011 (UTC)

== Very Little Information ==

All that this new system tells a reader, at the moment, is whether or not more than 50% of the raters like the article. That can be an insignificantly small distinction, or a very very large one -- for any useful information at all, the reader ''needs'' to read the ratings on the talk page, which begs the question: why bother putting the voting results on the front of the page at all? It would be far more informative if a percentage of favor were displayed, and a number of votes so readers have an idea of the significance of that percentage (something like "<big>'''33%'''</big> &mdash; 2 of 6 raters liked this article."). --[[User:DanielDraco|DanielDraco]] 22:11, 4 July 2011 (UTC)

:It actually tells people one of 4 things: whether more people like it than not, whether equal amounts of people like and dislike it, whether more people dislike it than not, and whether the article has enough more people liking it than not to qualify to be a community favorite. It also includes a link to the talk page to look at the ratings themselves, which is something people should do since we're letting pretty much everyone rate and the substance of the votes is as important as the votes themselves IMO (and that is not something we have room for on the main article). So yeah, it is less informative than your suggestion, but I don't see why giving them more information is helpful in this case. Even if we give percentages they still probably need to look at the substance of the ratings, and the percentage itself could be misleading because of the community favorite bit (since the percentage could be between 100 and 50 depending on rater sample size and still qualify an article). It's also not that "2 of 6 people liked this article", it's that 2 people liked it and 4 people disliked it enough to write a negative review. That distinction seems lost in your wording, especially in cases where "0 of 4 people liked this article". That sounds much more neutral to me than the reviews are going to be.

:That said, I'm not opposed to changing the display. I'll even put this one up if enough users want it and I get outvoted. I just think that we have enough information to direct people who are interested in the ratings to look into them, and that's about as good as we can get. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  22:34, 4 July 2011 (UTC)

::Well, my concern is the fact that a difference of one person is never statistically significant*, and that that's (for odd numbers, at least) all that stands between the above fifty and below fifty thresholds. Only having a 50% threshold simply does not provide any meaningful information.
::<nowiki>*</nowiki><small>Nevermind the fact that we're not likely to ''ever'' have much statistical significance in this, but...may as well maximize it.</small>

::Maybe taking a page from Rotten Tomatoes' book and terming it more like "X of Y ratings were favorable" would be more apt. And what's this about community favorites? I don't see anything explaining that on this page. --[[User:DanielDraco|DanielDraco]] 23:50, 4 July 2011 (UTC)

:::If we want to track two numbers (and you're suggesting likes and total here), why not just track likes and dislikes directly? "X users like this, and Y users dislike this" or whatever? It's extra numbers in the author template, but it's not asking people to do subtraction so it's probably a wash. Unfortunately, it's also not something that's easy to bot edit, so someone would need to do these changes manually (which ''is'' a mark against it as far as I'm concerned, but something that I'd work with if there was enough support for it).

:::As for the community favorite thing, that's just what we're calling the featured articles now (though I guess the name is still up for debate since no name ever got real support behind it). It's any article with 4 more likes than dislikes (though we could add a percentage requirement to that as well if we wanted, like 75%), that someone has written up a short blurb on and put into the rotation on the main page (requires minor admin work due to cascading page protection). It's not a supported thing yet (need to figure out a new main page layout since it will displace something, the doc and functionality is easy), and since I wanted to get the ratings up for users yesterday I blew off putting it in the page for the time being. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  01:24, 5 July 2011 (UTC)

== Community Favorites ==

The section is filled in. Both the name and the 4 more likes than dislikes thing to qualify them were pulled from the previous discussion. If anyone wants those changed, some suggestions would be nice. 

The random rotation thing mentioned in the section was talked about a long long time ago, and basically consists of doing two choose lists: 1 with the most recent 9 articles and another 1 with everything else. The main page would then show one of the 9 newest, or one of the older ones. We can adjust those percentages to whatever we want though, in case those numbers look bad for some reason.

Also, here's what I'm thinking of migrating the Main Page to in order to accommodate the favorites: [[User:Tarkisflux/sandbox/Main Revamp]]. The favorites would take over the current tribute slot (though the tribute would probably go on rotation), and that should give them a sufficiently sized box to display stuff and be noticed. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  05:24, 10 July 2011 (UTC)

== More recent stuff ==

Recently people have been talking about us, and some of us here have been talking back, and DD and I talked about the concerns that had been brought up and which ones were valid or not. You can find both the link to the thread and the chat log between us [http://pastebin.com/UDUrQT6x here]. --[[User:Ghostwheel|Ghostwheel]] 21:14, 16 April 2012 (UTC)

:Having read the chat log above, I'll give a bit of a tl;dr for those who may not be inclined to read it: I particularly agree with the premise that the initial rating system (i.e. "most people like/dislike this article", etc.) is rather misleading and could be improved upon, both because of the ambiguity of the word "like" and the issues with a +1/-1 grading system. Their proposal of a system of each rater giving a score of between -2 and +2 and then averaging them allows us to use a system that more accurately reflects how strongly we feel about something in one way or another. I think it has promise. - [[User:ThunderGod Cid|TG Cid]] 22:28, 16 April 2012 (UTC)

::This is a finer detail, but I still think 1 to 5 or 0 to 5 is better than -2 to +2. Firstly, it's a ubiquitous scale; everyone is very accustomed to using it. Secondly, it better reflects what we're actually judging. After all, shittiness is not actually a thing, any more than darkness or cold are things. It's just absence of merit. It's easier to just conceptualize it as a continuum with no qualitative categories than it is to throw in a turning point between "bad" and "good" at zero. --[[User:DanielDraco|DanielDraco]] 00:08, 17 April 2012 (UTC)

:::We've had this discussion before. I am not opposed to the idea, but here are some of the issues that need to be addressed if we retain the same template based system:
:::*A simple average is not sufficiently granular for determining CommFav or CommOpp IMO. I would want a sufficiently high/low average as well as a sufficient number of raters.
:::*There is no automatic way to determine the average. You are asking each new rater to do math, or to append their score to a list like 2+3+4+5+2, and increment the raters total by 1 (since there is no easy counting code). I think the latter is a better call given my preference for tracking the number of raters anyway.
:::*There is no easy migration process for this setup. Is the proposal that we scrap all previous ratings, or is someone volunteering to go through and manually update / delete things?
:::The above assumes that we're not moving towards a no-template system, like the stars or whatever that appear on other wikis. Anyone proposing we start using one of those needs to take it up with Surgo, as he'd have to install it. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  01:32, 17 April 2012 (UTC)

:::: 0-4 or 1-5 is better than 0-5 since it allows people to choose a point right in the middle if they don't feel good or bad about the material. 
:::: Also I mentioned that for the purpose of favored/opposed articles, there would have to be a rating average below X/above Y AND Z number of raters before it could become favored or opposed. And I'd be fine going back and updating my own favors--I think most people don't have a problem with that, it's easy enough. --[[User:Ghostwheel|Ghostwheel]] 02:09, 17 April 2012 (UTC)

:::::There is a category and a list of pages that a person has rated, broken down by person, specifically for the purpose of getting people to update legacy ratings, and most people still haven't done it. Some of the old raters aren't even active anymore. Any migration scheme needs to address what we do with old ratings, as I do not have much faith in the majority of them being moved over. I would be fine with a direct translation of the really really old ones to 0/2/4 points (instead of 0/1/2), but someone still needs to go through the pages and update all of the author blocks with the new rating values and the new num_raters field. Updating is a non-trivial and not (easily) possible to automate process that needs a roadmap and some serious commitment, or we need to dump everything prior and start fresh. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  03:40, 17 April 2012 (UTC)

I consider fresh starts (perhaps after a two-week grace period) for ratings to be the ''only'' reasonable position to take. We can't exactly ask inactive users what they think of the ratings they previously placed on articles, ''especially'' if those articles have changed since then. Frankly, I also believe that if you actually cared about your ratings, you would modify them. So yeah, I'm all for a clean slate approach. - [[User:MisterSinister|MisterSinister]] 04:49, 9 May 2012 (UTC)

:This is reasonably easy to code for and pretty workable. It's a migration approach I would support, since it would actually get done. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  04:54, 9 May 2012 (UTC)

::I suppose that's a fair idea, given our difficulties in reaching certain people. That said, there are certain legacies that can (and should) be updated, so I think the grace period should also be present to allow that to potentially occur. - [[User:ThunderGod Cid|TG Cid]] 04:58, 9 May 2012 (UTC)

:::I have determined a method to abuse string processing that will give us a total number of raters without having to run a second property. So we only need to list raters (like we already do) and some method of tracking their ratings (like we already do). If anyone wants to write up a new proposal for rating articles that isn't vastly more complicated than the present one, they should do that.

:::Additionally, I will flat out veto any fresh start migration proposal that does not include a 2+ week transfer period. So time to update legacies will be there, as well as an easy way to determine what needed updating, for any who would take advantage of it. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  06:05, 9 May 2012 (UTC)

::::Yeah, really, blank slate is easiest. It's not like we won't have a rating flurry once the system goes in place. Also it looks like more or less everyone is on board with a 5-scale (whether it's -2 to +2 or 1 to 5 is an aesthetic choice and can be decided in the final stages). The big thing that we're still missing to make the proposal complete is thresholds. But it occurs to me that we actually can't determine what a reasonable threshold would be without first getting a good number of pages rated -- what if we set the Fave threshold to 4.0 (with however many raters), but then we get a fuckton of articles in the 4-4.5 range simply because of the way people vote? We need to see where the distribution falls before drawing lines on it. So what do you think of the idea of implementing it for a trial run without thresholds? We'd need a page explaining (as guidelines rather than rules) what we want people to consider in rating, and that's about it. After it's in place for a while, we look at where the numbers fall and get a sense of where we should set the thresholds. We could just use the current list of community favorites in the meantime. --[[User:DanielDraco|DanielDraco]] 18:38, 9 May 2012 (UTC)

:::::I'd rather define a metric for the rating (and thus for the CommFavs) and ask people to work from that than try to deal with determining thresholds based on voting patterns that are likely to change over time in the absence of a guideline. Telling people that "a 1 is for things you really hate, a 3 is for things you are pretty neutral on but would probably allow if someone wanted to play it, and a 5 is for things that you love and would run in your games right now" doesn't seem like a stretch. It also allows us to just set the CommFave line at 4.75 average and above (assuming ratings of 1-5) and be done with it. That's just setting it such that, on average, 3/4 of the people who rated it liked it as much as they could and 1/4 liked it just slightly less than that. And that sort of top end community backing seems like the sort of thing we'd want for an actual CommFave. Setting it at 4 or so just means that most people liked, and doesn't really suggest an actual favorite to me in any sort of arrangement where granularity is supposed to matter. If you want to argue that we'd still get a ton of articles in that range anyway or that people aren't going to want to rate on that sort of scale, that seems like an argument for either more granularity (0 to 10 maybe) or a still higher average, not an argument for a potentially flexible requirement. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  03:43, 10 May 2012 (UTC)

::::::Yeah, I guess that works. And yeah, if a requirement as high as 4.75 yields too many Faves, that would probably be a reason to increase granularity. The greater concern with that threshold, I think, would be too few Faves -- in which case it would need to be lowered. But it's not that big a deal to tweak it later. It was just a thought. --[[User:DanielDraco|DanielDraco]] 04:55, 10 May 2012 (UTC)

:::::::Tarkis asked me to chime in here. I really, really like the current like/dislike system (though maybe it shouldn't show up in the author template with so few ratings, we don't really have enough people rating constantly to do anything else). I think it's what we should have had from the very beginning. It's simple enough that anyone who shows up can understand it and say they like or dislike something. It gives us a good pool to put articles on the front page. It does exactly what it needs to, and no more. [[User:Surgo|Surgo]] 03:21, 17 June 2012 (UTC)

::::::::Yes, but people generally understand a 1-5 system pretty well too. There are two issues that I see with a binary system. Firstly, it depends on a relatively large number of votes to give any helpful information. We don't get many votes. Secondly, it is more like Rotten Tomatoes, where we might prefer Metacritic -- that is, instead of saying how good most people think it is, all it can tell us is how likely someone is to "like" it to any degree at all. It might be a masterpiece of homebrew, or it might be okay -- the vote will be counted the same, and the users will not get information to distinguish between them. --[[User:DanielDraco|DanielDraco]] 03:26, 17 June 2012 (UTC)

:::::::::I'm not looking to give ''that'' much information. I think "Most users like this article" is plenty of information. They can then read the article and see if they agree. A 1-5 system will suffer from the same issue of needing a large number of votes. [[User:Surgo|Surgo]] 03:27, 17 June 2012 (UTC)

::::::::::Well, not really. If a user sees that one person likes an article, that tells them next to nothing. They learn the vaguest possible notion of one person's opinion. If a user sees that one person gave an article 5/5, they at least know just how impressed that one person was. It provides a little more information in a way that is equally succinct and equally easy to understand -- I don't see any downside whatsoever to changing to a 1-5 system, and I can certainly see several upsides. --[[User:DanielDraco|DanielDraco]] 19:17, 17 June 2012 (UTC)

::::::::::And also, even if we were to stick with like/dislike, we ''certainly'' need to render the information better than "most users like this article" or "most users dislike this article". If the range of people's possible reactions were to be numerically represented in a perfectly accurate manner, it would be a continuum. We're lowering that resolution down to a binary system. Which is alright, since it gives us some different data to work with -- a Rotten Tomatoes system rather than a Metacritic system, to go back to my earlier comparison. But it does end up simplifying the information and losing data fidelity. So then when we take the trends we find in that simplified data and lower ''that'' resolution into ''another'' binary system, we've lost even more data fidelity. It's worse than giving a measurement with only one significant digit, because even a single-significant-digit datum in a base-10 number system gives FAR more information than we provide. We're doing an unforgivable amount of rounding, and there's no good reason for it. When the statements "50.01% of users thought this article was pretty okay" and "100% of users thought this article was better than sex" are rendered as equivalent outcomes, there are some really big issues at work with how data is being displayed. --[[User:DanielDraco|DanielDraco]] 19:27, 17 June 2012 (UTC)
{{Discussion Indentation Revert}}
:I'm not opposed to listing actual numbers of likes/dislikes, but I am 100% opposed to ever moving back to a scale. We tried that multiple times, and it never worked. [[User:Surgo|Surgo]] 19:58, 17 June 2012 (UTC)

::Opposed such that you'd vote against it, or opposed such that you'd veto it Surgo?

::And since I've figured out how to do more stuff with lists of names, showing totals of people who like, dislike, and neither (which probaly needs another name) an article along with a rating based on the average is pretty easy and would probably simplify the rating process slightly. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:13, 17 June 2012 (UTC)

:::I'm not Green Dragon, I don't do nonsense bullshit vetos against the community. But I will '''very strongly''' speak against it. We have tried it a number of times, even going all the way back to the paleowiki, and ''every single time'' it has completely failed at every objective. [[User:Surgo|Surgo]] 22:55, 17 June 2012 (UTC)

::::Fair enough. 

::::I'm pretty neutral on a larger scale or a not larger scale, but if people want to display more information or have different qualifications for ratings categories, we can totally do that. Here, I'll even write up a proposal for it:
::::*A rater adds their name to one of three parameters within the Author template: |liked by=X, |disliked by=X, or |<whatevered> by=X. There are no longer any numbers to update.
::::*These names are turned into numbers and used to determine an average rating. For articles with less than 4 ratings, the total is still divided by 4 so that we get nice numbers for the following ratings categories.
::::*Community favorites are articles with an avg rating of 0.8 or above. This would correspond to 4 likes (due to the minimum divisor), 4 likes and a pass, or 9 likes and a dislike. I suspect the latter one will be hard to reach, but it's possible to do. It's also harder to overcome than our current setup.
::::*Generally liked articles are those with an avg rating between 0.4 and 0.799. This would correspond to 2 likes, 3 likes and a pass (or a dislike), or 5 likes and 2 dislikes.
::::*Mixed or insufficiently reviewed articles would be rated between -0.399 and 0.399. Since generally liked would start at 2, the only time insufficiently reviewed would happen would be with 1 rating only, but it could be split out if we wanted.
::::*Generally disliked and Community opposed articles would have negative numbers ranges equivalent to their liked opposites.
::::*Numbers of likes, dislikes, and neutral or whatever are shown in the ratings data, along with the general ratings category. Average rating is not.
::::*Ratings data is displayed in a more obvious area withing the author template, and changes color with the rating category. This section replaces the Homebrew notification banner.
::::Aside from visual differences, the only real change in the above is the move to average instead of net for ratings categories. It puts more weight on dissenting views, which I think is probably a feature rather than a bug. It also broadens the central neutral category quite a bit, which is something I want to do anyway.

::::This would still require a lot of manual updating of rated pages, but would simplify future ratings additions by removing the current totaling step. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  00:05, 18 June 2012 (UTC)

::::: Regardless of how we do it, one thing that I feel adamant about is the need for more granularity--a 0 to 4 rating system would well. The reason for this is because time and again I find an article that I think is decent but not amazing, and have no way to differentiate from articles that are amazingly good. IMO this is a ''big'' problem, and regardless of how we do it, we need more granularity in the rating system. --[[User:Ghostwheel|Ghostwheel]] 03:58, 18 June 2012 (UTC)

Frankly, I think we need to address the bigger elephant in the room - more than half the damn articles on this wiki are not rated ''at all''. Whatever granularity we do or don't have makes no difference if ''nobody is fucking using it'', which is increasingly what I'm seeing as time goes by. This system only works if everyone rates - and currently, the number of people actually rating anything on this wiki seems to seriously number in the single digits.

Thus, before we screw around with any granularity changes, I'd like to repeat something I proposed to Tarkis a while ago - make a really ugly banner for unrated articles that either goes away or becomes less ugly when these articles are rated. This would incentivize people to actually get ratings for their articles, and if we use those banners to hold rating information, it would both become more visible ''and'' make people care about it more. 

Thus, until this is addressed in a satisfactory manner, I'm opposed to ''all'' changes to the rating system.  Full fucking stop. - [[User:MisterSinister|MisterSinister]] 04:42, 18 June 2012 (UTC)

: Maybe some sort of "credit" system, where you can create something if you've rated 3 articles for every one you create? Not perfect (and is very bad--don't do it, heh) but any other ideas for promoting rating? --[[User:Ghostwheel|Ghostwheel]] 06:48, 18 June 2012 (UTC)

::Ghost has stated that he specifically does not want to rate things without greater granularity. His contention is that with greater granularity there is a lower barrier to liking something, and people (including himself) would rate more. I think.

::And I should have put "unrated" and "uncommented" in the above proposal, since I want those as separate rating states on the ratings information box. And I want them to be ugly, the "uncommented" slightly more than the "unrated". - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  07:15, 18 June 2012 (UTC)

::(Further comments that were here before MS pulled his) I agree with encouraging ratings, but we need to do it concurrently with any ratings change. If we wait to get a big pile of ratings before considering any changes, there will be sufficient conversion overhead as to make a change extremely difficult. And one of our migration plans is to burn them all down after all. Does anyone really want to get a bunch of ratings and then get rid of them or ask people to do it again? So while I'm off making a table addition to annoy people into rating and commenting on things (tomorrow, it's late), let's carry on with discussions of what to populate it with afterwards. Or not, I really don't care. I just had this conversation a couple of years ago and it didn't go anywhere then either. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  08:28, 18 June 2012 (UTC)

:::I saw someone suggesting a Metacritic form of rating... It'd be much, much easier if there was a way to see how people on other popular tabletop gaming forums see the homebrew, then draw that as a form of "critic rating". Of course, it wouldn't work given the idea of the posters on each forum (WotC no longer having the same people as before and mostly focusing on 4e/Essentials/5e, Paizo oogling it to Pathfinder, GITP being just too darn nice, Min-Max Boards being a sort of "control board" and The Gaming Den being the meat-grinder), unless you put a disclaimer indicating how each forum tends to behave regarding homebrew. That's the closest thing I can see to Metacritic as possible (and to Rotten Tomatoes, as well).<br>The Wiki definitely needs some serious cross-promotion, however, for people to check it out. That should help, at least, to promote the Wiki beyond the alternative (which I'll refuse to mention on grounds of being the complete opposite of this Wiki). - [[User:T.G. Oskar|T.G. Oskar]] 09:28, 18 June 2012 (UTC)

::::Tarkis, is there any particular reason for displaying the rating average in terms of categories, rather than just giving a number? If it's on a familiar scale (likely percentile if we stick with a binary system), then people will certainly understand it and be able to draw more information from it. Another thing I'd like to note about your proposal is that it might be better to calculate all averages faithfully, and have a "minimum votes required" criterion attached to Community Favorite status. That would distort the data less, and have the same end result.

::::But I'm going to have to continue agreeing with GW on greater granularity, and insist along with him that it is the better basis for a rating system. If we have a binary system, all we have is an indication of how likely someone is to like an article ''at all''. We provide no information whatsoever about ''how much'' people like it -- which I think is far more valuable information, and which I think more directly correlates to overall quality. Toy Story 2 has 100% on Rotten Tomatoes. But few people would name it among the greatest movies ever made. It's just broadly agreed that it's quite good. (And Oskar, ''that'' is what I meant by comparing Metacritic and Rotten Tomatoes. Rotten Tomatoes works off a binary system of "do you like it or do you not", whereas Metacritic works off a normalized average of "how much do you like it". I don't think drawing a consensus from a forum would be feasible, and I don't think giving them any sort of implicitly elevated prestige as critics would be constructive.)

::::As for cutting out this conversation "full fucking stop," I agree with TF. Encouraging votes under a system likely to change is a waste of time and effort. We need to fix either this first and that second, or both of them simultaneously. Starting a new discussion heading for that issue might be productive. --[[User:DanielDraco|DanielDraco]] 17:46, 18 June 2012 (UTC)

:::::Any averages we do without 1) a large pool of raters or 2) an extremely granular ratings system are going to be very swingy. An article with 1 like / 1 dislike has a percent average of 50% and an article with 2 likes / 1 dislike has a percent average of 66%. An article with 1 like / 1 neither would have an average of 75%, while one with 2 likes has a completely boring 100%. These are not uncommon occurrences in rating, and the jumps between them suggest we don't need the granularity you want because we're not using it. Further, all of those numbers look really of-putting to someone who is used to percentage ratings. I don't want to help people pre-judge an article based on information that is too granular for its inputs. Even moving to a 5 point scale instead of the current 3 point doesn't resolve that, it just makes the gaps between jumps slightly smaller. It can work for metacritic because they have a more granular set of inputs and a greater number of them, but it's a bad fit here. Using granularity on the order of our ratings granularity doesn't have the same problems because we control the break points. The categories serve the same role as the percentages at metacritic, it's a thing you can look at to get a sense of the overall rating without being misled. Yes it's less information, but the information you want to give out is misleading or destructive so there's no reason to do it.

:::::Also metacritic is a terrible example for what you want. It leans high and uses inputs that we don't. If anything, Amazon or Netflix with their 1-5 star ratings would be what you want to focus on, because that's what you're actually proposing. How useful do you feel their averages are when they have 1-2 ratings

:::::As to calculating averages faithfully, not doing that was a conscious decision to deal with very small data sets. 2 perfect scores should not be reflected on the page in the same way as 4 or more perfect scores, not if we're trying to give people something simple to look at to understand how well the article is regarded. We could add a "based on X ratings" to the bottom of each score or just list the number of ratings or each type a page has received, but I see that as less helpful and requiring more work on the users part than just obscuring small data sets from the start and using categories. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  19:58, 18 June 2012 (UTC)

::::::I really don't see the link between the swings present in a binary system and ''anything'' in a more granular system. Binary systems are ''far more'' swingy, because every vote is ''always'' at one extreme or the other. The jumps that happen with each vote suggest nothing except the fact that the span separating any two dissenting votes cannot possibly be greater. If one voter disagrees with another voter, the difference between them is always 100%. That will cause big swings from slight disagreements. More granular systems allow for less distance. If two users vote and disagree with each other, the odds are that the span between their opinions will be less than 100% of the entire range of options -- so it will, in most cases, jump less. The only way a more granular system can be as swingy as a binary system is if every user votes either the minimum or the maximum, with nobody in between -- a self-imposed binary. But they won't do that. There will be 2/5s and 4/5s. It will sway more gently, and zero in on a central point far more quickly.

::::::And TF, frankly, I have to say that I don't tihnk "all of those numbers look really of-putting to someone who is used to percentage ratings" is a valid argument for what you're proposing. The numbers look bad, so we lie about them and say that they're other numbers? That's not serving the user at all. If people don't understand the notion of a sample size, a better accomodation would be to just not display any rating at all until a certain threshold is reached. That way we're not misrepresenting the data, but we're still avoiding the pitfall of less math-minded users thinking that one person's rating of 0/5 is the gospel truth. I do understand wanting to avoid publishing insufficient data. But the way to do that is ''not publish insufficient data''. Distorting the data in any way does nothing but make it even less relevant. If it's not worth showing, don't show it. In fact, I would go so far as to propose this, as an element to whatever system we use with whatever level of granularity:
::::::* An article will not display voting data or an abstraction thereof until it has received 4 votes. Instead, it will display a note to the effect that more votes are needed. (The exact nature of this note and the prominence of it will be decided in the discussion below.)
::::::* An article is ineligible for rating-based categories such as "Community Favorite" until it has received 4 votes.

::::::For the movie site analogy, I guess Metacritic isn't a great example. I just chose it as a counterpoint to Rotten Tomatoes, because they ultimately use the same data in different ways to achieve different results, and Rotten Tomatoes' way of handling the data is much like our current system. The pitfalls of RT are really the greater part of that analogy, anyway. But sure, let's use Netflix as the other side of the coin if that comparison is used again. --[[User:DanielDraco|DanielDraco]] 18:11, 19 June 2012 (UTC)

:::::::The argument was not against a more granular system, which I don't actually care about. It was against a displayed rating average that was more granular than it's inputs. Moving to a 1-5 rating makes it better, as I noted, but it does not resolve the fact that a straight percentage system is full of data points that we are unlikely to ever use. That those percentages also come full of preconceptions and biases is just more reason to not use them. I'm less arguing against a 1-5, rounded to nearest .1 or .2, shown total than I am straight percentages, because that is a granularity closer to the scale we're actually using. But in no way is a derived shown value like I'm proposing misrepresenting data. I'm not giving as much as you want, because as much as you want is both useless and stupid given voting realities here, but I'm not saying untrue things with it or planning on hiding the way it's generated.

:::::::As to the rest, that's crap. You're actually proposing that instead of just dealing with small numbers of ratings in such a way that they don't over or under sell an article, we just don't display them at all. No rating at all on an article with less than 4? That's most things on here. That straight up doesn't work without major changes in rating behavior, and doesn't even look like an honest attempt to deal with the realities of rating on this site.

:::::::And as nothing new is being added at this point but bickering, I'm going to end it here and just write up a set of voting options. Granularity: More/Same. Rating:Category based on average, Average, or Numbers of each. If average: Weighted (minimum divisor), Minimum Required, or Straight. We have some things to decide on. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:13, 19 June 2012 (UTC)

::::::::Edit conflicted with Tarkis with this. Swinginess of a system average doesn't mean shit if we aren't displaying an average it in the first place, which is what I thought the case was with like/dislike, where the average is used ''only'' to calculate community favorite and not actually displayed in the author template (that display is likes/dislikes only). [[User:Surgo|Surgo]] 20:15, 19 June 2012 (UTC)

== No One Using Rating ==

Just bringing this up under a separate heading so it doesn't get lost among the noise. Nobody rates because there's no easy one-click solution to do so. I don't know if I'll have the time to program one. Maybe someone can find a canned solution on the mediawiki extension listings or something. If so I'll check it out. Basically such a solution should let the user click like/dislike, say why, then as its output drop exactly that onto the Talk page just as is done now. No difference in what the ratings are, just how they're inputted. [[User:Surgo|Surgo]] 20:31, 18 June 2012 (UTC)

:It would also need to update the author template of the article page, or set a property on the article page that could be referenced. Otherwise we don't get the visibility that we want. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:38, 18 June 2012 (UTC)

::How difficult would it be to program this kind of functionality? I'm done with finals now, and thus, I could probably do it if someone told me what I'd need to learn in order to make it happen. - [[User:MisterSinister|MisterSinister]] 20:38, 18 June 2012 (UTC)

:::There are some extensions that handle the numerical voting bits, some complete and some experimental (see this [http://www.mediawiki.org/wiki/Category:Rating_extensions useful category page]), but only 1 comes with an attached comments box ([http://www.mediawiki.org/wiki/Extension:Rating this one]). None of them really work for what we want though. So it looks like we'd need to code from scratch or sacrifice some desired functionality. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:51, 18 June 2012 (UTC)

:::Actually, it doesn't need to edit the author template specifically. We could do a new template for the page called <nowiki>{{Raters}}</nowiki> or whatever that set properties, and the author template could just look them up in the page's metadata. Or if there's a way to do a direct sql query from a template (which I think there is, though I'm not sure how), we could skip editing the author page entirely and have the ratings code just drop the lists of raters in a table somewhere and query that. We just need to be able to update such a table when ratings are changed / removed. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:16, 18 June 2012 (UTC)

::::Difficulty depends on how familiar you are with PHP. You'd need to learn the MediaWiki codebase. I highly doubt you can make a SQL call from a simple template (it would be crazy easy to DDOS the site let alone all the security holes you could imagine), but you can certainly do it from a template that calls a function where the function itself is implemented in PHP code. EG <nowiki>{{#if: }}</nowiki>, which doesn't use SQL but you get the idea. [[User:Surgo|Surgo]] 21:45, 18 June 2012 (UTC)

:::::Since I wrote that I remembered you can query the properties of other pages and list them out (and then do string tricks on them). So if we turned on smw in the talk space, we can set ratings with the rating template. And then query those values to do stuff on the article page. So if we wanted to simplify things without worrying about writing a form equivalent to append things to a page, we can reduce it to a single template addition on talk (with some backend changes). - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  22:00, 18 June 2012 (UTC)

::::::Ok, so SMW in talk is turned on now and the author template has been updated to make use of it. All that is needed to rate something is to put the template up on the article's talk page. We probably don't need to worry about any extensions at this point, though something like a preload that allowed you to append a pre-formatted block of text to an existing section in a page would be nice (since the preload can only create new pages or new sections at present). - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  18:44, 19 June 2012 (UTC)

:::::::If we want to actually get a volume of ratings, we absolutely do need a one-click system for doing so. [[User:Surgo|Surgo]] 20:11, 19 June 2012 (UTC)

== Voting ==

Another wiki vote. Yay.

'''Rules:'''
*Add your signature to vote.
*We're doing instant run-off in 2 categories: Granularity and Rating Display. You get 1 vote for your first choice in each category, and 1 vote for your second choice in each category.
**In case it's unclear what instant run-off means, it means we look at 1st picks first. If one of them has a 50% majority, it wins. If none of them do, we eliminate the worst performing option and apply the second choice votes from the users that picked the dead option first.
*Keep comments out of the voting lines for legibility.
*Voting runs for at least 2 weeks.

'''Voting is now closed, results follow.'''

===Granularity===

====Current Like/Neither/Dislike====
'''First Choice'''
* [[User:MisterSinister|MisterSinister]] 21:06, 19 June 2012 (UTC)
* [[User:Surgo|Surgo]] 23:34, 19 June 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 01:52, 3 July 2012 (UTC)

'''Second Choice'''
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:52, 19 June 2012 (UTC)
*[[User:Foxwarrior|Foxwarrior]] 21:32, 19 June 2012 (UTC)

====1 to 5====
'''First Choice'''
*[[User:Ghostwheel|Ghostwheel]] 21:01, 19 June 2012 (UTC)
*[[User:Foxwarrior|Foxwarrior]] 21:32, 19 June 2012 (UTC)
*[[User:DanielDraco|DanielDraco]] 01:01, 20 June 2012 (UTC)
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:52, 19 June 2012 (UTC)
* [[User:Franken Kesey|Franken Kesey]] 02:21, 20 June 2012 (UTC)

'''Second Choice'''
* [[User:T.G. Oskar|T.G. Oskar]] 22:06, 20 June 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 01:52, 3 July 2012 (UTC)

====0 to 10====
'''First Choice'''
* [[User:T.G. Oskar|T.G. Oskar]] 22:06, 20 June 2012 (UTC)

'''Second Choice'''
*[[User:Ghostwheel|Ghostwheel]] 21:01, 19 June 2012 (UTC)
*[[User:DanielDraco|DanielDraco]] 01:01, 20 June 2012 (UTC)

===Rating Display===

====Derived Category====
Ex: Most users liked this article / No clear feelings are obvious for this article / Most users disliked this article<br />''Note:'' If derived wins out, there will be a followup discussion to determine the categories and placement requirements. Holding off for now, because it's more voting that might not even matter.

'''First Choice'''
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:52, 19 June 2012 (UTC)
* [[User:MisterSinister|MisterSinister]] 21:06, 19 June 2012 (UTC)

'''Second Choice'''
*[[User:Foxwarrior|Foxwarrior]] 21:32, 19 June 2012 (UTC)
* [[User:Surgo|Surgo]] 23:34, 19 June 2012 (UTC)

====Average====
Ex: 75%, 3.2<br />''Note:'' If average wins out, there will be a followup vote to determine the version displayed and the method of calculation. Holding off for now, because it's more voting that might not even matter.

'''First Choice'''
*[[User:DanielDraco|DanielDraco]] 01:01, 20 June 2012 (UTC)

'''Second Choice'''
* [[User:Ghostwheel|Ghostwheel]] 21:04, 19 June 2012 (UTC)
* [[User:Franken Kesey|Franken Kesey]] 02:21, 20 June 2012 (UTC)
* [[User:T.G. Oskar|T.G. Oskar]] 22:06, 20 June 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 01:52, 3 July 2012 (UTC)

====Straight Numbers====
Ex: 2 Likes, 1 Dislike; 1 user rated this a 4, 1 user rated this a 5

'''First Choice'''
* [[User:Foxwarrior|Foxwarrior]] 21:32, 19 June 2012 (UTC)
* [[User:Ghostwheel|Ghostwheel]] 21:04, 19 June 2012 (UTC)
* [[User:Surgo|Surgo]] 23:34, 19 June 2012 (UTC)
* [[User:Franken Kesey|Franken Kesey]] 02:21, 20 June 2012 (UTC)
* [[User:T.G. Oskar|T.G. Oskar]] 22:06, 20 June 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 01:52, 3 July 2012 (UTC)

'''Second Choice'''
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:52, 19 June 2012 (UTC)
*[[User:DanielDraco|DanielDraco]] 01:01, 20 June 2012 (UTC)

===Comments and whatever===
Categorization is my preferred method, but if it or Average wins I would want to display the straight numbers in a spoilered box at the bottom of the ratings display, to make it easier to see how the numbers fall out at a glance. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:52, 19 June 2012 (UTC)

:When we only get 1 to 5 votes, we could even potentially show ''who'' likes and dislikes the page. --[[User:Foxwarrior|Foxwarrior]] 21:32, 19 June 2012 (UTC)

:: Just reiterating: we tried scales on multiple occasions, at no point did it ever work. I don't see it working now either. [[User:Surgo|Surgo]] 23:34, 19 June 2012 (UTC)

::: It never failed for that reason, though. On the paleowiki, it failed because it was poorly moderated and it had a really awful voting categories thing. In the RC days, it failed because those of us in the committee weren't as active as we had anticipated. Not once has a scale system failed ''because'' it was a scale system.--[[User:DanielDraco|DanielDraco]] 01:01, 20 June 2012 (UTC)

:::: I'm with DD on the scale--we *need* some granularity to differentiate between "yeah, this is alright," and "I FREAKING LOVE THIS!" (and vice versa for disliked stuff). I'm still not completely closed on the rating display stuff, but [http://pastebin.com/YJx5QGcD this] might sway some people one way or the other. --[[User:Ghostwheel|Ghostwheel]] 01:36, 20 June 2012 (UTC)

:::::And on that note, I'd like to reiterate my point about the display, at the risk of belaboring it. I was very rambly when I was making it before. Hopefully I'll make more sense this time.

:::::When you take the raw data -- like, for example, "67% of users like this article" -- and simplify it into "most users like this article", you are displaying less information. Now, you might think this is good, because it encourages people to dig more deeply into the information. But it also encourages them to say "fuck it" and just stop using the wiki, because what a lot of users are after is a simple, easy-to-use resource for quality homebrew. I doubt very many of us came to this site and immediately focused on the community. We had to be drawn in by the site itself first. Easy reference is a good attractor. A helpful summary of an article's quality will draw in users. But that summary ''does'' need to be actually helpful. If the difference between 100% favor and 0% favor is the same as the difference between 51% favor and 49% favor, that is NOT a helpful summary. It provides next to zero information. Users will find more utility in a more informative display.

:::::Also, a note about the linked conversation. TF raises a good point, namely that a percentage display gives far more granularity than we will ever use, the negative effect of which is that it might imply that we actually are ''utilizing'' that granularity. Which is a fair point, and must be considered if we stick with like/dislike. But the thing to remember is that it is a point against percentages, but that does ''not'' make it a point ''for'' "most people like/dislike". I think this issue with the percentage system is much, much smaller than the "0% == 49% and 51% == 100%" issue of the threshold system. Besides, the issues of the percentage system pretty much go away if you put it side-by-side with the straight numbers. --[[User:DanielDraco|DanielDraco]] 02:17, 20 June 2012 (UTC)

::::::''When you take the raw data -- like, for example, "67% of users like this article" -- and simplify it into "most users like this article", you are displaying less information.''

::::::Err, as far as I am aware we were not planning on doing that, but rather saying "N users like this article, M dislike it, K neither like nor dislike." [[User:Surgo|Surgo]] 02:20, 20 June 2012 (UTC)

:::::::Umm, duh? I told Ghost I wasn't arguing against greater granularity in ratings (and have just swapped my votes). If he left that out, you can be annoyed at him for giving insufficient context. It takes >25 ratings to be able differentiate between 75% and 74% in a 1-5 system, and >50 ratings to do the same in a dislike/like system. Arguing for percentages instead of an average (weighted or not) is suggesting that we will someday get to that point. I am already laughing in your face if you intend to seriously make that claim.

:::::::I ''am'' arguing for ratings display granularity that is not an order of magnitude greater than the ratings granularity, one that we will actually use within 5 ratings. Because that's what we are likely to get. A 5 qualified state setup that is average driven satisfies that, as does a straightforward average. Since I also want to cut the crap and give the short version of what the average actually means, because I think it's easier for users, I prefer the 5 state setup over any form of average. At worst, it's cutting the Y off of a X.Y style average.

:::::::Surgo - displaying the average is one of the display options. If you want the straight numbers of ratings, you should swap your votes around. Straight numbers should be your first vote, not your second. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:34, 20 June 2012 (UTC)

::::::::Surgo: The "most users like this article" thing is [[Aging_Overhaul_(3.5e_Variant_Rule)|what we're doing now]], and is one of the display options above. In fact, you voted for it.

::::::::TF: I'm not really trying to target my criticism of binary ratings at you. It's for whoever still supports it. Which, as it's turning out, looks to be relatively few people, so I may have wasted my effort :P

::::::::And arguing for percentages is ''not'' equivalent to an argument that we will reach a point where a vote can achieve a change of 1%. I agree with you that display granularity should be very near the actual granularity of the data, but another necessary consideration is readability. On a 1-5 scale, that's easy: show the average. It's trickier on a like/dislike system. You can use fractions, but those are hard to compare. Percentages are easier to compare, and to grok the value of.

::::::::But that's hypothetical. Speaking practically, it seems relatively certain that 1-5 will win, so all the talk of percentages is rather wasted. For a 1-5 system, an average of some sort is, I think most people would agree, the best way to go. But is there any real reason to summarily forbid decimal results? I think we should allow one decimal place, at least. With as few as 4 votes, a single point of change can achieve a variance of 0.25. It's not 0.1, but it does still mean that X.Y is closer to the true granularity than just X is.

::::::::Another thing. Would you really consider an average rounded to a certain digit to fall under the Derived Category heading? I would call that an Average still. I see the logic, but Average seems a better fit to me. --[[User:DanielDraco|DanielDraco]] 05:50, 20 June 2012 (UTC)

:The current setup is technically trinary, but whatever. I prefer it over the 0-10 one because there is such a thing as too much granularity in a rating, and I think it worse than not enough. And no, if we displayed the number that would clearly be an average. It's even listed in the average section as an example. But if we did something with that number, like tell people what it meant straight out instead of asking them to understand the nuances of our ratings scale first, then it would be a derived category. The point was that if you take the average, round it to cut off the decimal, and then give a textual explanation of that value instead of the value itself, then you have a Derived Category. Since any average you take will be closer to one of those than anything else, the only data errors you bring into it are rounding errors unless you bend your scale slightly (such as proposed for community favorites). You may recall a while back when we were talking about community favorites and a 1-5 scale. If 1-5 wins, the discussed hate / dislike / neutral / like / love scale is going in the directions (if not being the actual scale) so that the community favorite threshold makes sense (as previously argued, and which you agreed worked). And if that's what the numbers mean, then displaying that value instead of the points average is a simplification for users because it cuts out the lookup step.

:Now I also want to deal with small numbers of ratings by not counting them fully until there are enough of them to be representative, and so that we don't mislead users by saying "this article is generally loved (by 1 user)", but that's dealing with a different aspect of the ratings problem. It is introducing data errors, but since I also want to include the raw data in a spoiler within the ratings listing it's not something that I see as problematic in the slightest. It is less misleading than a straight average with too few data points and more useful than not giving an average at all until it could possible be a community favorite. It is not a perfect solution, but it is a better one than the proposed alternatives. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  07:59, 20 June 2012 (UTC)

::I only like the second Granularity, 1 to 5, the 0 through 10 option is too many. (1-7 sounds more reasonable.) Therefore I have only voted for one in granularity. Also, why is the option -2 to 2 not here, it is still viable. --[[User:Franken Kesey|Franken Kesey]] 18:06, 20 June 2012 (UTC)

:::Ah, alright, I understand what you're talking about now TF. Carefully constructed, I could see a system like that working well. I think it's better to simply give the average, but at this point I don't see any productive way of arguing for or against either, because ultimately both have the potential to be good systems, and both have relatively minor advantages and disadvantages as compared to each other.
:::-2 to +2 is probably left out because it is mathematically equivalent to a 1-5 scale. The only difference is in the voting process -- a detail to be ironed out once these foundational concepts are set in stone. And you might want to reconsider only voting for one. Your secondary vote means that if your first choice loses, your voice is still heard in choosing the lesser of two evils. --[[User:DanielDraco|DanielDraco]] 18:44, 20 June 2012 (UTC)

::::Open question here for the people supporting a display of the bare numbers. How would you feel about putting the average next to it or below it or whatever? It could be formatted or positioned in such a way as to minimize its prominence, but it would help in comparing articles' ratings. No sense in communicating the quality of an article if you can't look at two of them and determine which is better, right? --[[User:DanielDraco|DanielDraco]] 23:51, 24 June 2012 (UTC)

:::::I haven't really commented on here since I don't care one way or another, but since I didn't spot it, I feel I should bring up the real big reason why voting is hard... you need to get the template and not even I have memorized it.  I always copy pasta from another article.  Is there a way to have a button on top which either showed the code or inserted the code directly?  I'm thinking like how most forums you can click a button and it'll insert your hyperlink properly formatted and the like. -- [[User:Eiji-kun|Eiji-kun]] 01:57, 3 July 2012 (UTC)

===Bumps===

Bump. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  17:11, 24 June 2012 (UTC)

: Bump-of-the-fist. --[[User:Ghostwheel|Ghostwheel]] 18:15, 24 June 2012 (UTC)

::Return bump-of-the-fist. 5 days to go. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  04:46, 29 June 2012 (UTC)

:::Interrupt Fisttabump! -- [[User:Eiji-kun|Eiji-kun]] 04:51, 29 June 2012 (UTC)

:::: Thank you for the bumps. I am slow. --[[User:Ganteka Future|Ganteka Future]] 01:52, 3 July 2012 (UTC)

===Results===
With more than 50% of the 9 members voting, we are moving to a 5 point scale and listing straight numbers of ratings on the pages. I'll set up a new vote so that we can choose the form of <s>our destroyer</s> the 5 point scale as well as if we're going to show any information other than straight numbers in a bit. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:34, 7 July 2012 (UTC)

==Next Step in the Rating Change==
It occurs to me that we might not even need another vote at this point, as a brief discussion might just get it done if there's nothing contentious coming up. So I'm going to hold off on it for now. Instead, let's just talk about what our options are and see if we have a consensus already. And if we need a vote afterwards, we'll at least know what we need to vote on.

Even though the winning option was called 1-5, we could just as easily to 0-4 or -2 to +2 or any other number set (though I would argue that anything other than those three is silly). We could also do hate / dislike / indifferent / like / love scale. Or we could do both, since it's not hard to code for it. I have a preference for 0-4 and hate -> love together, but I'm good with any of these really. What I will push hard for regardless is a codification of the hate -> love scale in whatever numeric representation we wind up going with. I mostly want this so that the community favorite qualifier makes sense, but also so that there's some consistency between the values that different people assign to an article.

DD recently requested that we include the average in the display along with the lists of ratings. I'm pretty indifferent to the request, since I have to compute the average anyway for CommFave qualifications. But if it's going on the article with the individual numbers, I don't want it show when there are less than 4 ratings. I don't think an average even matters when there are that few ratings, so I'd rather not show it at all than show something I think misleading. 

Lastly, regarding transitioning from old ratings, we previously discussed phasing / throwing them out and we can still do that. But a 5 point scale opens up other options. The old RC ratings went from 0 to 2, or 2 to 4 on the new scale (assuming 0 to 4) since there was no option for disliking something back then. So we could just map those directly. Similarly, the current -1 to +1 setup could be mapped straightforwardly to a 1 to 3 in the new setup (again, assuming 0 to 4). Now, this means that a lot of those old ratings won't take advantage of the new granularity, and some of the CommFaves get messed up, but it might be worth doing if we're worried about people not updating them before they're purged. I'd still prefer a great purge, but I wanted to put the option out there. And I'd probably keep the current crop of CommFaves in rotation until they received enough votes to requalify (and then remove them if they didn't).

So, to sum up:
*0 to 4 and hate -> love, both in the description of what the ratings mean
*Average shown only after 4 ratings, if at all
*"Purge old ratings" style migration, with current CommFaves being grandfathered in until they officially didn't qualify
Is where I'd go from here. What about the rest of you? - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  06:35, 7 July 2012 (UTC)

:Given that I was for a ratings purge to begin with, I still am now. The rest, whatever - stuff that's happening is already something I fundamentally oppose, and thus, it makes no difference to me. - [[User:MisterSinister|MisterSinister]] 11:03, 7 July 2012 (UTC)

::I like the idea of rendering it as a 0-4 scale. It just makes more sense to me to have the lowest possible rating, which would include "I utterly hate this and see no redeeming qualities," be rendered as zero. Giving an article 1/5 rather than 0/4 can sort of imply that it has earned 20% of the favor one can possibly give. It also means that the central rating will be similarly associated with the "half" value: 2/4.

::I agree that secondary/extrapolated data (i.e. the average) should only be displayed after a threshold, and 4 seems good to me as a threshold. In a nutshell, the reason I think we should have it shown is to make it easier for users to compare articles' relative quality.

::I think it would be a mistake to consider any one rating of a lower-granularity system to be equivalent to any one number in this new system. It adds noise to the data. So yeah, I agree that we should purge. I also agree with keeping CommFaves, just so there isn't a period where we don't have any CommFaves. I'm sure they'll quickly populate with ratings to either affirm or undo their status, anyway, so the artifacts won't last too long.

::In short, I agree with TF on all points. It was bound to happen one day :P --[[User:DanielDraco|DanielDraco]] 22:01, 7 July 2012 (UTC)

:::A wiki’s quality is mostly dictated by the amount of comments and ratings – a thing that this wiki is severely lacking. This is the means by which newcomers can more clearly comprehend if an article is right for their game. To remove the already scarce ratings that we have would further deplete this dilemma. And to expect the original raters to update all their ratings – even if they are otherwise active on this site – has proven in the past a false hope. Instead it may be best to flag all old ratings as older ratings (to be frank, most articles have not been changed since they were rated); and continue pushing the policy: to have the majority of articles rated. --[[User:Franken Kesey|Franken Kesey]] 19:06, 8 July 2012 (UTC)

::::I don't think anyone would advocate actually removing the ratings from the talk pages -- we'd simply not use them in calculations, and we'd alter the template to show that they are of an obsolete form. --[[User:DanielDraco|DanielDraco]] 19:17, 8 July 2012 (UTC)

:::::I hadn't actually thought about actually removing them or not, but "obsoleting them" instead of "purging them" would be a better way to go. So yes, I agree with DD. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  19:36, 8 July 2012 (UTC)

::::::The idea of obsoleting (ie, not using them in calculations) OR purging the ratings makes me so angry it's hard for me to even type this. I just managed to convince a bunch of people to rate stuff and get stuff done, and now it's not even going to matter? Fuck you guys. [[User:Surgo|Surgo]] 23:56, 8 July 2012 (UTC)

:::::::Sorry Surgo, but it seems to be for the best. The possibility of obsoleting ratings has been explicitly on the table through this whole system revamp. It's the reason that when MS suggested that we work on encouraging ratings before we change the system, he was shot down so fast and hard. I, for one, thought everyone was on board with (or at least tolerated) the notion that we want to solidify all these changes, and ''then'' push people to rate more things. --[[User:DanielDraco|DanielDraco]] 00:05, 9 July 2012 (UTC)

Admittedly, I'm with Surgo, but the point is, I was against changing the system from the ''outset''. I consider that, given that the system is changing with (or without) me liking it, a ratings purge is the ''only'' way to proceed. I still disagree with the idea of changing this system, but since the majority seem to be for it, I can only optimize an inherently-unpreferential environment. - [[User:MisterSinister|MisterSinister]] 00:23, 9 July 2012 (UTC)

:Let me make sure I have this straight. The consensuses is to remove the ratings that we have (about 10% of articles have them), then start at zero – because nothing encourages more ratings than knowing that one may have to continue to update their ratings? One should note that three of the nine, who are active on this site, have been vocally opposed to this choice. --[[User:Franken Kesey|Franken Kesey]] 02:18, 9 July 2012 (UTC)

::It's not a consensus apparently. I don't blame Surgo for being pissed at the suggestion. He popped in after the discussion was largely done and didn't catch me cautioning people against rating until this was resolved. As I said above, I don't mind doing a migration plan. There's even a clear way to do it. There's less push on users with old ratings to update them to make use of new granularity, and this may result in a number of current Comm Faves no longer qualifying, but if there is a desire to maintain current volume rather than update to new setup, we can do that. It's not even hard. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:48, 9 July 2012 (UTC)

:::Sounds like a very good solution. I will second your proposal.--[[User:Franken Kesey|Franken Kesey]] 03:51, 9 July 2012 (UTC)

::::Brought it up elsewhere, but make sure there's a way to get the template for rating easily, as atm any time I rated I had to copypasta a previous rating and change accordingly. -- [[User:Eiji-kun|Eiji-kun]] 03:53, 9 July 2012 (UTC)

:::::Aarnott thought of using the preload functionality to do this, and I've tweaked the link enough that it works as a one-click ratings template drop in. It can only append the rating to the bottom of the talk page and you have the option of making a new section for it, which might make the pages look weird in future. We're looking into new stuff that would automatically fill in your user name and multiple preloads so that you could "like" or "hate" something and just have to fill in the reason rather than the other stuff, but it already works for now. The link is in the author box where it talks about the rating of the article and asks you to add one, but that's subject to change if someone has a better idea.

:::::I'm going ahead with the ratings updates under the assumption that the migration plan is "map to new" rather than "obsolete". If we decide to remove them later, we can do it at the time without much extra work on anyone's part (other than mine). - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  19:34, 9 July 2012 (UTC)

::::::No opposition here. I would still advocate obsoleting, but I'm not against mapping to new as such. I do think, though, that we should include at least some small note in the old templates that indicates that they are using an old scale. And I also think we should make an effort to at least gently encourage people to update ratings. --[[User:DanielDraco|DanielDraco]] 19:40, 9 July 2012 (UTC)

== Voting, Round 2 ==

There have been enough stray comments to make it obvious that we probably need to vote out a format for the new 5 point scale. So let's get it done.

'''Rules:'''
*We're doing pseudo-instant run-off for . You get 1 first choice, and 1 second choice. You do not get to rank the options more than that.
*Add your signature to the section with the title indicating the style for which you wish to vote.
*Keep comments out of the voting lines for legibility.
*Voting runs for at least 2 weeks.
*Whatever wins will be what we put into the ratings template (i.e. |rating=X) and will show in the ratings display on the article.
**Note that regardless of what wins, the scale and some description of what they're supposed to mean will be included on the project page. This is to ensure that when people put down the bottom or the top, or near there, that they mean about the same as everyone else. Otherwise "community" favorite and opposed bits don't mean quite the same thing.

====Use 0 to 4 on stuff====
'''First Choice'''
* [[User:Ghostwheel|Ghostwheel]] 02:54, 11 July 2012 (UTC)
* [[User:DanielDraco|DanielDraco]] 22:49, 11 July 2012 (UTC)

'''Second Choice'''
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:51, 11 July 2012 (UTC)
* [[User:Franken Kesey|Franken Kesey]] 03:41, 11 July 2012 (UTC)
* [[User:Aarnott|Aarnott]] 15:27, 11 July 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 17:57, 11 July 2012 (UTC)

====Use 1 to 5 on stuff====
'''First Choice'''
*

'''Second Choice'''
*

====Use -2 to +2 on stuff====
'''First Choice'''
* [[User:Franken Kesey|Franken Kesey]] 03:41, 11 July 2012 (UTC)

'''Second Choice'''
* [[User:Ghostwheel|Ghostwheel]] 02:54, 11 July 2012 (UTC)
* [[User:Surgo|Surgo]] 03:01, 11 July 2012 (UTC)

====Use 5 point word scale on stuff====
The scale would be hate / dislike / neutral / like / love, and would display 0-4 as well for average purposes.

'''First Choice'''
* [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:51, 11 July 2012 (UTC)
* [[User:Surgo|Surgo]] 03:01, 11 July 2012 (UTC)
* [[User:Aarnott|Aarnott]] 15:27, 11 July 2012 (UTC)
* [[User:Ganteka Future|Ganteka Future]] 17:57, 11 July 2012 (UTC)

'''Second Choice'''
*[[User:DanielDraco|DanielDraco]] 22:49, 11 July 2012 (UTC)

===Comments and whatever===
New 5 point scale is currently live and can be set with the word scale listed above (though it's not live on article ratings lists yet, but that's mostly done too). As any of the above versions are trivial to migrate to from where we are, don't let this vote stop you from rating if you want.

The migration plan is not being voted on, because the obsolete option I'm only comfortable with when it's near unanimous. Things have been mapped already, and anything that is an old vote of yours can be found in "Category:Legacy Rating <Your User Name>". RC Favor templates will need to be updated to new Rating templates, and Rating templates just need the |OldRating=True line removed after you've decided whether to update or not. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  02:51, 11 July 2012 (UTC)

: I think using stars (kinda like what Amazon or Newegg does) could be kinda cool... Just need to figure out how to have radio buttons as a voting mechanism. --[[User:Ghostwheel|Ghostwheel]] 02:56, 11 July 2012 (UTC)

::As pics in the rating blocks, this is easy. As "click it on the page to rate it", this is less easy (unless we don't care about hooking into averages with other functionality, like commfaves). And it requires getting over rating reasons, which is still contentious. But stars as the numerical images if we're not using words would be easy, including negative stars or wahtever for some scales. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  04:22, 11 July 2012 (UTC)

:::The pure positive displays state that even a horrific article is still positive. --[[User:Franken Kesey|Franken Kesey]] 09:53, 12 July 2012 (UTC)

::::Are you... positive? -- [[User:Eiji-kun|Eiji-kun]] 11:27, 12 July 2012 (UTC)

:::::When you peruse reviews for a movie, and find that critics consistently gave it 0-1 out of 4 stars, do you interpret that as positive? No, probably not. You probably correctly perceive those as quite negative reviews, because you are aware of the scale's top and bottom. If a user is capable of understanding the basic notion of "0 is the worst and 4 is the best", they're certainly going to understand that numbers closer to 0 than they are to 4 are closer to bad than they are to good. --[[User:DanielDraco|DanielDraco]] 13:50, 12 July 2012 (UTC)

::::Then why would you make the 0-4 scale your second choice? [[User:Surgo|Surgo]] 14:02, 12 July 2012 (UTC)

===Eventual bumps===
There's been enough things going on today, that this could use a *bump*. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  00:15, 13 July 2012 (UTC)

:And another bump. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  04:24, 14 July 2012 (UTC)

::And another. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  00:06, 16 July 2012 (UTC)

:::Voting closes tomorrow, around 3pm PDT. If you wanted in on it, you should do that. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  05:04, 25 July 2012 (UTC)

===Results===
With more than 50% of the voting members selecting it as their first choice, the word scale with numeric interpretations stands. The voting scale will remain "love / like / neutral / dislike / hate". There appear to be no further votes needed on the topic at this time.

The project page will be updated to reflect these changes, as well as the updated Community Favorite / Opposed requirements and policies shortly. Then I plan on archiving most of this page, to clear it for discussions, clarifications, and whatnot on the new bits. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  22:43, 25 July 2012 (UTC)

==Community Favorite and Community Opposed==

Does anyone want to change the current standards of 4+ votes, and average of ≥3.75 for favored and ≤0.25 for opposed? --[[User:Franken Kesey|Franken Kesey]] 20:56, 16 July 2012 (UTC)

:Well, if someone is going to go start the topic, yes, I kind of do. I think >= 3.5 for CommFave and <= 0.5 for CommOpp, still with 4+ ratings, would be a bit better in that we get more CommFaves and can sandbox more things that are really poor. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:09, 16 July 2012 (UTC)

::I like 4+ votes for community favorite. Community Opposed should require more votes -- we don't want to essentially remove something from the wiki based on the opinions of only four people, since putting it into a sandbox (presumably) makes it ineligible for further votes that might have saved it. In that same consideration, there should probably be a grace period -- once an article meets the requirements to be sandboxed, it has some amount of time to allow the author to appeal for more ratings.

::As for the averages, I agree that 3.5 and 0.5 are more reasonable. ''Anyone'' can rate, including, as we must remember, total idiots. Without pointing fingers or suggesting that anyone should, think of someone whose opinion you think is idiotic (we all have at least one, I'm sure). Imagine that that one person gives a truly perfect article a 2/4, because they're a nitpicky jerk or whatever the reason. It would require a realistically unattainable 7 votes of 4/4 to offset that one vote enough to reach 3.75. --[[User:DanielDraco|DanielDraco]] 21:15, 16 July 2012 (UTC)

:::Right now, it's difficult for IPs to rate. They can't use our nifty frontend. This is really unfortunate, because while I wouldn't mind getting IPs to rate, it's not really possible to hook it up to our CAPTCHA backend either, so it would be a source of spam. Also I agree with DD, right now the bands are *way* too tight. [[User:Surgo|Surgo]] 21:28, 16 July 2012 (UTC)

::::I second DD's opinion on all three counts of increasing community opposed requirements, changing averages of favorite and opposed and offering a grace period. Two weeks sounds reasonable – for we give one month for sourcebooks and major books, and a week for delete. In the middle sounds fair. --[[User:Franken Kesey|Franken Kesey]] 21:31, 16 July 2012 (UTC)

:::::Doing different #s of votes for CommOpp would add to the logic in the template, which I'm not really fond of because "exceeded template include size" is a real error that I don't want to get us any closer to than I have to. I would rather add a delay period, say 7 days, when the author could make adjustments or others could pop in to up-rate it before it got sandboxed. And even if it is sandboxed, it's eligible for additional ratings and changes that would entitle it to be moved back into main nav (if only until old ratings have been updated and find the article still sufficiently lacking). It does not have to be a permanent thing.- [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:40, 16 July 2012 (UTC)

::::::I'd like to note that this appears to be a way better mechanism for quality control than past efforts, provided ratings happen a lot. [[User:Surgo|Surgo]] 21:42, 16 July 2012 (UTC)

:::::::I think two weeks is a good delay. In only a week, it may be hard to get the votes needed. But even with a delay, I do still think that increasing the required number of votes is the ideal way to prevent false alarms. If that means we also increase the required number of votes for CommFave, so be it -- it just makes us a little more selective in what we consider the best of the wiki. Although it may be that making the change only to CommOpp would not make the template too big; the way I see it, an oversized template is only a problem when it actually presents itself as a problem. Let's cross that bridge when we come to it, instead of refusing to walk further down the road for fear that we might reach a bridge. --[[User:DanielDraco|DanielDraco]] 00:14, 17 July 2012 (UTC)

::::::::Let's say that an article is sitting at 0.5 avg and 4 votes, either because it has 2 hates and 2 dislikes or 3 hates and a neutral. If such an article gets another rating that is not another hate, it's out of the pile. Dislike, neutral, or something actually positive and it's safe. If an article has 3 hates and a dislike, it needs a single neutral or better to not be moved or 2 dislikes. If an article has 4 hates it needs a single like or better, or 2 neutrals. These are not large numbers of votes we're talking about. If you can't get a single person to legitimately rate your class well, can't successfully dispute a bullshit rating, or make an effort to address criticisms raised in the ratings (and an effort on its own should be cause to not move for a while), then you're either not trying very hard or it's an actually poor article that we don't want on the wiki. And even if you don't get to it, we have mechanisms to put it back in main and get it re-rated. We don't need to set the bar higher than 4, and doing so would just make it harder to control quality like we keep claiming we want to do. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  01:44, 17 July 2012 (UTC)

{{Discussion Indentation Revert}}

:You know what seems like a misfeature for the current setup? That it's possible to knock an article out of being a Community Favorite by "liking" it. [[User:Surgo|Surgo]] 19:00, 17 July 2012 (UTC)

::I don't think so. There are things on here that are good, but not great. And if enough people came in and said "I like it, but it's not great and it's not one of my favorites", then that should push an article down. It's the same as if it had received the ratings in a different order and never qualified in the first place. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  19:27, 17 July 2012 (UTC)

::[Edit] The possibility of pushing articles out of these regions is a decent argument for waiting a week or two to move them or add them to rotation though. That should help prevent an article from being added to rotation, just to be taken down again with extra votes. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  19:33, 17 July 2012 (UTC)

::[Edit Again] Alternately, we could drop the average requirement entirely and just go with a net positive ratings setup like we had before. We could double count loves and hates in such a setup even. Then more people liking something doesn't make it less a favorite and more people disliking something doesn't make it less an opposed. The categories stop meaning "highest rated" and just "generally well liked", but that's a workable change if it's what people want. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  20:15, 17 July 2012 (UTC)

:::I agree with your pre-edit sentiments. The CommFaves should be the best of the best, not simply things which are passably good. Besides, your second edit proposal is mathematically equivalent to just changing the requirement to an average of 2 or higher. --[[User:DanielDraco|DanielDraco]] 21:09, 17 July 2012 (UTC)

::::I lacked detail above, but was I was actually referring to would be something like 2*love + like - neutral - dislike - 2*hate >= 6 and 4+ ratings. Which is not equal to any particular average number, and in fact supports articles with an average between 2.46 and 2.85 as being maybe a favorite. In short, it's a bit weird but addresses Surgo's concern. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:32, 17 July 2012 (UTC)

:::::Ah, yeah, subtracting neutral does make it a little different. But still, it seems better to be able to say something is ''good'' without launching it towards an undeserved CommFave. --[[User:DanielDraco|DanielDraco]] 22:21, 17 July 2012 (UTC)

::::::Alright, with the only thing in contention being the possibility of changing the minimum votes, shall we move forward with the generally-agreed-upon proposal of putting thresholds at 0.5 and 3.5? And let's just set the grace period to a week for now at least -- that part is really easy to change later if it proves insufficient. I don't foresee it being enforced at the end of the seventh day on the dot anyway. --[[User:DanielDraco|DanielDraco]] 19:51, 21 July 2012 (UTC)

==Adoption Ratings==

If one rates a work, then some time later adopts the work, should they remove their previous rating?--[[User:Franken Kesey|Franken Kesey]] 21:31, 16 July 2012 (UTC)

:Any time an article is changed significantly, all ratings should be nullified. --[[User:Aarnott|Aarnott]] 21:34, 16 July 2012 (UTC)

::Nullified is hard to do. They should not be purged though. Removed from a rating block and kept as commentary instead? Probably. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:40, 16 July 2012 (UTC)

::Or I could add a new parameter to the Rating template that would make it not count a rating that was outdated. Probably better than shuffling text around. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  22:40, 16 July 2012 (UTC)

:::A parameter seems best. But as for the notion of an article being "changed significantly", do we have specific guidelines on that? I think most of us can pretty intuitively understand the reasonable threshold, but guidelines can help newbies and those of poor judgement. --[[User:DanielDraco|DanielDraco]] 00:17, 17 July 2012 (UTC)

::::The tag is in. I'll write up some guidelines as well as how to apply the tag when I get to rewriting the rest of this page. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  23:32, 25 July 2012 (UTC)

== Small Bug ==

It appears that an article with only Hate ratings shows up in the author box as being Unrated. --[[User:Ganteka Future|Ganteka Future]] 21:20, 21 July 2012 (UTC)

:Link to example please? - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  21:58, 21 July 2012 (UTC)

:In case you were referring to [[Magically Infused (3.5e Feat)]], that wasn't a bug. The page had not purged its cached version within the last two days, when the ratings were applied. A manual page refresh (with the wiki refresh option, not F5) fixed it right up. Those should be pretty rare, but I can talk to Aarnott and Surgo about making sure the cache is being updated properly.

:If it was a different page, try the wiki refresh first, and if it's still broked please link me. - [[User:Tarkisflux|Tarkisflux]] <sup>[[User talk:Tarkisflux|Talk]]</sup>  22:03, 21 July 2012 (UTC)

::My bad then. I had seen it on an earlier page edited a few days ago and noticed it again when checking Magically Infused, but it must have been before a cache update, as checking on some other pages, I see it works just fine. Good to know it currently might take a while then for caches to update. --[[User:Ganteka Future|Ganteka Future]] 22:14, 21 July 2012 (UTC)