|
Introduction:
As you may know, the review scoring system changed roughly a year ago from an out-of-ten scale to an out-of-five scale. You may also know that I personally dislike and oppose this change. I probably should have addressed this issue earlier, but I wanted to give it a fair chance first. As I believe I've given it long enough now, I'm going to explain in detail why I oppose it and why I prefer the original out-of-ten scale more. You may agree, you may disagree. All I ask is that you hear me out and let me get this off my chest. Yes, it's a little long, but hopefully you'll find it a little interesting. If not, you can just skip to the end and read the conclusion.
First of all, why is the scoring system such a big issue? After all, the most important part of a review is the review itself, not the scoring. We should be able to get a clear idea of the reviewer's opinion of a game by simply reading the review. In an ideal world, this would mean any scoring system we choose would work fine. We could even do away with the scoring system entirely if we wanted. Dracos's Soulriders forum has an example of a fully functioning review section that doesn't use or need scoring. In order to find out the reviewer's opinion of a game, the reader must at least skim the review. This process helps prevent people from making uniformed comments on a review. So, why is the scoring system such a big issue? Well, honestly, it really isn't. The only reason I'm focusing on this part is because the other, more important, sections of the review have already been reasonably refined.
Since it would probably cause some kind of massive uproar if we did away with scoring here, we might as well make the best of it. There are actually quite a few ways that scoring can be useful. Since the world isn't ideal, it is quite possible for people to forget what was said in a review they read months or years ago. In this case, scoring can be used as a quick refresher. It is a sad fact that most people just don't have time to read every review written for every game they're curious about. Xenogears alone, for example, has somewhere around twenty reviews. In cases like this, they could use scoring as a reference for picking a few reviews to read, such as the highest score, the lowest score, and a few in the middle. Another useful thing scoring can be used for is comparing your opinion to that of a reviewer. With the way our archives are set up, it's very easy to scan through and see what a particular reviewer likes or doesn't like. If there's a review in a Points of View update for a game you haven't played, it's a simple matter of clicking their name to see how they liked other games. This makes it much easier to see if you generally agree with their opinion or not. It's also possible to find a reviewer that generally shares a similar opinion this way, which would help out in the Xenogears example I used earlier. Obviously, nothing compares to reading everything in existance, but scoring can help those that are pressed for time. After all, we do have several hundred reviews in our archives. I'm probably leaving out some other uses of scoring, but I think I've shown that scoring can be a useful tool.
Now here's the problem and where the debate begins. Scoring can be a useful tool, but what scale should be used to maximize that usefulness? If the scale is too small doesn't have enough choices, it won't tell the reader enough information to be particularly useful. On the other hand, a scale that is too large would most likely lack clearly defined standards. What's the difference between 100000 and 100001? So then, what's the best scale to use? Out-out-six? Out-of-ten? Twelve? Thirty? One hundred? ABCs? To be perfectly honest, I really don't know what the best scale is. I'm just here today to show you why, in my humble opinion, I think the old out-of-ten scale is superior to the current out-of-five scale.
Out-of-Five versus Out-of-Ten:
Now we come to the meat of the editorial. Naturally, I'm arguing for the out-of-ten side, so this section will be a little one-sided, but I've tries to represent the arguments for the other side as well as I could. As a result, I've compiled the arguments that many people on both sides have made, elaborated on them, and added several of my own.
The primary reason for the conversion, or at least the reason 7/10s become 3/5s, is the fact that a number of reader reviewers used 7/10 as average despite the fact that 7/10 was defined as "Good: School children everywhere are saying right now that '7' is average. Only in school is this true. A '7' class game can still have some major flaws, but overall the title is trending toward greatness. Typically, once you get past this point, there is some level of genius in the games design." This resulted in several games having a slightly inflated average score, despite the fact that the majority of the reviewers used 5/10 as average. Now that the change in scales has been made, the average now leans just slightly above a 3/5.
This, however, brings up the question of whether or not those who gave 7/10s to games generally considered average indeed truly felt that the game was good or not. Inflated scoring could certainly be the case for a number of games, but there are also cases where the 7/10s were accompanied by 8/10s or higher. Such is the case with Xenogears, Zelda II: Adventure of Link, and Super Mario RPG: Legend of the Seven Stars, just to name a few. Is putting the average games in with the good games really the best answer? It's true that the average score for most games was higher than a 5/10, but this could have also been due to the fact that people simply tend to avoid or not finish games they don't like. Not everyone is on a masochistic quest for the worst game of all time like I am.
Another reason many people support the out-of-five scale is the fact that the new numbering is simple. This is indeed true. It is sometimes difficult for a reviewer to narrow down scoring choices for those that are borderline. As there are fewer scoring options, there is a much smaller chance of anything being borderline. This makes life easier for reviewers sometimes. The problem is that many also feel that the system has been oversimplified. While it may make life easier on the review when they are scoring something, fewer options also means less information for the reader.
It has also been argued that people really only have the mental ability to distinguish between five levels of how good something is. I'm no expert on the human mind, but the out-of-ten scale seemed to be working just fine with ten.
Now that I have addressed and countered the arguments I have heard for out-of-five, I will present several reasons to switch back to out-of-ten.
My primary concern with the original out-of-five scale was the fact that 4/10, 5/10, 6/10, and 7/10 all became 3/5. Fortunately, this has been altered slightly to just 5/10, 6/10, and 7/10, but that is still a significant number of scores piled together. While games that receive a 1, 2, 4, or 5 tell the reader a reasonable amount about the game, games that receive 3s could be anywhere from average to just slightly less than great. To make matters worse, a considerable portion of games now fall into this category. If most games fall here, it makes the scale pretty useless for quick reference since it tells the reader very little. To quote Red Raven: "For what it's worth, I felt uncomfortable converting Michael's .hack review scores from 6/6/7/4/7/6/5/7=6 into 3/3/3/3/3/3=3 as per guidelines."
To get around the constraints of the 3/5, many people have suggested that reviewers simply convert 7/10s to 4/5s if they felt like it. Unfortunately, this creates a new problem. Scoring has become much more dramatic from certain people due to the fact that there are fewer choices and many games are "too good" to receive the score they would have earned otherwise. Not only does this occur in 3/5s -> 4/5s, it has also happened with 4/5s -> 5/5s. The opposite is also true, as there has been a vast increase in 1/5s because the game is "too bad" to receive a 2/5. There has been a massive increase in 5/5s and 1/5s lately that is just as the problem concerning 7/10s as average ever was. The system breaks down a bit if some people are giving excessively high or low scores while the rest are following the scale carefully.
Another large concern that was brought up during the change was the alteration of reviewer intent. To give an example of this:
Before score conversion:
Final Fantasy Adventure (Seiken Densetsu) 7/10 3 Reviews
Final Fantasy Anthology 6.5/10 4 Reviews
Final Fantasy Adventure outclassed Final Fantasy Anthology originally...
After score conversion:
Final Fantasy Adventure 3 Reviews 3.33/5
Final Fantasy Anthology 4 Reviews 3.5/5
...but now Anthology is rated higher than Adventure. This change may not be incredibly substantial, but it does go against the original collective intent of the reviews. Someone trying to decide between these games would be given false information.
It has also been suggested that the scoring change has been responsible for a decrease in reader reviewers. I don't know how accurate this statement is, but it does apply to a least one reviewer. Solon: "And I think that scale has been one of the reasons people stopped writing as well.. I know for sure that was one of the reasons I stopped"
I believe kweee, intentionally or not, sums it up well with the following quote: "On a scale of 1 to 10, I would give the 1 to 5 scale a 4. On a scale of 1 to 5, I would give the 1 to 10 scale a 3. Does that make sense? Am I losing it yet?" While the score he give the out-of-five scale is very clear, the score he gives the out-of-ten scale could be anywhere from average to good. This easily demonstrates how inflexible the out-of-five system is.
The Transition Back:
As we have not fully completed the change to out-of-five yet, as can be clearly seen from the place-holder images in our review archives, we really need to decide which way we're going to go, whichever way that may be. If this editorial gains as much support as I hope it will, we will be switching back to out-of-ten scoring. Naturally, switching back won't be easy. We would have to reload the archives with the out-of-ten data from last March, manually add all of the reviews we've received since then after tracking down and asking the reviewers their scores out-of-ten, and fix all of Points of View's subpages including the review templates.
Who would all of this work fall upon? Well, the answer is actually pretty simple. Of course, one of our programmers would have to reload the archives and reviewers would have to re-score a few of their reviews, but the majority of the other work would fall on the Points of View curator. While those of you who are unfamiliar with the section may think that I am being cruel to the curator, the rest of you already know that I am, in fact, the curator that most of the work would be placed on. I'm well aware of the consequences of changing back, but I feel strongly enough about this issue that I'm willing to do so without hesitation.
Conclusion:
Scoring is not the most important issue, but the rest of the review seems to be in good shape, so I'm focusing on scoring right now. We could do without scoring if we had to, but we might as well make the best of it since we're pretty much stuck with it. I honestly don't know what the best scoring system is, but I've given several arguments as to why I believe out-of-ten is better than out-of-five. If we did switch back, I would be the one stuck doing most of the work, so please don't think I'm just casually suggesting this without thought of consequence. So there you have it. That's my opinion on the matter. Thank you for hearing me out.
|
RSS feed
iTunes