The new year is upon us and so it’s time to take a look back. For gaming websites like this one, that usually means some sort of game of the year award where the editors of the publication get together and vote/argue for their personal picks.

How myopic. It’s a new year; some perspective is in order.

Ever played that game where you try to guess the number of jelly beans in a jar? You know the one—you usually see it at fairs and consumer shows with a door prize for the closest guess. Ever won? Neither have I, and for the most part, guesses are outlandishly wrong.

However, if you get enough people together and average all their guesses, the number you get is eerily close to the number of jelly beans in the jar. It’s called the wisdom of crowds and it works because of the basic principles of statistical sampling.

If you don’t know how many jelly beans are in the jar, there’s about a 50/50 chance that you’ll guess too high or too low. So when you get enough people to guess, about half overestimate and half underestimate. If you average all the guesses, the over- and under-estimations cancel out, and the average ends up pretty close to the right number.

Why nobody applies this logic to game of the year is baffling. Ok sure, you could argue that the jelly bean thing only works because there’s an objective number of jelly beans, and the wisdom of the commons won’t work when you apply it to something like critical evaluation of video games. Of course, while you’re making that argument, you’ll also have to abandon numerical review scores and review aggregators like Metacritic and Gamerankings. Besides, unless you want to assert that there’s ABSOLUTELY nothing objective or universal to making a good game, there’s some good to be had from crunching the numbers.

The way that most of these awards work is a list of 5-10 contenders is assembled, from which a single game is picked. We took 6 major critical outlets (Gamasutra, GameRankings, GameSpot, GameSpy, Joystiq, and Spike TV) and aggregated their picks. We awarded 2 points to each game that won Game of the Year and one point to each runner-up. The complete list looked like this:

Fallout 3: 6
Metal Gear Solid 4: 6
Grand Theft Auto IV : 5
LittleBigPlanet: 5
Gears of War 2: 4
Braid: 3
Left 4 Dead: 3
Fable 2: 2
No More Heroes: 2
Shin Megami Tensei: Persona 4: 2
World of Warcraft: Wrath of the Lich King: 2
Castlevania: Order of Ecclesia: 1
Chrono Trigger DS: 1
Dead Space: 1
Far Cry 2: 1
Geometry Wars: Retro Evolved 2: 1
Mirror’s Edge: 1
NHL 09 : 1
Professor Layton and the Curious Village : 1
Resistance 2: 1
Rock Band 2 : 1
Super Smash Bros. Brawl: 1
The World Ends With You : 1
Valkyria Chronicles: 1
Warhammer Online: Age of Reckoning : 1
World of Goo: 1

Here’s a few factoids that jump out:

• Game of the Year was a tie between Fallout 3 and Metal Gear Solid 4. That said, there wasn’t a clear standout. The points were pretty evenly distributed, with many ties and no gaps in the scoring between the minimum of 1 and the maximum of 6.
• Of the top 11 (the games with more than one point) 7 were sequels.
• 2008 was a good year for the action-adventure genre, which emphasizes exploration gameplay, though FPS and RPG hybrids also had a strong showing.
• The Xbox 360, PS3, and PC all ran neck and neck this year, outpacing the next competitor (the Wii) by a factor of 4.
• There wasn’t a lot of love for portables or digital distribution. Either it’s been a bad year for the casual market, or these awards implicitly ignore that demographic.
• World of Goo and Professor Layton deserved more nominations than they got. Seriously, what is wrong with those people?

Some caveats: the list of critical outlets wasn’t ideal—IGN and 1up haven’t posted their 2008 picks yet, and the gamerankings list was aggregated from reviews over the year rather than a formal award. Furthermore, the scoring is a bit rough-and-ready and doesn’t really account for the fact that some lists were half as long as others (which means that top 10 lists produced about twice as many points as top 5 lists).

So how did this list line up with your favorite games of the past year? Any suggestions for how to do the data processing when I run this again next year?

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Tags:   2008 · game of the year · statisticsNo Comments

Trackback to this article.