Who Reviews the Reviewers?

By Nelson Schneider - 08/28/11 at 11:18 PM CT

Critics have always served as curators of culture, passing judgment on what has value and what has none. Yet in the environment of Web 2.0, everyone with a connection to the Internet has suddenly been empowered with this capability that was once limited to the connected and elite few. Sites like FilmCrave and MeltedJoystick are built upon this new paradigm and retail outlets like Amazon are relying more and more on customer input, in which mobs of users are free to share their own personal opinions to build-up or tear-down a product or piece of media, almost like a form of vigilantism. While ‘professional’ critics still exist, they are becoming less and less relevant, drowned-out in the ocean of other voices shouting opinions. To paraphrase Juvenal, “Who reviews the reviewers?

Professional game reviewers have always battled off-and-on with a perception of corruption – that they receive free, early copies of games in exchange for favorable reviews. I have seen accusations of bias leveled at critics ranging from EGM to Ars Technica. Yet the only time a review is truly compromised is when the reviewer has a stake in the success of the product reviewed. How can a magazine like Nintendo Power be trusted when it is simply a mouthpiece for a company that makes many of the games reviewed therein?

But the question in the end always comes down to the age-old philosophical debate between objectivity and subjectivity. While objectivity is usually preferred and is incredibly important in journalism, criticism cannot exist without subjectivity. While most journalists are called upon to report ‘just the facts,’ critics must also interpret those facts for virtue and fault.

While the professional review model relies fundamentally on authority, the user review model relies fundamentally on trust and consensus. It’s like the difference between religion and science, where in one model an infallible source declares what is true and in the other numerous fallible sources share information until truth is discovered. MeltedJoystick assumes that users will write their honest opinions about games, and we throw our staff reviews into the same pile as all the others. Indeed, our staff reviewers review games we have bought ourselves and want to play. But, sadly, user reviews can’t always be trusted, as those with vested interests have been greasing palms with cash to flood websites with fake reviews. While there are already technological efforts in screening for this planted information, the risk for false positives is incredible and risks damaging the user review model as a whole.

So, what makes a good review? Also, what makes a good critic?

In the egalitarian Internet, finding a good critic – for anything – is primarily a function of reputation and rapport. Learning a reviewer’s background – what games they’ve finished, their favorite/least favorite genres – can be very eye-opening. It’s also the basis for determining how your opinions align with theirs and what kind of scores you should expect to see. Because I make it no secret that I’m not a First-Person Shooter fan, it would be silly for a “Call of Duty” junky to read my review of the latest “Modern Warfare” title (so silly, in fact, that I have no intention of ever writing a review of a “Modern Warfare” title). But by that same token, the fact that I greatly praised “Half-Life 2” and “Portal” in my review of “The Orange Box” should alert readers that those games must be something special if they can appeal to a non-fan of the genre. Where subjectivity dominates, like-mindedness is the foundation upon which trust is built.

When writing a good review, it is important to tie objectivity and subjectivity together. Don’t just state that “x sucks” or “y is awesome.” First lay down an objective description of the experience, then qualify it by adding subjective opinions. Doing this ensures that even when the reader disagrees with your final review verdict, they can still learn something useful from the review. I am also a fan of using a review template, in which different aspects of a game are discussed in separate segments, rather than rambling around a variety of topics and turning the result into a confused mess. Doing this helps readability and allows readers to skip to the segment they want to know more about. In my review template, I focus on “Presentation,” which covers graphics, art design, music, sound, voice acting, and technical problems; “Story,” which covers the game’s premise, narrative, and characters; and “Gameplay,” which covers the meat and potatoes of how the game actually works and what they player will actually be doing for the duration of the game. Some other reviewers like to add sub-sections for “Replay Value,” “Challenge,” and the like, but I find that all of those other sub-categories can be covered by one of the three that I use (in this case, Replay Value and Challenge would be covered under Gameplay). Regardless of whether you use a template or not, it’s important to share details. Assume the prospective player will ONLY read your review.

In the end, the answer to Juvenal’s question is simple: They watch each other.

Comments

Sign Up or Log In to post a comment.
Are you sure you want to delete this comment?
  
Are you sure you want to delete this comment?
  
Are you sure you want to delete this blog?