- Bargaineering - http://www.bargaineering.com/articles -

Beware Wine Expert Ratings & Rankings

Since we’ve been doing a lot of “cooking as entertainment,” my wife and I have started drinking a bit of wine to help make cooking more entertaining. 🙂

Always price conscious, we set out to buy wines that we enjoyed but didn’t bust the bank. As twenty-somethings, we really don’t need our palates reaching beyond our pocketbooks, if I may be so bold in my use of alliteration. Fortunately, our tastes are plenty satisfied with bottles that cost only in the “teens” (or less, as is the case with many Rieslings). For some of our wine-buying tips, I invite you to read an earlier post title How to Save Money on Wine [3].

So I wasn’t surprised to read a WSJ article [4] that basically called wine critics full of crap. Actually, full of crap is probably a little harsh, but the point of the article was that wine critics are inconsistent, to the point of being random, and that wine ratings is big business.

In his first study, each year, for four years, Mr. Hodgson served actual panels of California State Fair Wine Competition judges—some 70 judges each year—about 100 wines over a two-day period. He employed the same blind tasting process as the actual competition. In Mr. Hodgson’s study, however, every wine was presented to each judge three different times, each time drawn from the same bottle.

The results astonished Mr. Hodgson. The judges’ wine ratings typically varied by ±4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ±2 points.

There’s more to the story, and more alarming statistics, than what I quoted but the conclusion is the same – the wine rating system is flawed, the raters aren’t as good as they’d like to believe, and that a higher rating might not necessarily mean a better wine… just a better day.

Which shouldn’t be surprising… if you hate chardonnay, you’ll hate it regardless of the score. In fact, the better the chardonnay, the more you might hate it.

Which brings me back to my point… try a bunch of wine, buy what you like, and ignore rankings and ratings. We know this when it comes to movie and restaurant reviews, but somehow with wine it’s different. My favorite places to eat are absolutely dives with little decor, no ambiance, emotionless (sometimes, rude!) wait staff, but the most delicious food I’ve ever had. Restaurant critics would slam them if critics reviewed those types of places, but they’re perfect to me.

So the next time you see wine rankings and scores, remember that there’s can be a four point swing either way (on average) and that you are the sole judge of what you’ll like.

(Photo: mr_t_in_dc [5])