Frugal Living 

Beware Wine Expert Ratings & Rankings

Email  Print Print  

Glass of WineSince we’ve been doing a lot of “cooking as entertainment,” my wife and I have started drinking a bit of wine to help make cooking more entertaining. 🙂

Always price conscious, we set out to buy wines that we enjoyed but didn’t bust the bank. As twenty-somethings, we really don’t need our palates reaching beyond our pocketbooks, if I may be so bold in my use of alliteration. Fortunately, our tastes are plenty satisfied with bottles that cost only in the “teens” (or less, as is the case with many Rieslings). For some of our wine-buying tips, I invite you to read an earlier post title How to Save Money on Wine.

So I wasn’t surprised to read a WSJ article that basically called wine critics full of crap. Actually, full of crap is probably a little harsh, but the point of the article was that wine critics are inconsistent, to the point of being random, and that wine ratings is big business.

In his first study, each year, for four years, Mr. Hodgson served actual panels of California State Fair Wine Competition judges—some 70 judges each year—about 100 wines over a two-day period. He employed the same blind tasting process as the actual competition. In Mr. Hodgson’s study, however, every wine was presented to each judge three different times, each time drawn from the same bottle.

The results astonished Mr. Hodgson. The judges’ wine ratings typically varied by ±4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ±2 points.

There’s more to the story, and more alarming statistics, than what I quoted but the conclusion is the same – the wine rating system is flawed, the raters aren’t as good as they’d like to believe, and that a higher rating might not necessarily mean a better wine… just a better day.

Which shouldn’t be surprising… if you hate chardonnay, you’ll hate it regardless of the score. In fact, the better the chardonnay, the more you might hate it.

Which brings me back to my point… try a bunch of wine, buy what you like, and ignore rankings and ratings. We know this when it comes to movie and restaurant reviews, but somehow with wine it’s different. My favorite places to eat are absolutely dives with little decor, no ambiance, emotionless (sometimes, rude!) wait staff, but the most delicious food I’ve ever had. Restaurant critics would slam them if critics reviewed those types of places, but they’re perfect to me.

So the next time you see wine rankings and scores, remember that there’s can be a four point swing either way (on average) and that you are the sole judge of what you’ll like.

(Photo: mr_t_in_dc)

{ 22 comments, please add your thoughts now! }

Related Posts

RSS Subscribe Like this article? Get all the latest articles sent to your email for free every day. Enter your email address and click "Subscribe." Your email will only be used for this daily subscription and you can unsubscribe anytime.

22 Responses to “Beware Wine Expert Ratings & Rankings”

  1. Baloo22 says:

    Bottles in the teens? You must be rich. LOL…
    When I’m looking for a good, inexpensive wine I start with price and work from there (usually $10 or less). After trying a few I know exactly what I’m after. I love telling my friends how much I paid for the wines they are drinking. The looks on their faces are priceless.

    • Martha says:

      Man! Its hard to find bottles of wine (not including White Zinfandel)for under $10 here! When I lived in Germany I could purchase a $2.50 bottle of wine that was excellent. 🙂

  2. lostAnnfound says:

    I’m with you, Baloo22. Low price has no bearing on which wines we will drink.

    A great way to try new wines before buying is to go to a wine tasting. One of the larger package stores in our area runs tastings on Saturdays from October through this Saturday (11 weeks total). $5.00 donation, which goes to a local charity, will allow you to try anywhere from 25 to 35 wines each week. I’ve tried a $6.99 Cabernet and at the same tasting a $64.99 Cabernet. One table of German Rieslings (eight different varieties from 8.99 to 24.99). This way I can try before I buy regardless of the price.

    Anyway, taste is subjective. What one critic may not care for may be my next favorite.

  3. CK says:

    Amen, judging wine is a bit like judging movies. Most everyone can agree on what is really terrible and what is extremely superb, everything else in between is your own taste. Buy what you like.

  4. Fascinating. Wine rating is such an art.

    I wonder if the industry would blow itself up if they said like every other type of statistical survey, “there is a +/- 5 point margin of deviation”!!!

    Being in San Francisco, wine is everywhere with Napa Valley. Chateau Montelena was the first winery in St. Helena/Calistoga area to put the US wines on the map in the 1970’s Chardonnay tasting comp in France!

    Everywhere come out to the SF Bay Area and spend your money and support our economy! Facebook is going public next year, so you need to come visit before the newly minted millionaire/billionaires drive up prices further here!!

    • Jim says:

      They probably would blow up… +/- points on a 95 point wine means it’s either a 90 or a 100! 🙂

      • Like reviewing movies, or being a food critic, if you’re a wine buff and review wines for a living, it must be the best job on earth!

        I was thinking of scoring some free food at my favorite restaurants in SF with my wife if I did a review of their site… but I don’t think that will work. But maybe! Ya never know! 🙂

  5. Christianne says:

    There’s much to be said about different palates tasting differently and different preferences to styles when it comes to wine. I won’t get in to the argument about wine ratings and how well judges fare during contests. It’s not worthwhile arguing about. I prefer to add my 2 cents on buying wine that you like for a good price.

    Attending tastings is a great way to try new wines before you buy them. When first starting out with wine, it helps to be open to trying many different styles and varietals so you can learn what it is you like and how to describe it. Telling someone at the wine store that you like dry wine won’t help. Unless you drink desert wines or other sweet wines, all the rest are dry. Something closer to – “I like new world wines, with more complexity and definite earthiness. I’ll be drinking it alone or with my pasta and meatballs.” Obviously not everyone can do that. But the benefit of a better description of what you want will help a good store sales person get you what you want. And when you find that salesperson, and trust their judgement, you won’t be wasting moeny on your wine. It’s worth it even if you spend a buck or 2 more. If all you want is the alcohol, then buy the big jug wines. Part of the fun of wine is trying new varietals or winemakers and experiencing the nuances that each person may find in the wine. Add good food and good conversation to the mix and the evening is perfect!

    On a side note: Have fun with the wine, just because you aren’t tasting licorice doesn’t mean that you or they don’t know what they are talking about. Too many people try to intimidate others with their knowledge – much of which is inaccurate or made up. When I teach classes, that’s the one point I try to always get across.

  6. ziglet19 says:

    My husband and I are just learning to enjoy wine as well. We usually just pick out a bottle and try it. If we don’t really care for it, we don’t buy it again. Not the most sophisticated system, but it works for us!

  7. Maddhatter says:

    For my tastes a +/- 5 rating would not be a big deal. I think as a newbie to wine, the region a wine is made in can make a big difference. That is how I tend to make my wine selection, I’m sure it is just as flawed, though.

  8. cubiclegeoff says:

    I read a story a couple of months ago where the researchers took white wine and gave it to professional wine tasters, which used typical white wine adjectives to describe the wine. The researchers then took the same white wine, used food coloring to make it red and gave it to the judges. They generally all described the wine using adjectives associated with red wine. Basically they don’t know anything (unfortunately I can’t find a link to the study).

    • lostAnnfound says:

      “Expectations can even play havoc with experts. A few years ago, Frederic Brochet, a cognitive psychologist at the University of Bordeaux, conducted a rather mischievous experiment. He invited 54 experienced wine tasters to give their impressions of a red wine and a white wine. Not surprisingly, the experts described the wines with the standard set of adjectives: the red wine was “jammy” and full of “crushed red fruit.” The white wine, meanwhile, tasted of lemon, peaches, and honey. The next day, Brochet invited the wine experts back for another tasting. This time, however, he dyed the white wine with red food coloring, so that it looked as if they were tasting two red wines. The trick worked. The experts described the dyed white wine with the language typically used to describe red wines. The peaches and honey tasted like black currants.”

  9. Dave says:

    Great post! I’m not a huge wine fan, but I thought that in order for a particular wine to be called a chardonay, reisling, etc, that they had to have a certain percentage of that type of grape? I would think that the flavor has to be pretty close no matter what, if that were the case.

    I’m much more of a beer snob, and the same holds true there. Other than your “cheap” beers, bud, coors, etc, any “good” beer will be a little differnt than another one of the same type, but that doesn’t make one better than the other or deserving of a higher price tag.

  10. Chris says:

    My wife and I shop for wine at ABC Liquor store. They classify the wines by taste (sweet, bitter, fruity) rather than ranking.
    I hear Two Buck Chuck is pretty darn good.

    • Martha says:

      I hear that $2 Chuck is good too (its from Trader Joes for those of you who haven’t heard of it…) but we can’t get it in our state! Boo, liquor licensing rules!

  11. aua868s says:

    i have come across people who claim to rate the quality of wines just by smelling them….have seem them give high rating to an $8 bottle Gallo wine!….agree that there are real experts at it…but most are just cooked up guys!

  12. JNichols84 says:

    I only recently started drinking wines. I have found that my better tastes come from less expensives bottles, one of my favorites is a $10-$12 Sutter Home Moscato. Those snobs that turn their nose up are making the money by endorsing someone else’s brand! 🙂

    • lostAnnfound says:

      Yum! That Moscato is one that my friend usually has at her house. We love to have it while sitting out on the patio on a summer evening. Delicious!

  13. T. says:

    Aside from the obvious, that wines taste differently depending on a million factors, I was always taught that a good wine is what you like, a bad wine is what you don’t like.

    I use the wine critics to filter out what I *don’t* want. I don’t like tannins, or “a long finish”. I don’t like chalky reds or overly oak-y whites. I prefer dry to drink, sweeter to cook with. All these are just my preferences. Wine critics usually have different likes. For example, the “ashtray” and “tobacco” flavored wines. Ick. 😀

    And for the record, those fancy white wine sauces you get in restaurants usually use a jug Chablis — the alcohol burns off anyway, and it’s sweet enough to carry the sauce but cheap.

  14. Izalot says:

    I like wine but I don’t see myself every paying more than $10 for a bottle. Maybe a special bottle of champagne? Time and time again in blind test studies show that most people cannot tell the difference from an under $10 wine with a more expensive one.

    Cheers for everyone here this holiday season. I’m drinking a glass of Shiraz as I type!

  15. eric says:

    lol what professionals 😛

  16. RR says:

    This article really nailed it. The wine ratings system is a sham. It really doesn’t matter how other people rate the wine, their tastes are going to be different than yours.

    I personally try different brands of wine and stick with ones I like, no matter the price or label. Most of the ones I really enjoy are well under $10 per bottle. I have had more expensive wines that do not taste any better.

Please Leave a Reply
Bargaineering Comment Policy

Previous Article: «
Next Article: »
Advertising Disclosure: Bargaineering may be compensated in exchange for featured placement of certain sponsored products and services, or your clicking on links posted on this website.
About | Contact Me | Privacy Policy/Your California Privacy Rights | Terms of Use | Press
Copyright © 2016 by All rights reserved.