What Good Are Ratings?

[sailor22]

On the flip side I find that blind tastings among knowledgeable consumers provides one the best and most honest evaluations of a product. A tasting group of 12 reasonably knowledgeable individuals each tasting 2oz. of "blinded" spirit over several days with no results shown until all are in will IMO produce about as honest a rating as one can hope to get.

I understand your point but keep in mind a rating generated from many people is in the end a popularity contest. Valuable only in so far as you have some knowledge of the other tasters palates and preferences. Use it as a rough guide for what to try next perhaps but trust your own taste first.

[cigarnv]

I understand your point but keep in mind a rating generated from many people is in the end a popularity contest. Valuable only in so far as you have some knowledge of the other tasters palates and preferences. Use it as a rough guide for what to try next perhaps but trust your own taste first.

Steve... agree with you fully... if it is "blindly" popular I will usually give it a try....

[politely]

I like ratings and they do affect my purchasing decisions, but I don't really focus on the exact ratings. I care more whether it's highly rated by the specific reviewer within their own framework, and whether other reviewers have similar opinions. I do take note when a reviewer marks an expression as special or extraordinary. But I think the reviews provide, at best, a subjective view and I take the reviews with some salt. I suppose reviewers could be a special breed with amazing consistency, but I find that my own opinion of a particular bottle changes from time to time, depending on the day, my mood, taste, and changes in the bottle (ie, time and volume). My guess is that if a reviewer checked back, their rating of the last pour would likely be different from the first pour. Actually, I'm not sure that the rating at the opening of the bottle is necessarily the most accurate or best. People generally don't finish a bottle the day they open it, and so, in my view, the changes over time are important, but aren't really given any consideration.

[Buffalo Bill]

In my bourbon inventory I rate both taste and value on a three-point scale (+/o/-) corresponding to better-than-average, average, worse-than-average -- where "average" corresponds roughly to Jim Beam Black.

2 months ago I hit a string of rare JB Black nips that were way better than average. Went back and bought a box, average.

Go figure...

BB

[Flyfish]

In my bourbon inventory I rate both taste and value on a three-point scale (+/o/-) corresponding to better-than-average, average, worse-than-average -- where "average" corresponds roughly to Jim Beam Black.

You are very close to the two-point system I use to rate bourbon: 0 = Don't like it; 1 = Like it. Off the top of my head, can't remember any 0s. :grin:

[Tom Troland]

I think whiskey reviews are best regarded as entertainment, stuff made up more to amuse than to enlighten, just like comic books. The best reviewers, like Paul Pacult, are clever writers. They amuse us. For example, Pacult once described a distilled spirit (whiskey, I think) as “buxom”. So what is a buxom spirit? Is it stacked with flavor? Despite my ignorance of his meaning, I was amused by his word. And I gather that buxom is good, at least in his eyes. We all have our preferences.

Most whiskey reviews contain very little useful information. Just like comic books. Whiskey reviewers often invent ten or more descriptors for a given sample. John Hansell is fond of this approach. For example, his recent review of Elijah Craig 20 Year Old identifies aromas of “nutty toffee, pecan pie, apricot, berried jam, and nougat, peppered with cinnamon, mint, cocoa, and tobacco”. Not to mention ”polished leather and dried spice” in the finish. Now, with all due respect to John, how could anyone possibly imagine all of those aromas combined? John might as well be describing his latest acid trip. The description is fascinating, but it is unlikely to resemble anyone else’s experience with the same substance.

The pointlessness of whiskey reviews (except as entertainment) is well illustrated by the reviews in Whisky Magazine. In an honest attempt to be more informative, Whisky Magazine offers two independent reviews of each whiskey. But if you read the two reviews, the texts have almost nothing in common. Imagine cutting out each whiskey review in a given issue of the magazine. Now try to match up the pairs of reviews that describe the same whiskeys. I am quite sure you would do little better than random chance.

Beyond the words of whiskey reviews there are the numbers. Especially the 100 point scale. Numbers normally imply quantitative measurement, at least in the science field where I work. But reviewers’ numbers are all made up. You might as well rate women’s beauty on a 100 point scale. Or guys’ good looks. Or acid trips. And Jim Murray believes his palate is so sensitive that even the 100-point scale is too coarse. So he awards half points. As if he is measuring body temperature with a thermometer under his tongue. Or somewhere else.

Of course, whiskey reviews are quite harmless. They offer good entertainment for many (like me), bragging rights for distillers, and an opportunity to collect and drink by the numbers for those who are so inclined. But we should at least acknowledge the obvious. This stuff is all made up. Just like comic books.

[cowdery]

Tom,

You're being unfairly hard on comic books.

[Jono]

Well, in a similar vein, most of us would agree on a broad rating of 7s, 8s, 9s and 10s for looks...be it men or women...and though the exact rating may differ I bet a large % would be within 10 pts. So, there is probably value in identifying the best from the average and bad. Once you find a reviewer with a similar palate the reviews carry more weight. I cannot tell you if a 87 pt whiskey or wine etc. is worse than a 91 pt but I assume both are good.

A 74 pt product should be noticeably worse than a 85+. General agreement is probably valid with some outliers expected.

I only pick up 2-4 flavor notes in most whiskey or wine so the pantry list of notes usually just adds color. I am not saying they don't exist, but most are passing experiences that come and go quickly in the overall taste. I think the mind can create an expectation and the note will be found if searched for in the mix.

[camduncan]

Tom,

You're being unfairly hard on comic books.

Too funny Chuck, too funny! :slappin:

[Tom Troland]

Chuck,

You caught my bias, I never did like comic books although I always enjoyed Mad Magazine.

Despite my comments about whiskey reviews, I do read them avidly. And I respect the whiskey knowledge of people like John Hansell, Jim Murray, Paul Pacult and yourself (even though you are not primarily in the business of reviewing whiskeys). If a new whiskey release comes out, and it is expensive, I'm more likely to try it if respected reviewers rate it highly. But I don't give much weight to the exact numbers on their scales nor to their words of description unless they are funny.