Cigar Science with Rob XIV: Are Blind Reviews the Best Method for Evaluating Cigars?

[bobbyg29-cl]

For this edition of Cigar Science I tackle a somewhat controversial topic that I hope with generate lots of debate here on Cigar Live: the value of blind cigar reviews. As you will see, I do not think that blind reviews are the best way to evaluate cigars. Due to the powerful and unavoidable role expectations play in normal, everyday cigar smoking I think the practice of artificially removing expectation with blind reviews is logically flawed. I came to this conclusion after looking at research on consumer psychology and through my own experiences doing blind and non-blind reviews here at Cigar Live.

I again teamed up with Doc Stogie Fresh for an article in his Stogie Fresh Journal: “Great Expectations: What Blind Cigar Reviews Really Tell Us.” You can find the article by going to http://www.stogiefresh.com/ and clicking on the link in the upper right corner of the page under ‘Journal’.

Comments, questions, and any contrary opinions are more than welcome! In particular we would be interested in hearing opinions from manufacturers and cigar press members. I think this is an important topic to discuss.

Cheers!

Rob

[tx_tuff]

"To further illustrate this point let’s return the hypothetical rating scores for the Opus X and a new $5 cigar discussed above (i.e., non-blind: Opus X=95, New cigar=89/ blind: Opus X=90, New cigar=90). What implications do equal scores in blind ratings have for the reader that is looking for guidance in purchasing a cigar? If two cigars have equal blind ratings does that mean the reader should not bother wasting their money on the more expensive cigar (the Opus X in this example) because a cheaper cigar gives the same level of enjoyment? That would be a logical conclusion to draw from the blind ratings in the example above, given that both received exactly the same score. But the problem is that, although the blind ratings suggest that the two cigars will give an equal smoking experience, this does not tell us what will happen in a “real-world” environment, where the cigars are smoked in a normal, non-blind, expectation-filled setting. In fact, the non-blind rating scores in the hypothetical example suggest that if the reader were to purchase the two cigars and smoke them at home or their local B&M they would likely get much more enjoyment out of the Opus X (in large part due to the expectations associated with it) than they would get from the new $5 cigar. The conclusion drawn from the blind ratings (i.e., equal quality) is completely erroneous in this situation because expectations are a major component of normal cigar smoking. Whether we want to admit it or not, the expectation factors associated with an Opus X (e.g., rarity, price, knowledge of production methods, etc) make it a more enjoyable cigar for most cigar smokers. Expectations do not “contaminate” or “bias” the cigar smoking experience, on the contrary, they typically augment and enhance it. Thus, they should be taken into account when reviewing and rating a cigar."

I'm not saying I don't agree about the science of the blind vs non-bling reviews. But the Quoted section above from the article I just don'e agree with. If this was 100% true why do any reviews at all , we could just go buy what we are told is the best and not smoke anything else! I don't run out and buy anything just because of a review blind or not, but it does help. And if a cigar I haven't heard of or smoked before gets a good review in a blind test it doesn't tell me that its going to be a great smoke but it does tell me this is something I should try and see for myself. Thats why I think the blind reviews help alot. I have seen cigars I really enjoy get bad blind reviews and cigars I don't like get good cigar reviews, which just tells me that people have different taste. Any way its all good, I like both kinds of reviews and enjoyed the article, keep it up!

[baboruger]

Now, I think that there is more to the blind review than just a novelty game. I think that there is value maybe not to the cigar smoker, although, I am sure that there are some cigars from the last review panel that I will now try because of the blind smoke, but really for the marketers and the manufacturers. While I smoke a known cigar, I have an idea of what to think. I have a history of that brand or something, so i know what I am getting into, but that is because of the notions that I have.

For example, I picked up a Padilla Miami 8&11 because of all of the things I had heard about it, and I was really sort of disappointed. I would not really spend that money again. Looking at the blind reviews on Cigar live for that same cigar, I think the average was like a 5.8 or something, meaning ok but not great. The think I found funny about this was that people who listed this smoke as one of their favorites gave really bad reviews for it. Do I think that the blind review was different in taste, no, but it was the truer of the two reviews, not influenced by anything other than the smoke.

That is really what I am interested in, the true enjoyment of my cigars. My ratings are different than others because of my tastes, and other factors, so I think that taking the hype out of a cigar is what I really want to know.

As for the paragraph that Frank quoted, I think that with the Opus X or anything like that I would rather be blind because my expectations are never met and my enjoyment of the cigar always suffers. I think that it is each their own, some people like a Rolls Royce, and others the functional Toyota fits just fine, I think that it is so personal it is hard to do any review that is perfect or better. In the example Frank mentions, I am not sure that I would enjoy the OpusX more, I would most likely enjoy the $5 cigar better because it would be a nice surprise, and not being wealthy I would be even happier to find a bargain. In fact the first time I tried an Oliva for $3 I was really happy with the smoke because it was only $3 and really tasty, but I am 99% disappointed with the OpusX because I have higher expectations built up by others.

To me I like the blind reviews better because I want to know what people thought of the cigar rather than the hype around the cigar. I don't want the fact that it is an OpusX to influence the ratings. But I'm sure many others think differently.

All in all a really interesting article that really makes you put your thinking cap on. I do enjoy these and will like the discussion after. Even though I don't agree with the conclusions I do really enjoy these Rob, keep them up!

[smakudwn]

What is nice about the example posted was that the Opus still rated a 90.

[bobbyg29-cl]

Thanks for the great feedback guys...this is exactly what I wanted to do...start a lively discussion. I will give more specific comments about your posts later but consider this in the meantime....

Ok, let me flip the Opus X example around. Imagine that it scored a rating of 90 when reviewed blind and an 85 when reviewed non-blind (because when people knew what it was they felt it did not live up to it's price and hype and gave it a low score...like many people here do). As a reader of reviewers wouldn't you want to know whether the Opus is over-hyped and not worth the price? If you only saw the blind review you would have no way of knowing that. Price, name and all the other things influence our experiences when we smoke normally so why shouldn't they be part of the review?

And Frank whether or not we should use reviews at all (blind or non-blind) when buying a cigar is a totally different issue that I will be addressing next

[ilovecl-cl]

I understand that the price, rarity, and marketing all affect the pricing, and hype built over a cigar, but is not the blind review made and designed to take all of this away and rate the "cigar" and not everything the company wants us to experience? The blind review is made strictly to take away all of the outside influences and banish them. It is for reviewing the cigar and the cigar only. I tried a few 8-12 dollar cigars in a few blind tastings and then a few 2-5 dollar cigars. Some of the less expensive cigars were far greater than some of the more expensive cigars. I believe the blind reviews are something that needs to be done by the people who give these a national score. This would then show the rating of the cigar itself, instead of the "hype".

[bobbyg29-cl]

Originally Posted by John51277

I understand that the price, rarity, and marketing all affect the pricing, and hype built over a cigar, but is not the blind review made and designed to take all of this away and rate the "cigar" and not everything the company wants us to experience? The blind review is made strictly to take away all of the outside influences and banish them.

Yes, I agree totally. But my point is why do we want to take these away? They will always be there when we normally smoke a cigar so why shouldn't they be involved in the reviewing process. Why would we want to know how well a cigar "performs" in an environment totally different then the one we are going to smoke it in? To me, that would be like me basing a decision to buy a new truck on ratings of how well it is going to handle in snow when I plan to drive it in Arizona.

[Doc Stogie Fresh]

I fully agree with Rob on this one. I don't buy cigars blind, so why review them that way? In fact, I want MORE information on a cigar before buying, not less.

I found I could never rely on a blind test that was done by someone other than me. Everyone's palate is different. Even my own palate is different from day to day; there are just too many factors that can influence my taste and experience of a cigar on any given day.

And, if you lump numerical ratings from a whole bunch of people together using statistical averaging, you can only know what that group thought about the cigar in general, but it gives you no information on how your own likes or dislikes would compare with that average. That's why numerical rating systems are flawed as a tool for helping us to decide which cigars to buy.


In order to fairly blind taste a cigar, you would have to taste more than one or two. Manufacturers would readily admit that, in any given box of cigars, probably anywhere between 8-15 people have rolled those cigars. So, you would have no real idea of the consistency of construction (or even blend) by smoking one cigar. If I had to wait until I could taste a cigar blind before I purchased it, I wouldn't be buying many cigars... It's simply not practical to blind taste several of the same cigars prior to purchasing them.

That is one of the reasons for REVIEWS (as opposed to numerical ratings), in my estimation. You may not have the same palate as someone else, but it is useful to read their research and the story behind that cigar and how it was aged, what tobacco was used, and what interesting techniques were used in aging or rolling or storing, or whatever. It's even good to read their descriptions of the flavors and body of the cigar: this gives you a baseline on which to compare your own experiences. Whether you agree or disagree, you have a target to aim at. This just makes you a more informed consumer. Of course, like any consumer, you must decide which of the factors are most important to you, and which reviewers are the most knowledgeable and trustworthy. Nevertheless, having more information is always a good thing, in my opinion.

[tx_tuff]

I see both sides of the picture, and there is no doubt that when I am buying a cigar I want to know what tobaccos are used and where it is made etc....... I am really enjoying this one Rob!

[bobbyg29-cl]

Originally Posted by tx_tuff

I see both sides of the picture, and there is no doubt that when I am buying a cigar I want to know what tobaccos are used and where it is made etc....... I am really enjoying this one Rob!

Thanks Frank...I appreciate it!

[bobbyg29-cl]

Originally Posted by baboruger

For example, I picked up a Padilla Miami 8&11 because of all of the things I had heard about it, and I was really sort of disappointed. I would not really spend that money again. Looking at the blind reviews on Cigar live for that same cigar, I think the average was like a 5.8 or something, meaning ok but not great. The think I found funny about this was that people who listed this smoke as one of their favorites gave really bad reviews for it. Do I think that the blind review was different in taste, no, but it was the truer of the two reviews, not influenced by anything other than the smoke.

I think this a great example of the point I am trying to make Brent.
In the review panel there were people that had smoked the Miami 8/11 before and enjoyed it enough to call it one of their favorites yet when they smoked it blind they gave it a low score. Doesn't that tell you that blind reviews are not really useful? Should the people that really enjoyed this cigar before and chose it as a favorite suddenly now stop smoking it because they gave it a bad blind rating score? No, because when they smoke in the normal, non-blind way again they will presumably enjoy it again. Were their enjoyable experiences they had with the Miami 8/11 before the blind review somehow invalid because expectations about the cigar were involved? No, I don't think so. What matters is whether we enjoy a cigar or not when we smoke it in the typical non-blind way (which they did). And blind reviews just do not give a good indication of this.

And, interestingly enough, I was one of the reviewers on the blind panel with the Miami 8/11. I also gave it a very poor score (4.8/10). I can almost guarantee you that I would have enjoyed this cigar more and given it a higher score if I had done the rating non-blind. The fact that I am a huge Pepin fan combined with this cigar's price and positive reviews I have read about it about this cigar would have generated expectations that would have altered my perception of flavor in the cigar. So for me, my blind review is not a good indicator of how much I will enjoy the cigar when I eventually buy one myself and smoke and it is not going to deter me at all from trying more 8/11's

We seem to have this view that being influenced by things other than the tobacco in a cigar is a bad thing. I just don't understand why. Our perceptions are distorted by the context all the time in life. It is normal part of being a human being. And even if we do think these effects are bad they are unavoidable in normal, non-blind cigar smoking (unless you take all the bands off your cigars at home and pick one randomly each time) and need to be taken into account.

[tx_tuff]

Originally Posted by bobbyg29

The fact that I am a huge Pepin fan combined with this cigar's price and positive reviews I have read about it about this cigar would have generated expectations that would have altered my perception of flavor in the cigar. So for me, my blind review is not a good indicator of how much I will enjoy the cigar when I eventually buy one myself and smoke and it is not going to deter me at all from trying more 8/11's

Did you know that Pepin is not making cigars for Padilla anymore! I heard this at the RTDA. Also I believe it was in the new CA that I read Oliva is making cigars for Padilla, I don't know if its just one blend or all of them. Does anybody know about this?

[Doc Stogie Fresh]

Originally Posted by tx_tuff

Did you know that Pepin is not making cigars for Padilla anymore! I heard this at the RTDA. Also I believe it was in the new CA that I read Oliva is making cigars for Padilla, I don't know if its just one blend or all of them. Does anybody know about this?

This may be true, however, the new Padilla 1948, that released at the recent RTDA, was blended and rolled by Pepin and company.

[tx_tuff]

Cool, I didn't get one of those, they had handed them all out by the time I got there LOL. But I did get the 1932? Is it the same blend but different size? It was a great stick.

[Doc Stogie Fresh]

Originally Posted by tx_tuff

Cool, I didn't get one of those, they had handed them all out by the time I got there LOL. But I did get the 1932? Is it the same blend but different size? It was a great stick.

Different blend. And better than the 1932 IMHO.