Cigar Weekly blind reviews

[Mad Dawg]

Last summer, I took on the task of managing the formal blind reviews that have been a part of Cigar Weekly since the beginning. A total of 7 reviews were published in 2008, and we'll certainly pass that number in 2009. Our esteemed editor TommyBB suggested that it would be a good idea to post some key information about the reviews, so here goes!

The process usually starts when a cigar manufacturer contacts one of CW's owners or editors and indicates an interest in getting some feedback on one of their products. We ask them to provide a minimum of 16 cigars so that I can send 2 each to our 7-member tasting panel, and try a couple myself.

When the cigars get to me, I remove all bands and other packaging (tubos, etc.) that could identify the cigar and send a pair to each of our reviewers. We always want to send two cigars in case one of them is a stinker. You know the one...that crappy cigar that occasionally slips past the quality control process? We send two so that the manufacturer gets a second chance in case we get one of those. It has happened.

In any case, we send a pair to each of our reviewerss Our panel currently consists of 6 permanent members drawn from CW's ranks:

TommyBB
ronin
SLR
flipflop
Fishin' Phil
Elk Twin

Those guys provide a degree of continuity to the reviews.

Beginning in late 2008, we added four rotating members to our tasting panel in order to broaden the base on which our ratings are provided. Our current rotating members are:

AnejoMofo
Augie754
Black Plague
H311oLHD

Those guys provide new and varying input to the reviews.

When they smoke the review cigars, our panelists fill out a form that captures standardized numerical scores and allows free-form comments in three key areas: Pre-Smoke, Smoke, and Summary. Elements like construction, burn, ash, flavor, aroma, and many others, are ranked by each reviewer.

The reviewers then send their completed forms to me. I compile the numerical scores in a spreadsheet I built for the purpose, and transfer their comments (unedited) into my writeup. The numerical scores are totaled for each reviewer, and the high and low scores are tossed out. The average of the remaining scores is the basis for the CW Star rating that we assign each cigar.

Cigar Weekly’s 50 Point Scale:
0 – 9.......1 star...........Not Recommended
10 - 19....2 stars..........Below Average
20 - 29....3 stars..........Average
30 - 34....3 1/2 stars.....Above Average
35 - 39....4 stars...........Excellent
40 - 44....4 1/2 stars.....Outstanding
45 - 50....5 stars...........Superior

I never score the cigars myself, because I necessarily know what they are and I don't want to introduce any question of bias into the final rating. I have offered up my own impressions of reviewed cigars in my write-ups from time to time, but any mental scoring I do is never counted in a cigar's formal rating.

This blind review system is designed to give cigar manufacturers unbiased feedback regarding the quality of their products. By removing any identifying packaging from the cigars, we remove the possibility of brand-based bias and enable the reviewers to focus solely on how the cigars look, feel, smell, and taste. That feedback has proven to be very valuable to manufacturers, and has helped to prevent at least one factory mistake from becoming a PR nightmare.

The finished reviews are sometimes published on the front page, but eventually they all find their way to the Reviews page: http://www.cigarweekly.com/magazine/reviews

The owners, administrators, editors, and moderators of Cigar Weekly hope that these reviews provide useful information for the members of the CW community, as well as to the manufacturers who provide cigars for the reviews.

As a final note, I'd like to say that quality reviews - in any form - are always welcomed and appreciated in this room. We read them all, and in fact that's how we identified potential candidates for the rotating panelist positions. We're always looking for good reviewers.