Certainly Uncertain Part 3: Listen | Measure

True or False: The best music is the music with the greatest dynamic range.

False. This is a false proposition because focusing on dynamic range ignores the quality of the music. To put it another way, would you prefer to listen to music you hate yet exhibits great dynamic range, or music you love that doesn’t measure as well? That’s not even a real question.

Some people will have you believe that measurements of hifi gear, or perhaps even a single metric, are all you need to know in order to determine the best hifi gear. As is the case with music, suggesting that measurements alone determine the best hifi gear is also a false proposition since it fails to take sound quality into account.

Do you remember the original Ayre QB-9 DAC, released in 2009? It included a rear-panel DIP switch that offered two settings — Listen and Measure. Why would Ayre include a switch on a commercial product that most buyers wouldn’t use? The short answer: reviews. The late Charley Hansen, the QB-9’s designer, knew that his DAC would be reviewed, and in the case of Stereophile, John Atkinson would put the QB-9 through its paces on the test bench. To avoid the appearance of a poorly-designed DAC, Charley put in the Listen | Measure switch so that the QB-9 offered good measured results and good listening results.

According to Wes Phillips’ review and John Atkinson’s measurements, Charley succeeded. [footnote 1]

Schiit Audio has on offer two headphone amplifiers that cost the same $99.00 and they also share 1/2 of their name — Magni 3+ and Magni Heresy. Here’s Schiit on the differences:

Choose Magni 3+ for the ultimate expression of an affordable all-discrete current-feedback headphone amp.

Choose Magni Heresy for insanely great measurements from an all-op-amp based headphone amp that uses super high quality parts, including a multiple paralleled output stage with feedforward.

Why offer two models? The answer is to appeal to two different types of reviewers and buyers — those that rely on listening impressions perhaps in conjunction with measurements (Magni 3+), and those that rely on measurements alone (Magni Heresy).

iFi Audio also offers a Listen and Measure option in their nano iDSD Black Label Portable DAC, which takes a similar approach to the Ayre QB-9 in that the modes employ different digital filters.

It is clear that iFi, Schiit and Ayre do not believe that the circuit that measures the best is the best sounding. The design that measures best does not sound as good as the design that measures comparatively worse. By offering these options, iFi, Schiit and Ayre also short-circuit a common argument found among measurement-first proponents: that designer incompetence is to blame for gear that doesn’t measure up to their measurement standards. [footnote 2]

Running on par with our music discussion, would you rather listen through a hifi you enjoy listening to, or one that doesn’t sound as good but offers better measured results? That is a real question.

Getting back to music, imagine someone trying to tell you what music you should listen to based exclusively on dynamic range. Silly, right? But if you’re looking for the best release of a specific album, knowing the dynamic range of different releases can help inform your purchase in a meaningful manner.

Measurements, when coupled with clear interpretations — like dynamic range scores for music releases — can give hifi buyers a certain degree of certainty. It also makes the purchase process much less time consuming as compared to listening by replacing ears with eyes.

To be clear, measurements are an essential aspect of product design and development, and without this applied science, we’d be lost in a sea of uncertainty. But a measurement-first approach puts the cart before the horse, throws the horse away, and stinks of scientism not science. The notion that anyone with a spectrum analyzer and spare time can come to know more about hifi design than hifi designers is, to put it politely, a stretch. If you extend this presumed knowledge to the people who read measurements and pass judgment on hifi design and quality using their eyes alone, that stretch becomes a leap of faith.

Without a clear understanding of the design goals behind a product, coupled with time spent listening, measurement results that definitively declare winners and losers based on a few metrics are misleading at best, and a gross misunderstanding of the basics of determining sound quality.

“…it is as much fantasy to expect that measurements can predict sound quality.” John Atkinson

We are, when all is said and done, talking about personal preferences when it gets down to choosing hifi gear. Using measurements as one’s sole guide when deciding what to buy is as good a reason as basing purchase decisions purely on listening. Lines are crossed when one passes judgement on the other, believing their way is best.

Seeing as both approaches in their extreme form — just measure or just listen — fail to grasp the complete picture of sound quality and design quality, meeting somewhere in between the extremes is where reason resides.

  1. It’s worth noting that Stereophile’s approach is to offer listening impressions and measurements as part of every review, where the former occur without knowledge of the latter. Bias works both ways.
  2. Some measurement-first proponents typically mix in an unhealthy dose of conspiracy theorizing as part of the interpretation of measured results for reasons I can only attribute to psychology.