Certainly Uncertain: When Measurement Results Don’t Match

One of the greatest misconceptions in all of Audiophiledom is the notion that measurements are objective.

We can break the subjective intrusions on the measurement process into four main components:

  1. The person performing the measurements must choose their measurement tool(s)
  2. The person performing the measurements must choose WHAT to measure and HOW to measure (e.g. loudspeakers can be measured in ‘free air’ or in an anechoic chamber)
  3. The person performing the measurements must then choose what gear will be used to provide comparative data points
  4. MOST IMPORTANT OF ALL, The person performing the measurements must interpret their results

My interest in this subject was piqued by an Instagram post related to the Melco S100 Network Switch and the Editor’s Choice Award it received from HiFi News. And I thought, interesting. Interesting because HiFi News includes measurements along with the review proper, and measuring the effects of an Ethernet switch struck me as a very intriguing proposition.

Paul Miller, Editor of HiFi News, measured the performance of the Melco S100 Network Switch (£2000) and presented his results as follows:

The Mytek and Arcam DACs exhibit superb jitter rejection by default and the impact of the S100 was at the limits of measurement (~1psec). The Lumin D2 did resolve a repeatable difference, however, with correlated jitter reduced from 15psec to 10psec and uncorrelated noise squeezed by 0.4dB over a 146dB range.

The reviewer, Andrew Everard, whose listening results were presented as part of the full review along with Miller’s measurements, used a Naim ND 555 Network Player with the Melco. He concluded:

…if you’ve invested heavily in network playback, and want a simple, easy to install upgrade for your system, the S100 has much to recommend it. It just works, and can bring major sonic gains to any Ethernet-based set-up.

These results led me to search for other reviews of so-called audiophile Ethernet switches which brought me to Amir Majidimehr of Audio Science Review (ASR) and his YouTube video review of two audiophile Ethernet switches, the UpTone Audio EtherREGEN Switch ($640) and the Silent Angel Bonn N8 Audio Grade Ethernet Switch ($399). This was my first encounter with one of Amir’s video reviews and I found it very informative and entertaining in the best sense — Amir’s enthusiasm and excitement for the subject at hand are obvious and add interest to what otherwise could come across as a rather dull subject.

As you will see in his video, Amir found no measurable difference between these audiophile Ethernet switches and a bog standard $20 switch. He concludes:

The message here is very, very clear — these devices…cannot make a difference except for a very very very remote possibility — and that remote possibility is maybe with the junkiest DAC streamer you could ever come up with….

Here we have two people measuring the same type of device, audiophile Ethernet switches, using different test equipment and different associated gear who come up with different test results and conclusions. Any number of reasonable arguments can be made in an attempt to explain these differences, pick any of the above bullet points as a starting point, but the most important, at least to my mind, are the differing conclusions.

On one hand, Amir suggests that psychoacoustic phenomena explain why people hear a difference where he believes differences cannot reasonably exist, while Paul Miller finds measured results that show minor improvements in correlated jitter and uncorrelated noise.

HiFi News awarded the Melco S100 their Editor’s Choice Award while Amir suggests all audiophile Ethernet switches are ineffective and therefore a waste of money.

As is so often the case when digging into these matters, we find ourselves at the beginning of a conversation rather than at the end. Do Paul Miller’s measured improvements suggest that the Melco switch can audibly improve the sound coming out of the attached network DAC? His comments relating to the measurements don’t say, one way or another, but the reviewer is firm in his observation — with the Melco S100 simply revealing more of everything you play. Or is Amir correct in dismissing the notion that an Ethernet switch can improve sound quality at the output of the attached network DAC, a conclusion he’s obviously very comfortable sharing.

While I don’t have an answer, I do have an observation based on experience that suggests no one will be swayed from their current beliefs by either conclusion. Those inclined to look at the world of hifi with suspicion — and in some cases derision — will side with Amir. Self-proclaimed audiophiles who enjoy reading reviews, even reviews without measurements, will take the HiFi News results as all the proof they need.

“A man will be imprisoned in a room with a door that’s unlocked and opens inwards; as long as it does not occur to him to pull rather than push” ~ Ludwig Wittgenstein