Subjective sound quality and distortion metrics

morricab

Well-Known Member
Apr 25, 2014
7,027
3,032
653
Switzerland
It has been known for quite some time that the order of the harmonics should be weighted in order to get a clearer picture on the perceptibility of the distortion an amplifier (preamp, DAC, phonostage etc.). One of the more important findings was from D.E.L Shorter, a BBC engineer who published his findings in 1949.

From a 1949 paper by Shorter:

"It has long been recognized that high order harmonics are more offensive than low and it was thought that if allowance for this factor were made,
the discrepancies of Table 1 might be accounted for. The amplitude of each measured harmonic 'was therefore weighted by a factor proportional to its order relative to the 2nd, a procedure which was laid down in 1937 in an R.M.A specification for testing broadcast receivers though not widely adopted so far as is known. According to this procedure, the amplitude of the nth harmonic is multiplied by n/2 so that the figure for the 2nd
harmonic is unchanged, The result is given in Table II in which the systems are again presented in decreasing order of the distortion figure. It will be seen that the balance is now partially redressed but that systems A and B appear to be equally bad, which is far from being the case.

A more drastic weighting was therefore tried and the result is shown in Table III, in which the systems are arranged in decreasing order of the distortion figure obtained by multiplying the nth harmonic by n2/4 (it will be noted that the 2nd harmonic is again unaffected.) This time, the order of merit obtained from the distortion figures confirms that obtained by subjective tests and the difference between systems A and B is clearly
brought out."

Shorter then goes on:

"The importance of high order harmonics even when small in energy content individually, suggests that distortion should not be regarded so much in terms of the separate harmonics as of the complete series or the total waveform which this series represents. "

Furthermore:

"When the amplitude characteristic of a transmission system contains a discontinuity, the number of alien frequencies generated under programme conditions must be very 1arge: and it is therefore important that this form of distortion should be recognized whenever it occurs and objective tests so framed as to give appropriate numerical expression to its effects."

Cheever summarizes the history of trying to correlate subjective sound quality and proposes his own metric based on recent (as of about the year 2000) knowledge around human hearing, self-harmonic generation, increasing self-distortion with increasing SPL and weighting of harmonics in distortion.


What I find interesting is that Shorter comes to the conclusion that it is the whole waveform, as represented by the harmonic series in an FFT, that is important to perception and not just single harmonics.

This is echoed by Cheever and Geddes, who also developed a relative sophisticated metric that he found correlated well with controlled listening tests.



From Keith Howard's STereophile article:

"The most important finding was that none of the different patterns of nonlinearity sounded in any way preferable to the undistorted reference. They all sounded worse, albeit in different ways. Pattern 1 added a distinct "dirtiness" to the sound that was not unpleasant but did change the instrumental timbre and diminish the sound's sense of fidelity—there was something clouding the sound. Pattern 2 was much better, with the closest sound to the undistorted reference. But I thought I could also detect it just beginning to muddy the presentation. Pattern 3 was unpleasant, adding an edge to the sound that would surely become fatiguing over extended listening. Pattern 4 wasn't as bad, but there was still something unnatural about it. Although Patterns 3 and 4 both introduce less distortion than Pattern 1, it was Pattern 1 that proved less subjectively annoying. While I could hear its effect, it did not threaten to send me screaming from the room after 10 minutes' listening."

For reference Pattern 1 had descending amplitude even and odd orders from 2nd to 9th with a high THD of 3.33%
Pattern 2 was the same but with 20db lower distortion per harmonic and THD of 0.33%
Pattern 3 was only odd order at the same level as Pattern 1 THD 1.00%
Patter 4 was only even order at the same level as Pattern 1 THD 3.18%

No surprise that Pattern 1 was a bit "dirty" sounding... I would expect that level to be audible

Interestingly the 2nd lowest distortion was easily the worst sounding (all odd... like most push/pull amplifiers) and even the all even was next to last in sound quality.

"The GedLee metric was a much better indicator, although my listening suggested that it underestimates the relative subjective annoyance of Pattern 4."

This is interesting as it shows there are gaps in Geddes metric indicating it is not subtle enough to capture the real subjective impacts.

Of course most modern amps are much lower in distortion but where does the limit of audibility really lie? Even amps of similar very low THD and IMD subjectively sometimes sound VERY different and usually always somewhat different.

Keith concludes:
"There is a little support in these findings for Hiraga's thesis, in that Pattern 1—with its full complement of even- and odd-order harmonics—indeed proved preferable to the denuded harmonic Patterns 3 and 4. But neither Pattern 1 nor Pattern 2 sounded preferable to the undistorted reference. I entreat those with a particular interest in this issue to repeat my experiments and judge for themselves, but for me the issue is now settled."

Of course truly zero distortion is preferable but as such devices don't exist it seems that Hiraga and others about the pattern were basically correct that this is preferable to other patterns of distortion generated by modern amp concepts.
 
  • Like
Reactions: tima and kinch

Site7000

Well-Known Member
Jul 16, 2011
46
46
403
It has been known for quite some time that the order of the harmonics should be weighted in order to get a clearer picture on the perceptibility of the distortion an amplifier (preamp, DAC, phonostage etc.). One of the more important findings was from D.E.L Shorter, a BBC engineer who published his findings in 1949.

From a 1949 paper by Shorter:

"It has long been recognized that high order harmonics are more offensive than low and it was thought that if allowance for this factor were made,
the discrepancies of Table 1 might be accounted for. The amplitude of each measured harmonic 'was therefore weighted by a factor proportional to its order relative to the 2nd, a procedure which was laid down in 1937 in an R.M.A specification for testing broadcast receivers though not widely adopted so far as is known. According to this procedure, the amplitude of the nth harmonic is multiplied by n/2 so that the figure for the 2nd
harmonic is unchanged, The result is given in Table II in which the systems are again presented in decreasing order of the distortion figure. It will be seen that the balance is now partially redressed but that systems A and B appear to be equally bad, which is far from being the case.

A more drastic weighting was therefore tried and the result is shown in Table III, in which the systems are arranged in decreasing order of the distortion figure obtained by multiplying the nth harmonic by n2/4 (it will be noted that the 2nd harmonic is again unaffected.) This time, the order of merit obtained from the distortion figures confirms that obtained by subjective tests and the difference between systems A and B is clearly
brought out."

Shorter then goes on:

"The importance of high order harmonics even when small in energy content individually, suggests that distortion should not be regarded so much in terms of the separate harmonics as of the complete series or the total waveform which this series represents. "

Furthermore:

"When the amplitude characteristic of a transmission system contains a discontinuity, the number of alien frequencies generated under programme conditions must be very 1arge: and it is therefore important that this form of distortion should be recognized whenever it occurs and objective tests so framed as to give appropriate numerical expression to its effects."

Cheever summarizes the history of trying to correlate subjective sound quality and proposes his own metric based on recent (as of about the year 2000) knowledge around human hearing, self-harmonic generation, increasing self-distortion with increasing SPL and weighting of harmonics in distortion.


What I find interesting is that Shorter comes to the conclusion that it is the whole waveform, as represented by the harmonic series in an FFT, that is important to perception and not just single harmonics.

This is echoed by Cheever and Geddes, who also developed a relative sophisticated metric that he found correlated well with controlled listening tests.



From Keith Howard's STereophile article:

"The most important finding was that none of the different patterns of nonlinearity sounded in any way preferable to the undistorted reference. They all sounded worse, albeit in different ways. Pattern 1 added a distinct "dirtiness" to the sound that was not unpleasant but did change the instrumental timbre and diminish the sound's sense of fidelity—there was something clouding the sound. Pattern 2 was much better, with the closest sound to the undistorted reference. But I thought I could also detect it just beginning to muddy the presentation. Pattern 3 was unpleasant, adding an edge to the sound that would surely become fatiguing over extended listening. Pattern 4 wasn't as bad, but there was still something unnatural about it. Although Patterns 3 and 4 both introduce less distortion than Pattern 1, it was Pattern 1 that proved less subjectively annoying. While I could hear its effect, it did not threaten to send me screaming from the room after 10 minutes' listening."

For reference Pattern 1 had descending amplitude even and odd orders from 2nd to 9th with a high THD of 3.33%
Pattern 2 was the same but with 20db lower distortion per harmonic and THD of 0.33%
Pattern 3 was only odd order at the same level as Pattern 1 THD 1.00%
Patter 4 was only even order at the same level as Pattern 1 THD 3.18%

No surprise that Pattern 1 was a bit "dirty" sounding... I would expect that level to be audible

Interestingly the 2nd lowest distortion was easily the worst sounding (all odd... like most push/pull amplifiers) and even the all even was next to last in sound quality.

"The GedLee metric was a much better indicator, although my listening suggested that it underestimates the relative subjective annoyance of Pattern 4."

This is interesting as it shows there are gaps in Geddes metric indicating it is not subtle enough to capture the real subjective impacts.

Of course most modern amps are much lower in distortion but where does the limit of audibility really lie? Even amps of similar very low THD and IMD subjectively sometimes sound VERY different and usually always somewhat different.

Keith concludes:
"There is a little support in these findings for Hiraga's thesis, in that Pattern 1—with its full complement of even- and odd-order harmonics—indeed proved preferable to the denuded harmonic Patterns 3 and 4. But neither Pattern 1 nor Pattern 2 sounded preferable to the undistorted reference. I entreat those with a particular interest in this issue to repeat my experiments and judge for themselves, but for me the issue is now settled."

Of course truly zero distortion is preferable but as such devices don't exist it seems that Hiraga and others about the pattern were basically correct that this is preferable to other patterns of distortion generated by modern amp concepts.
Nice find. It would be very interesting to see these same tests repeated with modern, lower distortion speakers and electronics.
 

tima

Industry Expert
Mar 4, 2014
4,383
4,801
945
the Upper Midwest
Interesting, Brad.

What is Hiraga's thesis?

It would be enlightening to me to hear sonic examples. How does the distortion described by metrics correlate to what we hear? ...maybe that's a different thread.
 

morricab

Well-Known Member
Apr 25, 2014
7,027
3,032
653
Switzerland
Interesting, Brad.

What is Hiraga's thesis?

It would be enlightening to me to hear sonic examples. How does the distortion described by metrics correlate to what we hear? ...maybe that's a different thread.
Hi thesis revolved around amps sounding best when they have a monotonic distortion pattern. By this he meant an exponential decay from low order to high order. Cheever found the same thing and that it should be somewhere around 20dB drop in amplitude per step in harmonic order.
 
  • Like
Reactions: tima

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing