"What The Specs Don’t Tell You… And Why"

AP Jonathan

New Member
Jun 6, 2011
8
0
0
Hi All,

I hope you don't mind me reopening this thread but I just found it. I am the author of the RMAF talk and I am humbled by the amount of discussion this has generated. There are some very valid comments here. I apologize that my talk didn't offer any definitive conclusions. My goal for this talk is to get people subjectivists and objectivists to hold respectful dialog. There is only so much that I can accomplish in one hour and keep the topic somewhat engaging at the same time.

I am a governor of the AES and I hear a lot of mockery of the audiophile community among the scientific community. I think this is because many hi-fi vendors have hypothesized why their gear sounds better and have vociferously claimed these hypothesis to be fact without establishing any scientific proof. There is a big difference between hypothesis and fact. However, it is equally wrong of the scientific community to outright dismiss one's observations without providing equal scientific proof that those differences don't exist. Both sides need to work together and admit that there are still things we don't yet understand about the recording and playback of audio sources. Having given versions of this paper to many AES members, I know I have raised considerable awareness to the deficiencies in commonly reported specs.

IMHO, one of the biggest problems we have is differentiating between correlation and causality. I can say that water causes cancer. All persons that die of cancer have ingested water. And, if we deny water, those patients won't die of cancer. They'll die of dehydration. But look how well cancer deaths correlates to water ingestion. This extreme example shows that we can have a high degree of correlation that has no relevance on causality. The water didn't cause the cancer. History is filled with examples of correlation being confused with causality.

To me, it is perfectly acceptable to say that something works but the reason why remains a mystery. It is also perfectly acceptable to propose a hypothesis on why something works. In fact, I believe when something is stated as merely a hypothesis, it invites further research. We can certainly benefit from more research. Once research confirms causality, we can then work to quantify differences between products.

Please feel free to ask questions. I will try to check back and answer when I can.
 
Last edited:

dallasjustice

Member Sponsor
Apr 12, 2011
2,067
8
0
Dallas, Texas
Thanks for posting here Jonathan. I really enjoyed your presentation. I hope you do it again. I'll make sure Amir doesn't get ahold of the the mic next time. :D

I think there's flawed think in both camps. For example, measuring audio electronics is important but the electronics' contribution to what the listener hears is very small when compared to the loudspeaker. But you often see folks make the logical error called post hoc ergo propter hoc. IOW, just because something happens before something else in a sequence, that must be the thing causing the event. Although electronics measurement is valuable, IMO, it's overrated.

Michael.
 
Last edited:

FrantzM

Member Sponsor & WBF Founding Member
Apr 20, 2010
6,455
29
405
That's exactly why I thought, at the time, (and still do) that this presentation can help bridge the gap between the two camps. It certainly opened my eyes to how useless some specs really are and how easy it is for manufacturers to manipulate specs in their favor. In addition, it's easier for me to understand why I can still really enjoy listening to systems which don't sport the best specs. In particular, the important difference between cross-over distortion and clipping is very enlightening. It's clear to me there's still much to be learned about distortion. This means that those who claim to hear distortions which may not be readily apparent from basic testing should NOT be dismissed.

Having said all of that, I still believe that ALL real distortion phenomena can be measured. It's just a question of finding the proper measurement. That's the disconnect between myself and some others. For me, there's really no point in talking to someone who says they can hear a distortion but refuses to accept that what they are hearing can be measured. I hear what they are saying, but I won't ever understand them. ;)

Revisited this thread and find myself in complete agreement with you . Especially the second paragraph.

and AP Jonathan

Thanks for an an enlightening post. Would like to see post more.
 

AP Jonathan

New Member
Jun 6, 2011
8
0
0
Thanks for posting here Jonathan. I really enjoyed your presentation. I hope you do it again. I'll make sure Amir doesn't get ahold of the the mic next time. :D

I think there's flawed think in both camps. For example, measuring audio electronics is important but the electronics' contribution to what the listener hears is very small when compared to the loudspeaker. But you often see folks make the logical error called post hoc ergo propter hoc. IOW, just because something happens before something else in a sequence, that must be the thing causing the event. Although electronics measurement is valuable, IMO, it's overrated.

Michael.

Hi Michael, I appreciate that you have used the word "overrated" rather than "irrelevant". For experiments to be scientifically valid, they must be repeatable. Repeatability can be difficult to establish in the real world. We measure speakers in anechoic chambers and measure gear with sine waves because these approaches make results very repeatable. Unfortunately, the results are also less representative of the real world.

My talk focused on amplifier distortion. I never mentioned in the talk that all measurements were made into purely resistive loads. Real speakers are anything but purely resistive. I don't think the world will ever agree on a "standard" speaker load for testing amps. However, there are companies like AudioGraph that make load boxes that can test over a wide range of loads. These load boxes can be more expensive than the audio analyzers they are used with. In other words, we have the technology and know-how to do more meaningful load testing but price is currently an obstacle to widespread adoption.

Humanity's collective scientific knowledge 100 years ago pales to the knowledge we have today. Things that were impossible then are now ho-hum occurrences. We also know that our rate of new discovery is constantly increasing as well. This leads me to believe that our problems with life-like recording and playback experiences will eventually be mastered. I just can't say when.
 
Last edited:

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Very good posts, AP
I like what you said about test signals for measurements being those that are repeatable & as a result less like the dynamic, signals that the device is expected to handle when in operation. Unrealistic loads is another good point & one that applies throughout where measurements are often done on individual devices rather than devices & their system-wide interactions.

As regards measurements, I believe advances in this area will result from better understanding of psychoacoustics which can be used to interpret data (the measurements) into useful information. This will probably be achieved by the use of algorithms incorporating this psychoacoustic knowledge & post-processing the data into meaningful insights

An example of such will be found in this 1994 paper from Bob Stuart "NOISE: METHODS FOR ESTIMATING DETECTABILITY AND THRESHOLD."
 

KlausR.

Well-Known Member
Dec 13, 2010
291
29
333
Hi,

As regards measurements, I believe advances in this area will result from better understanding of psychoacoustics which can be used to interpret data (the measurements) into useful information. This will probably be achieved by the use of algorithms incorporating this psychoacoustic knowledge & post-processing the data into meaningful insights

In the area of room acoustics psychoacoustic data which can be used to correlate measurements to what we hear do exist, yet what users and acousticians do is measure a simple in-room response and base any judgement and advice on that simple measurement of absolute sound pressure level, without any consideration whatsoever of these psychoacoustic data: equal-loudness contours and perceived loudness, binaural de-coloration thresholds, direction depending head-related-transfer function/in-ear frequency response, none of this is taken into account. They basically completely ignore the fact that human hearing has a radically different way of processing data than a measurement microphone.

Klaus
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Hi,



In the area of room acoustics psychoacoustic data which can be used to correlate measurements to what we hear do exist, yet what users and acousticians do is measure a simple in-room response and base any judgement and advice on that simple measurement of absolute sound pressure level, without any consideration whatsoever of these psychoacoustic data: equal-loudness contours and perceived loudness, binaural de-coloration thresholds, direction depending head-related-transfer function/in-ear frequency response, none of this is taken into account. They basically completely ignore the fact that human hearing has a radically different way of processing data than a measurement microphone.

Klaus

Yes, Klaus, maybe it's just advances in application & understanding that is needed. Perhaps, we have all the necessary measurements we need but they are just being ignored or applied in uninformed ways. But I'm also of the opinion that we have a lot to learn about psychoacoustics & the area of Auditory Stream Analysis (ASA) or how we interpret & make sense of the auditory world. The further learnings in this area can only benefit this hobby of audio illusion production (I have decided to stop calling it audio reproduction)

Have you got any examples of room acoustics & psychoacoustic data to illustrate your point?
 
Last edited:

Fitzcaraldo215

New Member
Nov 3, 2014
394
2
0
Hi,



In the area of room acoustics psychoacoustic data which can be used to correlate measurements to what we hear do exist, yet what users and acousticians do is measure a simple in-room response and base any judgement and advice on that simple measurement of absolute sound pressure level, without any consideration whatsoever of these psychoacoustic data: equal-loudness contours and perceived loudness, binaural de-coloration thresholds, direction depending head-related-transfer function/in-ear frequency response, none of this is taken into account. They basically completely ignore the fact that human hearing has a radically different way of processing data than a measurement microphone.

Klaus

I agree that mikes do not hear like we humans do. But, there is some recognition of that in the typical downward sloping with increasing frequency target curves used by most DSP Room EQ systems.

Also, varying equal loudness contours with volume level have been built into Audyssey's Dynamic EQ feature for some time, not that I am a fan of Audyssey, which I formerly used myself. But, with that exception, it is true that most EQ tools do not adjust frequency response for different volume levels. All audio systems have been doing that for decades, ever since the old "dumb" and useless Loudness Compensation switch, which was a ridiculous oversimplification, was eliminated from preamps and receivers decades ago. Good riddance!

Head transfer functions and in ear response are quite variable and specific to one individual. So, it would not be a good idea to EQ or acoustically treat an entire room for one person's particular response for that and other good reasons. With headphones, yes. See the Smyth Realizer 8 headphone system which does precisely that.

I think you are oversimplifying the issue and what acousticians and EQ tools actually do measure, which is beyond just frequency response. But, frequency and time domain response are the biggest factors in the distortions that listening rooms create, unless there is reliable evidence to the contrary, which there is not. Correcting just for those is a huge improvement, as you can demonstrate for yourself by trying one of numerous DSP EQ tools. Dirac, for example, measures and corrects both the frequency and time domains. The tools may, like all things, be imperfect, but you do not want to talk yourself into dismissing them as a result: the perfect is the enemy of the good! Also, with Dirac, one can if one wishes create multiple target curves, easily selectable during playback for different volume levels or your own specific, preferred system voicing.

And, DSP Room Correction is still a very young technology. It has come a long way in the past decade, and it will likely further improve as computational speeds and capacities further improve.
 

KlausR.

Well-Known Member
Dec 13, 2010
291
29
333
Yes, Klaus, maybe it's just advances in application & understanding that is needed. Perhaps, we have all the necessary measurements we need but they are just being ignored or applied in uninformed ways.

I don't think we have all the necessary psychoacoustic data in terms of measurements correlated to perception, but the ones which are available today could be used. What I find rather striking is that the very center of this whole room acoustic exercise, i.e. the listener, is not considered at all!

Have you got any examples of room acoustics & psychoacoustic data to illustrate your point?

As a start, the equal-loudness contours: a 10 dB peak at frequency X and a 5 dB dip at frequency Y could be perceived as being equally loud. Of course, those curves are averaged so may not apply to the individual listener. Another road to explore may be the works of Toole on resonances (JAES 1988, p.122) and Bücklein on the audibility of frequeny response irregularities (JAES 1981, p.126), can’t remember if they have determined thresholds.

First reflections: there are many data on perception thresholds, but not for the scenario 2- (or multi-) channel with music as signal. The difficulty here is that human hearing is directional (different in-ear frequency responses for different directions, see Shaw, JASA 1965, p. 465), while a mike is not. Further, a late first reflection will be perceived as an echo while for the mike it makes no difference. Despite all that people measure then go and treat their reflections, most likely with absorbers which don’t absorb but only modify the spectrum (see https://www.audioholics.com/room-acoustics/room-reflections-human-adaptation).

Binaural de-coloration: in 1995 Salomons wrote her PHD thesis with the goal to correlate measurements to perception thresholds so that measurements in performance spaces would tell you whether or not the sound would be perceived as colored:

http://www.google.nl/url?sa=t&rct=j...CAsxXuwopzOb_Mv5w&sig2=BXP7yG3Phhgd4YgQ3Kk5pw



Klaus
 

cooljazz

Well-Known Member
Mar 21, 2012
28
0
76
TN
Somewhere on the web in the last couple of years, I ran across a mention in a thread, of a paper that talked about many types of distortions. I have not ever been able to find that paper or the mention of it again. Anyone know of it?

It was a fairly recent paper I think. What I liked was that it talked about the very many things that make audio have distortions that goes beyond the basic beginner THD numbers.

It might add to the discussion for some.

CJ
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Somewhere on the web in the last couple of years, I ran across a mention in a thread, of a paper that talked about many types of distortions. I have not ever been able to find that paper or the mention of it again. Anyone know of it?

It was a fairly recent paper I think. What I liked was that it talked about the very many things that make audio have distortions that goes beyond the basic beginner THD numbers.

It might add to the discussion for some.

CJ

Are you thinking of this Schiit post on Head-fi where he details the battery of measurements used in design & QA? He ends with this quote:
"So…while we putter around confidently with all of the accepted measurements, maybe there are still realms out there where “here be there monsters.”

That’s why we still listen. And measure. And come up with new measurements. And listen again."​
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
BTW, looking at that video again, I see someone in the middle of the audience who looks very like Ethan & who never raises his hand at any stage to register that he has perceived distortion :)
 

marty

Well-Known Member
Apr 20, 2010
3,025
4,173
2,520
United States
Hi All,

I hope you don't mind me reopening this thread but I just found it. I am the author of the RMAF talk and I am humbled by the amount of discussion this has generated. There are some very valid comments here. I apologize that my talk didn't offer any definitive conclusions. My goal for this talk is to get people subjectivists and objectivists to hold respectful dialog. There is only so much that I can accomplish in one hour and keep the topic somewhat engaging at the same time.

I am a governor of the AES and I hear a lot of mockery of the audiophile community among the scientific community. I think this is because many hi-fi vendors have hypothesized why their gear sounds better and have vociferously claimed these hypothesis to be fact without establishing any scientific proof. There is a big difference between hypothesis and fact. However, it is equally wrong of the scientific community to outright dismiss one's observations without providing equal scientific proof that those differences don't exist. Both sides need to work together and admit that there are still things we don't yet understand about the recording and playback of audio sources. Having given versions of this paper to many AES members, I know I have raised considerable awareness to the deficiencies in commonly reported specs.

IMHO, one of the biggest problems we have is differentiating between correlation and causality. I can say that water causes cancer. All persons that die of cancer have ingested water. And, if we deny water, those patients won't die of cancer. They'll die of dehydration. But look how well cancer deaths correlates to water ingestion. This extreme example shows that we can have a high degree of correlation that has no relevance on causality. The water didn't cause the cancer. History is filled with examples of correlation being confused with causality.

To me, it is perfectly acceptable to say that something works but the reason why remains a mystery. It is also perfectly acceptable to propose a hypothesis on why something works. In fact, I believe when something is stated as merely a hypothesis, it invites further research. We can certainly benefit from more research. Once research confirms causality, we can then work to quantify differences between products.

Please feel free to ask questions. I will try to check back and answer when I can.

This entry should be on the masthead of the WBF as mandatory reading. It might very well go a long way into ceasing the endless banal discussions posted here on the "objective/subjective" topic. Well done, Jonathan. And thanks to Michael for bringing this to our attention.
 

KlausR.

Well-Known Member
Dec 13, 2010
291
29
333
I agree that mikes do not hear like we humans do. But, there is some recognition of that in the typical downward sloping with increasing frequency target curves used by most DSP Room EQ systems.

I’ve read the corresponding papers, and there are quite substantial differences between what individual listeners prefer, hence there’s no one-slope-fits-all solution, but that’s not the point I was trying to make: without using correlations between measurements and perception an in-room response measurement can’t tell you whether or not there is a perceptible problem when listening to music and such measurement certainly can’t tell you if that “problem” is annoying. Better go and listen to your music and once you hear a problem that is indeed annoying use measurements to identify the culprit.

But, frequency and time domain response are the biggest factors in the distortions that listening rooms create, unless there is reliable evidence to the contrary, which there is not.

Of course there is no evidence to the contrary. However, where is the evidence that “the distortions that listening rooms create“ are perceptible in the first place and annoying in the second place, regardless of circumstances, in each and every case, because this is what we users are led to believe.

Klaus
 

Fitzcaraldo215

New Member
Nov 3, 2014
394
2
0
I’ve read the corresponding papers, and there are quite substantial differences between what individual listeners prefer, hence there’s no one-slope-fits-all solution, but that’s not the point I was trying to make: without using correlations between measurements and perception an in-room response measurement can’t tell you whether or not there is a perceptible problem when listening to music and such measurement certainly can’t tell you if that “problem” is annoying. Better go and listen to your music and once you hear a problem that is indeed annoying use measurements to identify the culprit.



Of course there is no evidence to the contrary. However, where is the evidence that “the distortions that listening rooms create“ are perceptible in the first place and annoying in the second place, regardless of circumstances, in each and every case, because this is what we users are led to believe.

Klaus

Yes, individual preference is all over the place, hence unpredictable for this, as with all things in audio. However, there have been numerous controlled studies to validate general target curve preferences across multiple listeners. Look for the one done some time ago by Sean Olive at Harman, for example.

Different studies have recommended slightly different target curves, but they tend mostly to all have a similar downward sloping with increasing frequency characteristic. There is nothing wrong with an EQ tool incorporating a specific default target curve chosen by the maker's research. The good news is with most competent tools, you also get to design your own target curve to your own personal liking. No one can tell you or predict exactly what you will prefer. But, frankly, I am quite happy myself with Dirac's default target curve, which appears to come from or at least closely duplicate the one empirically determined by instrumentation mike maker B&K's research.

Many of the same studies have identified the common distortions listening rooms create and their audit ability. There is tons of research on bass modal issues, for example. And, it is quite simple for any audiophile with a $75 USB microphone and REW freeware to measure proof positive of the existence of this in their own rooms. If you are not aware of this research or question its existence or auditability, you must not have spent more than a few seconds looking for it, because it is abundant and scientifically accepted.

It would also appear that rather than actually trying an EQ tool in your own room, you prefer to spend the time instead questioning the existence, validity or audibility of correction of common room issues. Try it. It will show you before and after measurements, and it will clearly demonstrate audible differences. Unlike much other audiophile folklore and mythology, this is for real. You will never know until you have tried it yourself.
 

AP Jonathan

New Member
Jun 6, 2011
8
0
0
Hi All,

I just wanted to let people know that it looks like I will be able to give this talk again at THE Show Newport this June. The exact date and time has yet to be set. It is also possible I may be giving it twice. Details are still pending. Unfortunately, it seems that the room for talks will be smaller this year than it was last year. I will probably be making some changes to the talk but the distortion demonstration will remain unchanged.
 

KlausR.

Well-Known Member
Dec 13, 2010
291
29
333
Yes, individual preference is all over the place, hence unpredictable for this, as with all things in audio. However, there have been numerous controlled studies to validate general target curve preferences across multiple listeners. Look for the one done some time ago by Sean Olive at Harman, for example.

AES paper 8994, that’s the one I was referring to: the range of preferred bass levels among the individual listeners was 17 dB, the range of treble levels 11 dB. The averaged curve had a bass boost of 6.6 dB and a treble cut of 2.5 dB. This result confirmed an earlier study (AES paper 7960) where room correction products with flat target curves were the least preferred.


Many of the same studies have identified the common distortions listening rooms create and their audibility. There is tons of research on bass modal issues, for example. And, it is quite simple for any audiophile with a $75 USB microphone and REW freeware to measure proof positive of the existence of this in their own rooms. If you are not aware of this research or question its existence or audibility, you must not have spent more than a few seconds looking for it, because it is abundant and scientifically accepted.

I have actually read those tons of research literature, a pdf of the file containing the list is attached. One specific result is that not all music is equally well suited for revealing modal issues. I quote from the respective paper: “Analysis of data reveals a striking difference in trends between results collected using Sample A and those using Sample B. Results for Sample A show that it is not particularly helpful in revealing perceptual differences between the systems under test—all systems lie within one z-score of the worst system and never cross the 5% significance level. We believe the characteristic temporal and musical differences between the samples are responsible for this. These characteristics in Sample B allow modal artefacts to be heard more clearly and thus lead to better informed judgment regarding the reproduction quality of each system. Results for Sample A lack the statistical evidence to warrant further discussion. However, this result is in itself of extreme importance since it establishes that the selection of tests sample(s), particularly in this type of “realistic” testing, is vital to extract meaningful and reliable results.”

In my own listening room, which is heavily treated with an acoustic ceiling (40 sqm of stretched synthetic fabric, above which an 26 cm air gap loosely filled with rock wool), all of the room modes are easily heard when playing pure sine tones. When playing music, however, there’s only a handful of tracks where at the listening position the 2nd order width mode is audibly excited. That’s the point I meant to make: a simple in-room response won’t tell anything about what is going to happen once you play music.


It would also appear that rather than actually trying an EQ tool in your own room, you prefer to spend the time instead questioning the existence, validity or audibility of correction of common room issues. Try it. It will show you before and after measurements, and it will clearly demonstrate audible differences. Unlike much other audiophile folklore and mythology, this is for real. You will never know until you have tried it yourself.

My speakers have an in-built digital 10-band EQ with both IIR and FIR filters. A handful of tracks exciting a 2nd order room mode for a short moment, that’s something I can live with. I therefore consider that my room does not have a problem hence I did not feel the need to use that in-built EQ for correction purposes. I do not question the existence, validity or audibility of correction of common room issues. What I question is that a simple frequency response at the listening position is able to tell me that I will have audible, and maybe even annoying, problems when playing music.

Klaus
 

Attachments

  • Literature room acoustics.pdf
    435.9 KB · Views: 29

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
When did this thread become about room response? Edit: I just now see that this thread is in the acoustics & room Eq section - so forgive my comment :)

The take-away from AP Jonathan is very clear - measure the stuff that is psychoacoustically important & understand your test signals & their relationship to the the dynamic, time-variant, high crest factor signals of music.

@AP, I've been talking about using multitone test signals as more appropriate & know AP scopes have some capabilities in this area - care to talk about that?
One thing I was interested in is using these multitone test signals to test for noise floor modulation - any ideas how this might be done?
 
Last edited:

AP Jonathan

New Member
Jun 6, 2011
8
0
0
@AP, I've been talking about using multitone test signals as more appropriate & know AP scopes have some capabilities in this area - care to talk about that?
One thing I was interested in is using these multitone test signals to test for noise floor modulation - any ideas how this might be done?
Multitone means different things to different people. For some it includes all twin-tone tests such as SMPTE IMD. For others it is a term reserved for wide bandwidth tests using synchronous (non-windowed) FFTs and three or more tones. I fall into the latter camp but either way, I have not done any investigations of this nature.

Multitones are steady state signals so I wouldn't expect them to highlight noise floor modulation. However, one could change the relative phase of the tones and thus change the crest factor. I wouldn't expect the noise level to change with crest factor but if it does, we may be on to something.

The AP multitone analysis uses a synchronous set of tones. This allows us to use a windowless FFT in order to get accurate results across a wide bandwidth. We can extract frequency response, noise, phase, crosstalk and total distortion with one acquisition. The total distortion number is a summation of the harmonics of all the tones and the IM products of each tone mixed with the other. The more tones you have, the higher the TD will be. This makes correlating TD to standard THD and IMD measurements difficult.
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing