Perception, science, engineering and high-end audio

rbbert

Well-Known Member
Dec 12, 2010
3,820
239
1,000
Reno, NV
It seems that in this forum, as in so many others, engineering is confused with science. It's not: from Wikipedia

"The American Engineers' Council for Professional Development (ECPD, the predecessor of ABET)[1] has defined "engineering" as:

The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property."

Science is based on the scientific method: again from Wikipedia

"Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge.[1] To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of reasoning.[2] The Oxford English Dictionary says that scientific method is: "a method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses."[3]

The chief characteristic which distinguishes a scientific method of inquiry from other methods of acquiring knowledge is that scientists seek to let reality speak for itself, and contradict their theories about it when those theories are incorrect,[4] i. e., falsifiability. Although procedures vary from one field of inquiry to another, identifiable features distinguish scientific inquiry from other methods of obtaining knowledge. Scientific researchers propose hypotheses as explanations of phenomena, and design experimental studies to test these hypotheses via predictions which can be derived from them. These steps must be repeatable, to guard against mistake or confusion in any particular experimenter. Theories that encompass wider domains of inquiry may bind many independently derived hypotheses together in a coherent, supportive structure. Theories, in turn, may help form new hypotheses or place groups of hypotheses into context."

So audio engineering, like any engineering, works from both scientific "facts" and scientific "theory" (where facts are not available). Unfortunately, audio perception has few "facts" available to define it. There is lots of empirical data, and some attempts to study various aspects of it with variable success. Very little of the data, and almost none of the proven (by the scientific method) facts, apply to our version of high-end audio, and that doesn't seem likely to change in the forseeable future (few or no resources for investigation and testing).

It's only partly true that we all perceive things differently. In fact, all people without specific disabilities/handicaps perceive things in about the same way. Without that, there would be little basis for communication and understanding. It's the little differences which make our lives more interesting.

Audio "objectivists" should realize that they deal in large part with theory, not fact. Like Newtonian mechanics, it works within its limitations, but it's not clear that we know exactly what those limitations are. Audio "subjecitivists" should realize that many aspects of our hobby (computer science, for example) are based on solid science, even though many other aspects (e.g., transducer behavior) are not.
 

caesar

Well-Known Member
May 30, 2010
4,300
774
1,698
It seems that in this forum, as in so many others, engineering is confused with science. It's not: from Wikipedia

"The American Engineers' Council for Professional Development (ECPD, the predecessor of ABET)[1] has defined "engineering" as:

The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property."

Science is based on the scientific method: again from Wikipedia

"Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge.[1] To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of reasoning.[2] The Oxford English Dictionary says that scientific method is: "a method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses."[3]

The chief characteristic which distinguishes a scientific method of inquiry from other methods of acquiring knowledge is that scientists seek to let reality speak for itself, and contradict their theories about it when those theories are incorrect,[4] i. e., falsifiability. Although procedures vary from one field of inquiry to another, identifiable features distinguish scientific inquiry from other methods of obtaining knowledge. Scientific researchers propose hypotheses as explanations of phenomena, and design experimental studies to test these hypotheses via predictions which can be derived from them. These steps must be repeatable, to guard against mistake or confusion in any particular experimenter. Theories that encompass wider domains of inquiry may bind many independently derived hypotheses together in a coherent, supportive structure. Theories, in turn, may help form new hypotheses or place groups of hypotheses into context."

So audio engineering, like any engineering, works from both scientific "facts" and scientific "theory" (where facts are not available). Unfortunately, audio perception has few "facts" available to define it. There is lots of empirical data, and some attempts to study various aspects of it with variable success. Very little of the data, and almost none of the proven (by the scientific method) facts, apply to our version of high-end audio, and that doesn't seem likely to change in the forseeable future (few or no resources for investigation and testing).

It's only partly true that we all perceive things differently. In fact, all people without specific disabilities/handicaps perceive things in about the same way. Without that, there would be little basis for communication and understanding. It's the little differences which make our lives more interesting.

Audio "objectivists" should realize that they deal in large part with theory, not fact. Like Newtonian mechanics, it works within its limitations, but it's not clear that we know exactly what those limitations are. Audio "subjecitivists" should realize that many aspects of our hobby (computer science, for example) are based on solid science, even though many other aspects (e.g., transducer behavior) are not.


Nice post. I actually have been collecting a lot of stuff on subjectivity that is based on solid research, but I have not had a chance yet to compile thoughts and post them.

Also, the theory of Flow that I frequently bring up is one of the most widely accepted theories of happiness.
 

microstrip

VIP/Donor
May 30, 2010
20,807
4,700
2,790
Portugal
I think that the ABET (Accreditation Board for Engineering and Technology) definitions are well suited to audio development and manufacturing:

ENGINEERING is the profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize economically the materials and forces of nature for the benefit of mankind.

ENGINEERING TECHNOLOGY is the part of the technological field that requires the application of scientific and engineering knowledge and methods combined with technical skills in support of engineering activities; it lies in the occupational spectrum between the craftsman and the engineer at the end of the spectrum closest to the engineer.
 

rbbert

Well-Known Member
Dec 12, 2010
3,820
239
1,000
Reno, NV
A nice summary here illustrating why typical audio DBT's are unlikely to uncover (or confirm) anything beyond "gross" differences between two items being tested. As I've posted in other topics on this same subject, expecting listeners to be able to focus on more than one audio characteristic is unrealistic. As an example, when listeners are able to identify and focus on the artifacts of MP3 coding/decoding, they are usually able to distinguish between MP3's and unencoded music; without that sort of focus on a particular sonic "signature", detecting that difference (in a test) is much less likely.

http://online.wsj.com/article/SB100...0069498164.html?mod=WSJ_LifeStyle_LeadStoryNA
 

FrantzM

Member Sponsor & WBF Founding Member
Apr 20, 2010
6,455
29
405
rbert

I don't get your point. I , for one know that Engineering is not science .. It is however the application of Science. Our perception, that we so much attach ourselves to be so special is regularly studied by science and its limits are well known .. Humans don't hear anything past 30 KHz (I know I am being generous most of us here don't hear squat over 15 KHz) ... The thresholds are known and very well ( for example 0.1 dB of difference NO WAY we can hear that) .. The study of how we perceive is scientific and is based on repeatable measurements and experiments not on "I-think-I-swear" or "I-know-I-heard-it-believe-me" anecdotes. Repeatable. We don't, know how to measure everything but let's at least agree that if it exist it is measurable.
Let's forget about DBT for a while.. There is to me some obvious differences between mp3 and un-mp3 .. but I am willing to bet I can confuse more than one audiophile with a 320 Kbps file ... if the reference is not known .. IOW despite claim to the contrary it is not easy to discern an mp3 "signature" if the data rate is high enough say 256 most people and that includes audiophiles will not know it is an mp3 .. It is a cruel joke to play on audiophiles but one you can check for yourself and believe me the differences are far from "gross" .. They are so subtle as to be well ______________ fill the _________ ;)

Unsettling? Yes .. For the record . I prefer Hi-rez to Redbook .. Can I hear a piece and declare it is Hi-Rez? No! .. Given however the choice to purchase between CD and Hi-Rez. I tend to go for the Hi-Rez version ... Call this a concession to audiophilia insecurities :)
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
This science stuff always confuses me. Evolution, is that just a theory? I mean, they call it the Theory of Evolution, right? So if it's just a theory, an unproven hypothesis, I can come up with a theory that says the first humans appeard in a flash of light on a beach in Scotland a thousand years ago, and, until Darwin "proves" his hypothesis and it becomes a "fact," my flash of light is in the same class - theory - as Origin of the Species? How about Relativity? That's a theory, no? And how do we really know that the earth revolves around the sun and not vice versa? How do we test that hypothesis?

Tim
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Regarding that bastion of science, the Wall Street Journal, and it's implications regarding DBT. I thought you had missed something critical. I didn't expect to find it in the second paragraph:


This ability to hyper-focus on one stream of sound amid a cacophony of others is what researchers call the "cocktail-party effect." Now, scientists at the University of California in San Francisco have pinpointed where that sound-editing process occurs in the brain—in the auditory cortex just behind the ear, not in areas of higher thought.

You've confused the ability to hyper-focus with an inabilty to not hyper-focus. DBT lives to be discredited another day.

Tim
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Nice post Rbbert - very well said!

Science is theory, not "facts" - I've been saying this all along. Some theories are stronger than others as they have been repeatedly tested & experimental evidence has correlated with them. BUT, they are not "FACTS" - they can be overturned any time by an experiment. The famous Michelson-Morley experiment of 1887 was designed to test the concept of the Aether (a popular concept at the time). Instead it gave contrary evidence & was considered a "failed experiment" on it's publication but was more correctly a null result. It is said to have been one of the factors that ultimately led to the special theory of relativity which in itself is an replacement for Newtonian mechanics> Newtonian mechanics only holds true when all the velocities are a small fraction of the speed of light.

So you can see that here are no "facts" in science - it's all theory!!

Again, I think stating that we know "the limits of our perception" is erroneous. Even the model for hearing has not yet been fully agreed!

Take relativity for example - the theory directed us to new experiments & discoveries of empirical evidence that we would not have considered before. So these things existed but we were not aware of them & didn't test for them until the theory. This is not the same as saying "We don't, know how to measure everything but let's at least agree that if it exist it is measurable." We have to be aware of or suspect the existence of something before we can measure it.

To many people science can be a bit like a religion - by & large, we accept & believe in what the scientists tell us are the predominant theories at the moment. But again these are theories & this is a belief :) So like any belief system, there can be a religious fervour associated with it & often we see this manifest. We don't have to look too hard for examples of this.
 

JackD201

WBF Founding Member
Apr 20, 2010
12,318
1,427
1,820
Manila, Philippines
If one wants some insights on science vs engineering and just being a technician, I recommend watching the TED lecture entitled 18 minutes with an Agile Mind. I would also recommend the one about how juries are fooled by improper use of statistics.

DBTs are not being discredited. There is however a growing trend of reading way too much into results gathered from sample sizes way to small to be anywhere but conclusive. What irks me most are the evangelists that say they are doing this in the name of Science when they do not bother to do their basic statistics right. Again watch that lecture on how juries are fooled for an easy to understand explanation of some of these basic errors in correlation.
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland

rbbert

Well-Known Member
Dec 12, 2010
3,820
239
1,000
Reno, NV
DBT's are an important part of scientific inquiry. When used for tests of perception, however, it is increasingly clear that they have to be narrowly focused on a single well-defined parameter. Few people are capable of concentrating on more than one specific thing at a time; even those who are capable have a great deal of difficulty devoting their complete attention to more than one thing at a time.

Our auditory perception changes hour to hour and day to day. It's dependent on our state of mind, state of health, state of nutrition, even about when we last ate, slept, etc. It's also dependent on the time of day, time of year, climate conditions, etc. Objective instrumented measurements are a far better evaluation of performance for every aspect of our music systems, but we don't know all the measurements to make and we don't know which ones matter. When I listen to music all that matters is how it sounds right then and how much enjoyment it's giving me, but if I want to make a more consistent improvement in my audio system over a longer time, other audio assessments are needed. Right now I don't know what those are as much as what they aren't. "Objectivists" in general put far too much faith in completely inconclusive tests of auditory perception and clearly outdated ideas of appropriate measurements. "Subjectivists" put far too much faith in the extremely imperfect and variable sense of auditory perception. Not enough listeners, and maybe not enough manufacturers (but here I suspect there is much more true objectivism than is commonly acknowledged) pursue appropriate auditory tests and instrument measurements.
 

JackD201

WBF Founding Member
Apr 20, 2010
12,318
1,427
1,820
Manila, Philippines
Has anyone looked into DBTs in wine - a similar industry to audio :)?
There is one here that I haven't fully read yet but it is based on 6,000 blind tastings "Do More Expensive Wines Taste Better? - Evidence from a Large Sample of Blind Tastings"

Do note the exclusion processes employed in this paper. Regardless, they did qualify their conclusions with these words "IN OUR SAMPLE". It also shows the discrepancy between the trained and untrained panelists.

Again I am not against informal blind testing. In fact I'm all for it. It can save a person a lot of dough that could be used for more important things in life than listening to a stereo. My thing is, don't do an informal test and then go out and say you know the truth. The best one can ever get is "for x percent of those tested, the results are y". To project this on the entire population would be BAD statistics.
 

kevinh

New Member
Mar 30, 2012
46
0
0
Sussex cty NJ
This science stuff always confuses me. Evolution, is that just a theory? I mean, they call it the Theory of Evolution, right? So if it's just a theory, an unproven hypothesis, I can come up with a theory that says the first humans appeard in a flash of light on a beach in Scotland a thousand years ago, and, until Darwin "proves" his hypothesis and it becomes a "fact," my flash of light is in the same class - theory - as Origin of the Species? How about Relativity? That's a theory, no? And how do we really know that the earth revolves around the sun and not vice versa? How do we test that hypothesis?

Tim


In science a theory can be disproved using the Null Hypothesis for the theory.

http://en.wikipedia.org/wiki/Null_hypothesis

or as Einstein said: “No amount of experimentation can ever prove me right; a single experiment can prove me wrong.”

The problem with audio is that each's persons ear brain system, what distortions are humans sensitive to, as well as what they listen for, and placebo effects are subjective and isolating that from objective criteria is problematic.


BTW in some cases even when a theory is proven wrong it can still be right in certain context, for example Newton's laws of mechanics break down at small distances and at large distances when quantum mechanics and relativity (gravity) rule, however in our every day world of hum experience they work well within the range of energies and distances we deal with.
 

microstrip

VIP/Donor
May 30, 2010
20,807
4,700
2,790
Portugal
(...)
Again I am not against informal blind testing. In fact I'm all for it. It can save a person a lot of dough that could be used for more important things in life than listening to a stereo.

Jack,

Most probably an informal blind test will be formally flawed and our biases will leak through it. But the conclusion will be considered valid, and as it will carry the benediction of being submitted to blind validation, will let us sleep much better! ;)

BTW, one subliminal bias that we are most of time accepting is choosing recordings that will favor the components or systems we are considering.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
DBT's are an important part of scientific inquiry. When used for tests of perception, however, it is increasingly clear that they have to be narrowly focused on a single well-defined parameter. Few people are capable of concentrating on more than one specific thing at a time; even those who are capable have a great deal of difficulty devoting their complete attention to more than one thing at a time.

Here's A. Here's B. Here's X. Is X A or B?

There's your basic AB/X test. When your eyes are wide open and you know you're listening to that high-res download you paid 50 bucks for, and you're comparing it to that 320kbps mp3 your buddy gave you on an Office Max thumb drive, the sound stage is expanded in both depth and breadth, the inner detail is extraordinary, the tonality is so pure, so natural, the dynamics are palpable...

Here's A. Here's B. Here's X. Is X A or B?

If A is the hi-res file and B is the mp3 and you can't pick which one is X more often than you'd get it by flipping a coin, does that prove anything? Of course. It proves that with those two samples, through that system, on that day, you couldn't tell which one was "expanded in both depth and breadth, the inner detail is extraordinary, the tonality is so pure, so natural, the dynamics are palpable..." and which one was the mp3. Is it science with a capital S? No. But it's a mighty good BS repellant. Use it or deny it. Your call.

Our auditory perception changes hour to hour and day to day. It's dependent on our state of mind, state of health, state of nutrition, even about when we last ate, slept, etc. It's also dependent on the time of day, time of year, climate conditions, etc.

True. Overstated, but there's an element of truth there. This is why there are multiple subjects, multiple trials, etc, etc. in more formal testing. You're running a simple, informal test at home and you think the results might have been impacted by fatigue, or mood, or blood sugar? Cool. Rest. Eat a Snickers bar, try again later. Let me know if that Snickers bar defines the difference between "expanded in both depth and breadth, the inner detail is extraordinary, the tonality is so pure, so natural, the dynamics are palpable..." and only guessing which one is the mp3 49% of the time.

Objective instrumented measurements are a far better evaluation of performance for every aspect of our music systems, but we don't know all the measurements to make and we don't know which ones matter.

I really don't think so. Not for the more dedicated subjectivist audiophiles. I put greater faith in the ability of good measurements to predict sound than most of them. Most deem the connection non-existent. So if you have one pair of headphones that measure with an obvious low midrange boost and a gradual roll-off above 10khz, and another pair that are dead flat through the lower mids with a slight rise above 5khz, and, as so many audiophiles say, this doesn't tell us anything about how they will sound, I think blind listening might be a better option for them. Personally, I've got a pretty good idea of what those two sets of cans are going to sound like.

"Objectivists" in general put far too much faith in completely inconclusive tests of auditory perception and clearly outdated ideas of appropriate measurements.

I don't speak for all objectivists, but I'm going to give it a shot anyway -- Nah. I think very few of us are buying equipment based on measurements, outdated or otherwise. And I'd bet no one is buying it based on blind listening. Blind listening is just a good tool to remove one obvious and powerful influence that can booger (the technical term) audio evaluation -- expectation bias. Even after it's eliminated, you still have to listen. In fact, you really have to listen with no support from how cool, expensive, accepted it is. This is really trusting your ears.

"Subjectivists" put far too much faith in the extremely imperfect and variable sense of auditory perception.

Maybe some do. I really can't speak for these guys, but I hope what they're doing is just buying the stuff that sounds good to them. That some go around after the fact creating reasons why their choices are pseudo-objectively superior is a bit annoying, but I still think they're just listening to what they like.

Tim
 
Last edited:

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
or as Einstein said: “No amount of experimentation can ever prove me right; a single experiment can prove me wrong.”

This is precisely my point. Scientifically, something can be very well-accepted, broadly believed to be true, proven for all practical purposes, and still be theory. The lofty standards of pure science are not a good arbiter of practical engineering.

Tim
 

kevinh

New Member
Mar 30, 2012
46
0
0
Sussex cty NJ
This is precisely my point. Scientifically, something can be very well-accepted, broadly believed to be true, proven for all practical purposes, and still be theory. The lofty standards of pure science are not a good arbiter of practical engineering.

Tim


Agreed.


In terms of engineering for audio, (and lots of other areas), since no devices are perfect it is a matter of trade offs and deciding what is most important, and of course in most cases the cost of components and the impact on cost of the finished product
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
That some go around after the fact creating reasons why their choices are pseudo-objectively superior is a bit annoying, but I still think they're just listening to what they like.

Tim

I hope everybody is listening to what they like. To do otherwise means you are listening to what you don't like.
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing