Yes, as a friend said to an engineer who asked your question, "yes, political science is a great deal like engineering."
Why do some Objectivists fear Psychoacousitics?
- Thread starter cjfrbw
- Start date
Here's my opinion of what I think the OP is getting at:
I'm of the opinion that A/B testing usually requires a different type of listening to our normal listening - it usually requires a focussed listening to find certain types of short, discernible differences between two audio examples This usually requires a type of auditory attention & focus during repeated playing of audio snippets. A type of attention that we seldom use in our normal interaction with our audio world. As a result many tire or lose focus very quickly because of boredom, when engaged in this & maybe don't immediately notice their focus has drifted - their choice is therefore the same as a random guess as they have stopped listening. I suspect that many blind test results suffer from this issue & we don't have any way of verifying this (unless controls to illuminate false positives are included in such tests). This issue is further compounded because such audio tests are really statistical in nature & require more than 5 or 6 repeat trials for any statistical significance. I would imagine that training helps in all aspects of this - being able to identify particular types of distrotions, being able to retain focus longer & being able to recognise when focus is lost. It might be interesting to do a statistical analysis of blind test trials to see if there is a statistical difference between results returned from the trials early in the test & results from trials later in such tests.
I've always favoured long-term listening as a preferable means of differentiating audio devices, etc. for a number of reasons. One reason being that normal relaxed listening is being used & therefore less prone to fatigue/loss of focus, boredom & being spread over a longer timeframe (therefore probably having less influence on the results). But the biggest reason I have for favouring normal listening is because it is more revealing of the type of auditory differences that seem to matter to our auditory perception. I believe that the objective of an audio system is to produce an illusion that best matches the rules our auditory perception knows apply in the real world.
It is becoming clearer how our auditory perception is continually parsing the signals from the vibrations arriving at our ears into the auditory objects & streams which makes sense of our audio world. Our auditory perception does this based on grouping by inference, certain auditory cues together to form these auditory objects (simultaneous grouping) & streams (sequential grouping). This sequential grouping is a continuous analysis occurring over time - continually ascertaining what cues in the sound field are part of the same auditory object/stream. Some cues - Fundamental Frequency and Spectral Regularity; Onset Timing; Correlated changes in Amplitude or Frequency; Sound Location. These cues are not always present at the same time - they come & go, they fluctuate. We are also not always consciously aware of these aspects/cues or of their change in an audio stream - rather we notice the gestalt of the audio playback - how we relate to it, how it retains our interest, how relaxed our listening is, how non-fatiguing it is.
This, to me is a different aspect of the audio presentation than is picked up by A/B style listening - it requires forming an opinion by listening to the audio playback over an extended period of days or weeks.
Of course results from such listening is anecdotal & isn't an accepted form of evidence for objectivists. Instead of trying to devising a test which addresses what is known as psychoacoustics, objectivists tend, not unsurprisingly, to play it safe - adopting tests from other disciplines, like DBTs, JNDs for simple rather than complex tones. Similarly in measurements - simple rather than complex test tones as test signals.
Instead objectivists want "proof" that reported differences are "audible" by demanding results from such simple tests. With the problems I see as inherent to most A/B testing, it's not surprising that null results are the predominant outcome of such tests.
I'm of the opinion that A/B testing usually requires a different type of listening to our normal listening - it usually requires a focussed listening to find certain types of short, discernible differences between two audio examples This usually requires a type of auditory attention & focus during repeated playing of audio snippets. A type of attention that we seldom use in our normal interaction with our audio world. As a result many tire or lose focus very quickly because of boredom, when engaged in this & maybe don't immediately notice their focus has drifted - their choice is therefore the same as a random guess as they have stopped listening. I suspect that many blind test results suffer from this issue & we don't have any way of verifying this (unless controls to illuminate false positives are included in such tests). This issue is further compounded because such audio tests are really statistical in nature & require more than 5 or 6 repeat trials for any statistical significance. I would imagine that training helps in all aspects of this - being able to identify particular types of distrotions, being able to retain focus longer & being able to recognise when focus is lost. It might be interesting to do a statistical analysis of blind test trials to see if there is a statistical difference between results returned from the trials early in the test & results from trials later in such tests.
I've always favoured long-term listening as a preferable means of differentiating audio devices, etc. for a number of reasons. One reason being that normal relaxed listening is being used & therefore less prone to fatigue/loss of focus, boredom & being spread over a longer timeframe (therefore probably having less influence on the results). But the biggest reason I have for favouring normal listening is because it is more revealing of the type of auditory differences that seem to matter to our auditory perception. I believe that the objective of an audio system is to produce an illusion that best matches the rules our auditory perception knows apply in the real world.
It is becoming clearer how our auditory perception is continually parsing the signals from the vibrations arriving at our ears into the auditory objects & streams which makes sense of our audio world. Our auditory perception does this based on grouping by inference, certain auditory cues together to form these auditory objects (simultaneous grouping) & streams (sequential grouping). This sequential grouping is a continuous analysis occurring over time - continually ascertaining what cues in the sound field are part of the same auditory object/stream. Some cues - Fundamental Frequency and Spectral Regularity; Onset Timing; Correlated changes in Amplitude or Frequency; Sound Location. These cues are not always present at the same time - they come & go, they fluctuate. We are also not always consciously aware of these aspects/cues or of their change in an audio stream - rather we notice the gestalt of the audio playback - how we relate to it, how it retains our interest, how relaxed our listening is, how non-fatiguing it is.
This, to me is a different aspect of the audio presentation than is picked up by A/B style listening - it requires forming an opinion by listening to the audio playback over an extended period of days or weeks.
Of course results from such listening is anecdotal & isn't an accepted form of evidence for objectivists. Instead of trying to devising a test which addresses what is known as psychoacoustics, objectivists tend, not unsurprisingly, to play it safe - adopting tests from other disciplines, like DBTs, JNDs for simple rather than complex tones. Similarly in measurements - simple rather than complex test tones as test signals.
Instead objectivists want "proof" that reported differences are "audible" by demanding results from such simple tests. With the problems I see as inherent to most A/B testing, it's not surprising that null results are the predominant outcome of such tests.
It seems to be that a double blind ABX test could easily be designed to deal with all of the issues that you have raised. I would propose for one that the test subject should be able to listen to a track or tracks for as long as they want, it a a nice relaxed atmosphere, telling the tester when to switch (or pretend to switch) when he or she is ready. I for one would volunteer to be the test subject in just such a test.
Yes but this is not what I'm saying - what you are suggesting is that the user can switch between tracks at any stage according to their whim. I think you will find that the test subject, after listening to the two tracks a number of times, has identified a short segment which he/she thinks can be used as a differentiator between tracks. The switching is being done at this point in the track - what has played before reaching this point is immaterial - it's the sound of the guitar, high hat, reverb, ambience or whatever that is being focused on as the differentiator. This is even addressed in Foobar ABX where just a segment can be isolated & played for each track - avoiding other parts of the audio track that could act as a confounder.
The big issue that is emphasised by objectivists, in such tests, is the unreliability of audio memory when there is more than about 4 seconds gap between one track to another so according to this logic playing a full track for it's gestalt & then the other track is not a recommended procedure. Indeed, short segments & instant switching between the two tracks are a must.
What I am saying is that this focussing changes our listening mode (& this is the bit of psychoacoustics that objectivists fail to recognise) & as a result introduces a new biasing variable. The very test rules out normal listening. If you ask Amir, who has participated & organised many ABX & DBTs, I'm sure he will tell us how often ABX is used in this manner - play one track fully for the feel of it & then play the other track, again for the feel of it & use this as the basis for selection rather than focussing on a particular part of the track.
What is often cited by objectivists is that "preference" is not a question & not being tested.
Are you familiar with Toole's book?? That happens to be exactly how they do their testing on loudspeakers. It's based on preference. As a matter of fact they can reliably predict how well a speaker will score just by looking at the frequency response/measurement set. This is all based on which speaker the blind listener thinks sounds better.
Rob
Not familiar with his book. A bit of familiarity with his tests. I find the use of one speaker rather than two speakers in his tests be two exactly what I'm talking about - objectivists wilfully ignoring psychoacoustics because, as Toole says, (I paraphrase), "it's a simpler test"Are you familiar with Toole's book?? That happens to be exactly how they do their testing on loudspeakers. It's based on preference. As a matter of fact they can reliably predict how well a speaker will score just by looking at the frequency response/measurement set. This is all based on which speaker the blind listener thinks sounds better.
Rob
I do believe they tried it both ways, stereo and mono. The stereo results didn't contradict the mono tests. But the differences found in the mono test were larger and more reliable. Essentialy they made it easier for the test listeners to reach a conclusion that they would also reach with more difficulty in a stereo test. I fail to see how that is a bad approach. It is a more discrinmating test than if it were more complex. They haven't ignored psychoacoustics.
A bit off topic. A common piece of advise given by people who mix recordings: listen to your mix in mono when you are done. Why? It is easier to hear when something isn't right about the mix in mono. Get it right in mono, and it will be fine for sure in stereo.
I think you will also find it to be very uncommon for human subjects to perform better in a complex test than a simpler test.
Do you have the research that shows this?
Again, without showing the research which correlates stereo & mono results, I find this unconvincing.
It's a bad approach because it eliminates a significant number of factors that we pay particular attention to in our normal listening - ITD. ILD, sound stage - width, depth, solidity - I'm sure there are others perceptual aspects I'm missing that rely on stereo reproduction. I find this approach similar to measurements using test signals very unrelated to the dynamics & crest factors in music - again for the reason that it is simpler to analyse.
I can understand the desire for using tests which are easier to analysis but the limitations of such tests should be recognised.
Maybe so, I don't know the recording side - maybe some who do can explain this?
Yes, I agree absolutely with this & would add that a lot of blind testing falls into this category - psychologically, it is a heavier processing load with more factors in play than normal listening.
Dr. Toole did the tests and said whenever speakers do better in mono they do in stereo as well. Keeping in mind most of his tests regarding speakers are blind testing for a preference of one speaker vs another. Once he obtained those results it made testing simpler to do it in mono. You'll have to ask him for the research data. His motivation wasn't for a simpler test. It was for the most disciminating test. That it was also simpler was a bonus.
If all the reasons you think stereo important (and without Tooles work I would have assumed the same thing for the most part) get less clear results than mono testing perhaps those aren't the issue you believed them to be. After reading those results it makes some sense. ILD, ITD are just between speakers. It isn't like one speaker has more of a time difference than another if positioned the same. Nor like the level difference will be more different with one vs the other. So what is left is those various stereo imaging and staging effects are probably due to varying frequency and phase response along with directional differences. It may be in mono, though no imaging is possible all those same differences cause a similar or even heightened differences in the preference. So the best testing speaker in mono will be the best speaker in stereo as the stereo result will be more accurate and predictable. Toole and Sean Olive have developed a model that they say allows them to predict which speaker will be preferred by testees with a very high degree of correleation. Much of that model does look at directionality of the speaker, early reflections, and a number of other things that would appear important for good stereo performance. You can read a bit about their ideas in this article from 2004.
http://www.edn.com/design/test-and-measurement/4384927/Sound-science
As for listening to mixes in mono, I don't know that it has been researched. It is some collective wisdom that a number of people working in that field suggest. Not everyone does it or thinks it necessary.
While blind testing may present a heavier load, it also seems to give more consistent results.
If you have not read the book you don't have a complete understanding of the testing. Why he went mono is all explained. All the data you are asking for to support that position is all either in the book or in the referenced papers in the back of the book. He chose the simpler test because of psychoacoustics, he used that as a tool to get more reliable results not ignoring it as you suppose.
Rob
Just an aside, the Harman speaker shuffler takes 3-4 seconds to switch speakers. Even though each speaker sounds very different, a lot of those differences get erased by the time you hear the other speaker! It requires fair bit of concentration to remember what the other speaker sounded like. The short term memory is really that: short term.Are you familiar with Toole's book?? That happens to be exactly how they do their testing on loudspeakers. It's based on preference. As a matter of fact they can reliably predict how well a speaker will score just by looking at the frequency response/measurement set. This is all based on which speaker the blind listener thinks sounds better.
Rob
As I asked already, can you link me to the research paper which shows the correlation between mono & stereo results? From what I read, the reasons given were to avoid the confounding affects of testing in stereo - comb effects, etc
Again, I haven't seen any research to show this. But I would also make the point that if one's listening is being directed towards freq & amplitude differences then yes, that perceptual aspect of our hearing is being ignored.
What results?
You are assuming too much here - the balance between speakers (matching of speaker drivers) will have an effect. The time alignment between the various drivers (in one speaker) may only become evident in stereo listening? As I keep saying where is th eresearch that shows the correlation between mono & stereo results?
Ah, do you mean to retrac what you just said then - a heavier processing load being a confounder is well known in psychology.
Consistent results don't mean that the results are correct, I believe.
If you have not read the book you don't have a complete understanding of the testing. Why he went mono is all explained. All the data you are asking for to support that position is all either in the book or in the referenced papers in the back of the book. He chose the simpler test because of psychoacoustics, he used that as a tool to get more reliable results not ignoring it as you suppose.
Rob
Ok, then there must be some papers that can be cited? Or even a summary explanation?
Good point. Perhaps this is another psychoacoustic aspect being ignored in this particular test?
Anyway, let's not get side-tracked into another Toole/Olive/Harmon debate. It was brought up here in answer to my post, as an example of blind testing for preferences & yet I think it's a good example of objectivists avoiding the complexity of dealing with psychoacoustics - using a mono speaker which are switched outside of our known auditory echoic memory time window.
You seem to be quite good at only listening to yourself. This isn't the first time. If you want to know about the data on Toole's work, get the book he wrote. I would go find it for you, there are at least a handful of interviews where he gives summary of those findings. But I was a bit put off by your dismissive attitude in your reply to me. So find it yourself or buy his book or bumble on needlessly. They did the tests, mono and stereo. Got better more reliable and more discrimnating results in mono. It is not what you are convincing yourself it is.
Then a whole diatribe to make it look as if I made big claims and didn't supply the results you need. The repeated lament, what results? Where is the data? Well, I told you where to find it. Go buy the guy's book.
+1 Here
Your points is that blind test is not perfect .. Everyone gets that. Has gotten it for a while and on both side there will be extreme positions. For the most part objectivists are quite clear that they can't always perform blind tests they make do with what is available.
Almost everyone know that Blind tests are not perfect. Nothing is. There are tests methodologies although imperfect that yield better , more reliable results than others.
As an aside it would be interesting for you to read this post by Ack:
Is your brain playing tricks on you? A case of expectation bias?
You will then make of it what you wish.
Last edited:
If you have not read the book you don't have a complete understanding of the testing. Why he went mono is all explained. All the data you are asking for to support that position is all either in the book or in the referenced papers in the back of the book. He chose the simpler test because of psychoacoustics, he used that as a tool to get more reliable results not ignoring it as you suppose.
Rob
I have read the book since a few years ago, but I also read the opinions from people who think otherwise. For example Siegfried Linkwitz :
"In this book Floyd Toole summarizes and explains conclusions from a lifelong involvement with audio. I highly recommend it to anyone interested in factual information about loudspeakers and listening rooms, about measurements, listening observations and their practical implications. It is lucidly written in easy to understand language, extensively illustrated and referenced. It deals with the reproduction of sound - which existed in a space - inside another space. My only regret is that the potential of 2-channel playback in doing so has not been fully explored and misrepresented. This is understandable because the conventional box loudspeaker with its frequency dependent directivity index has been used for almost all of the observations that are discussed. In fact, the particular interaction of a box loudspeaker with the listening room makes it more difficult for our ear/brain perceptual apparatus to hear the recording venue's space and acoustics, provided that such information has been captured in the recording process. "
Along his excellent book F. Toole often refers to articles of people who disagree with him. I find amusing that many of the followers of his "speaker rules" in WBF ignore and completely disrespect his views on room characteristics and interaction. As I have often said, IMHO Part One - Understanding the principles (that does not show in internet marketing articles) is by far the best part of the book. As always the most important is keeping an open mind, but people should first understand what is preference and the fundamentals of stereo reproduction. Yes, being audiophile is a preference.
Thank you for the psychological analysis, Dennis
Here's what I can find in Toole's book regarding this mono aspect
"All of the fidelity ratings used here came from monophonic listening tests. Over half of the loudspeakers were evaluated in stereophonic comparisons as well, but these data are not included. As pointed out previously [1], highly rated loudspeakers receive similar ratings in both stereo and mono tests, but loudspeakers with lower ratings tend to receive elevated ratings in stereo assessments. This scaling distortion, combined with the increased judgment variability in stereo tests, encouraged the use of the monophonic test results"
This far from settles the matter to my mind
Here's what I can find in Toole's book regarding this mono aspect
"All of the fidelity ratings used here came from monophonic listening tests. Over half of the loudspeakers were evaluated in stereophonic comparisons as well, but these data are not included. As pointed out previously [1], highly rated loudspeakers receive similar ratings in both stereo and mono tests, but loudspeakers with lower ratings tend to receive elevated ratings in stereo assessments. This scaling distortion, combined with the increased judgment variability in stereo tests, encouraged the use of the monophonic test results"
This far from settles the matter to my mind
Yes, unfortunately, this is the problem. To my mind, as long as the test is giving the results wanted (null results) this "making do" is accepted. Once the same test reports results not in keeping with "expectations" the whole tets procedure is questioned/analysed/changed. Witness the reaction to Amir's positive ABX results of Arny's test - investigations of IMD, sample rate converters, possibility of cheating, etc.
Contrast this with the reaction to the suggestion to include controls within such tests as a means of further improving them & raising them somewhat above "make do" status - mainly rejection or inertia in even considering implementing such controls. Usually with the excuse - "controls are used in properly run tests". Yes, but by your own admittance - the majority of blind tests are not "properly run".
It's this inertia, lack of interest, lack of desire for truth that strikes me as disingenuous
Who says this applies in the field of auditory perception when long-term listening is used? Please don't cite that Harmon test of Sighted Vs Blind as an example - it is one of the best examples of skewing a test I have ever seen - picking employees from Harmon as the test subjects, doh!!
It's fun
Last edited:
Similar threads
 | Steve Williams Site Founder | Site Owner | Administrator |  | Ron Resnick Site Co-Owner | Administrator |  | Julian (The Fixer) Website Build | Marketing Managersing |