Why do some Objectivists fear Psychoacousitics?

[Orb]

Jack,
although it is interesting the approach by some engineers who do not abx parts.
As an example John Curl has designed both products that have very tight margins and also those that are expensive, testing certain internal components such as capacitors is not done via abx per se as one needs to use either Spice or mostly as he does a breadboard; so either modelled or sighted on decision of internal components followed with measurements and sighted listening of intended components/design (that is also done the same with full product by many engineers).
Not directed at you and just to add, it is possible to overcome some decision orientated biases and I would say also some others such as mOFC pleasure when one has a lengthy and broad experience with knowledge combined with a framework-methodology on working (linked in the past some studies into overcoming biases).

Cheers
Orb

 

[JackD201]

Not surprised Orb. Someone like John would know what parts of a circuit would be more critical from all those years of experience. He could very well do with SPICE or other simulators.

 

[jkeny]

Here's the problem I have with all of that:
@JackD "Nothing beats a blind test when it comes to reliable results for DIFFERENCES." Differences/distortions are not constantly present - they are dynamic - they won't just pop out & present themselves in blind A/Bing - in fact it's almost certain you will miss them unless you are so lucky that you switch from A to B at the moment of distortion. So the only successful way to use blind A/Bing is to have previously identified the distortion & listen for it in A/Bing. This is done sighted prior to A/Bing. Blind A/Bing is only done as a confirmation - it's not a test to REVEAL differences. That's why John Curl & others do without - by & large they don't need the confirmation.

Now the majority of blind tests I see being run on forums make this same mistake - they think that the "night & day differences" reported by some subjectivist will be so evident in a blind A/B that they use it incorrectly - they use it as a test to REVEAL differences. Of course they get a null result which further reinforces their initial scepticism & ensures their expectation bias (no difference exists) is now set forever because they seldom ever return to the test having "proven" the issue to themselves.

The example I gave earlier of Vital's journey (on PinkFishMedia forum) is a worthwhile lesson in how, in the real world, these tests are used & their pitfalls. He was unusual in that he had enough self-analysis & inquisitiveness to continue his search even after he had done 2 blind test session in which all present heard no differences between DACs, sighted or blind. It was only when he did his fourth blind test session & someone was there that could identify sighted a difference to listen for that he could then do so blind. Somebody dismissed this earlier by saying that he learned how to listen but this is the great problem - we need training to pass these blind A/B tests, otherwise we will never hear the differences & the null blind test will reinforce this & create an nocebo expectation bias that requires exceptional conditions to overcome. He admits that a big part of him not hearing differences, sighted or blind, were due to the blind tests but actually I think it was due to nobody ever pointing out to him the exact thing(s) to listen for & the blind A/B sessions further reinforced & became his expectation bias.

So I don't see the differences between the sighted listening done prior to a blind A/B test & long term listening.

But I do see a major difference between sighted long-term listening & blind A/B testing - we keep a much more open mind in doing long term listening then in blind A/Bing. As I said above blind A/Bing mostly returns null results (due to invalid test conditions) which reinforces a nocebo expectation bias. Long-term listening is exactly the opposite of this - we are attempting to find the audio characteristics of devices & hence we do not form an opinion quickly in a short listening session.

So my advice is blind A/B testing should not be tried at home - it is only for those who know what they are doing.

 

[jkeny]

@tomelex - I answered your MOVING TARGET point in my post as follows - there is no moving target - the target is our auditory processing rules which are inherent in our brains & the same for everybody - we judge what we hear based on this rule base which tells us if something is realistic or not in the auditory world. Actually, it's not as black & white as this - it's a sliding window of realism

There is one aspect that I agree with you on & it's this - our auditory system is always using a best guess - it is continually using audio cues to create a best guess of the auditory scene - all this is governed by the underlying rules of processing & our past experience of sounds.

Around your point about preamp A - remember that this hobby is one of growth - we continually grow & become more discerning in what we hear as we are exposed to system that give a better illusion. It continually happens to me - we often don't notice a distortion until it is removed - thereafter we have a new benchmark sound. How do we recognise this is better? Because it rests better with our auditory processing rules & gives a more realistic illusion. So your preamp A could be disappointing you months later because you have become aware of a distortion hidden to you before & found a new benchmark which your preamp isn't meeting.

 

[JackD201]

Here's the problem I have with all of that:
@JackD "Nothing beats a blind test when it comes to reliable results for DIFFERENCES." Differences/distortions are not constantly present - they are dynamic - they won't just pop out & present themselves in blind A/Bing - in fact it's almost certain you will miss them unless you are so lucky that you switch from A to B at the moment of distortion. So the only successful way to use blind A/Bing is to have previously identified the distortion & listen for it in A/Bing. This is done sighted prior to A/Bing. Blind A/Bing is only done as a confirmation - it's not a test to REVEAL differences. That's why John Curl & others do without - by & large they don't need the confirmation.

Now the majority of blind tests I see being run on forums make this same mistake - they think that the "night & day differences" reported by some subjectivist will be so evident in a blind A/B that they use it incorrectly - they use it as a test to REVEAL differences. Of course they get a null result which further reinforces their initial scepticism & ensures their expectation bias (no difference exists) is now set forever because they seldom ever return to the test having "proven" the issue to themselves.

The example I gave earlier of Vital's journey (on PinkFishMedia forum) is a worthwhile lesson in how, in the real world, these tests are used & their pitfalls. He was unusual in that he had enough self-analysis & inquisitiveness to continue his search even after he had done 2 blind test session in which all present heard no differences between DACs, sighted or blind. It was only when he did his fourth blind test session & someone was there that could identify sighted a difference to listen for that he could then do so blind. Somebody dismissed this earlier by saying that he learned how to listen but this is the great problem - we need training to pass these blind A/B tests, otherwise we will never hear the differences & the null blind test will reinforce this & create an nocebo expectation bias that requires exceptional conditions to overcome. He admits that a big part of him not hearing differences, sighted or blind, were due to the blind tests but actually I think it was due to nobody ever pointing out to him the exact thing(s) to listen for & the blind A/B sessions further reinforced & became his expectation bias.

So I don't see the differences between the sighted listening done prior to a blind A/B test & long term listening.

But I do see a major difference between sighted long-term listening & blind A/B testing - we keep a much more open mind in doing long term listening then in blind A/Bing. As I said above blind A/Bing mostly returns null results (due to invalid test conditions) which reinforces a nocebo expectation bias. Long-term listening is exactly the opposite of this - we are attempting to find the audio characteristics of devices & hence we do not form an opinion quickly in a short listening session.

So my advice is blind A/B testing should not be tried at home - it is only for those who know what they are doing.

Come on John. It's not like someone is blind testing blanks and live ammo in a revolver in a room full of people. In the end, done blind or sighted a decision will be made. Whether or not that decision is different from what we might have made, it's none of our concern. If the guy is happy that's all there is too it. If not, we don't get to say "I told you so" either.

Now let's see what happens next. A blind test can help you be more certain if there is a difference or not. Question is, in the affirmative situation, is it better or worse? That would require a lot of listening to familiar material. Good luck doing THAT blind.

 

[jkeny]

Come on John. It's not like someone is blind testing blanks and live ammo in a revolver in a room full of people. In the end, done blind or sighted a decision will be made. Whether or not that decision is different from what we might have made, it's none of our concern. If the guy is happy that's all there is too it. If not, we don't get to say "I told you so" either.

I'm not sure what the "come on John" is in reference to, I know it's not life & death - I'm just relating what I see are the pitfalls of quick A/B blind testing - pitfalls that I believe are not fully acknowledged.

Now let's see what happens next. A blind test can help you be more certain if there is a difference or not.

Yes, I'll say it again, once that difference has been identified in the track (let's say it's two audio recordings that are being tested). This zoning in on the difference requires experience & training or someone points it out to you. Look at Amir's ABX tests where he was cajoled & harassed to provide exact timestamp & difference noted in the audio files. There's no question in my mind that this one of the biggest, unrecognised problems in A/B testing - it's not a test for discovery of differences - it's a sanity check of differences which have already been identified, sighted. If it is used to identify "night & day" differences reported elsewhere, without a handle on what & where those differences will be found - then it is doomed to failure.

 

[jkeny]

Here's something I just came across which states what I'm trying to say but in a more succinct way. It's from the Stuart paper 9174 "The Audibility of Typical Digital Audio Filters in a High-Fidelity Playback System"

ABX tests have a high sensitivity, that is, the proportion of true-positive results out of total positive results is high. However, ABX tests also have low specificity, meaning that the proportion of true-negative results out of total negative results can be spuriously low. Translating this into outcomes in psychophysical tests, the proportion of the time that a listener scores well on an ABX test by chance is low, but the proportion of the time that a listener can score poorly on a test in spite of being able to discriminate the sounds is high. An ABX test requires that a listener retains all three sounds in working memory, and that they perform a minimum of two pair-wise comparisons (A with X and B with X), after which the correct response must be given; this results in the cognitive load for an ABX test being high.

 

[jkeny]

If that new preamp is disappointing me later (for your idea of long term listening this should be what, weeks later at the least? How long is long term listening supposed to be before you can say that there is no issue?)

then it the preamp did not change, my head is still in the vice, and I am playing that same song or songs, and if it does not no sound right, its me who changed, and is the idea that it does not sound right now due to hearing differently, yes, to be more precise the ear/brain mystery center says hey this does not sound right, the same one that said it sounded fine when you auditioned it when it was first in your system.

OK, I didn't realise you were talking about the long-term listening situation. Well you are jumping to an unwarranted conclusion - i.e that it is some change of mind-set that has now made you unhappy with the sound of your preamp rather than what I'm suggesting, your auditory perception has found issues in the sound that you did not pick up on, initially. But we will agree to differ, I guess

How on earth can you believe that mystery brain center when it ITSELF can not notice the sound when you first heard it and liked it.?

Because not everything is immediately noticeable - that's why I said A/B testing is not suitable for some differences.

This is one reason why audiophiles keep churning gear, as they do not understand that their ear brain system is EASILY FOOLED (by your point that it sounded good but after time it did not...and nothing changed but you got a few weeks older) and as audiophiles the industry has BUILT INTO GULLIBLE MINDS that they are always improving stuff

(the truth is they and you too are only changing out a component to develop a new harmonic output spray and thus yes, it could sound different but BETTER is highly debatable. Thus the never ending churn, plant DOUBT in your mind, exclaim an improvement, which is just a different take on the harmonic distortion spectrum output spray or FR etc and there you have it, you hear something different, it sounds good at the time,

but until your inner resolve learns that you are the one changing all the time you are on the road to audio hell.

Unless you like to change out components because humans (me too) do like variety.

Now all this is talking about a solid state preamp as an example.

That's a particular viewpoint I see many objectivists take. I doubt that you deny people can be initially beguiled by the sound of a device & later realise they were wrong - what was initially technicolour sound is actually distortion. Nothing has changed - so is this just their mind changing? So what is different about the more subtle example of your preamp?

 

[jkeny]

Yes, it is my point, if you are bequiled by Technicolor at first, then your internal references are not to be trusted are they? They are not solid at all.

No, no, you are going around in circles, IMO.

Let's use another perceptual example - getting new glasses. I have received new glasses that seemed fine at first but after a few days I noticed headaches - brought them back, they redid new lenses & the new ones were OK, no more headaches. Are you telling me that my reference, my visual perception, was not to be trusted? Do you think I should have immediately known that there was a problem with my glasses? Do you accept that the issue with my visual perception was only noticeable with longer term usage? If these were reading glasses only, do you think it would have taken me longer to notice the problem? Do you think the problem was with the glasses or are you suggesting that the problem was in my head

What I think you are trying to say is that perception is a subtle & malleable sensation & what we sense is prone to lots of factors at play in the moment?

You are incorrectly, IMO, concluding that the perception is the reference - it's not - reality, is the reference i.e the stored model we have of reality (in this case an auditory model). It's this model that is mostly unchanging after we have developed out of infancy. In the case of the glasses, I was not seeing sensing that I wasn't seeing correctly but long term use looking at reality, gave me physiological feedback that all was not right.

 

[jkeny]

I don't believe your eye glasses example holds up as your eye muscles have to change to focus don't they? No expert but maybe hearing is not passive as we thought though either, so maybe our ear hairs muscles change for various reasons, after all these hairs must be held erect somehow....who knows but that does bring more lack of physical body knowledge into the equation, both of us are extrapolating from our own experiences and have come to different conclusions for sure, but neither one of us knows, we are just opining of course.

Sure, we are discussing this drawing on our personal experiences & conclusions that we have come to. Correct about our hearing mechanism - at the physiological level it is not passive but very much a dynamic & active

I just don't think our stored model is a reference that is fixed as much as you think. And the fact that if you go to an audio show and listen to several different sytems and one resonates with you, over all the rest, then all it means to me is at that moment, given those choices, you prefer it. You are saying that after a few months, and I come back to the same exact rooms etc, and listen again, I would choose the same room? I am saying there is a good chance I might just choose a different room because that internal reference is not fixed but varies by time and overall feelings and emotions etc.

I think the stored model is not reality at all. its is a variable subject to changes by mood, health, etc.

Agreed, the stored model is not reality, it is a model of reality in a very synthetic form (the same as our visual model is also a very highly abstracted version of the signals that impinges on our eyes.

Where I think we are talking past one another is maybe in this aspect - there is a set of rules & models that define our internalised auditory library of reality - - for instance we evaluate a sound & conclude things like the size & how far away it is. We can do this, in variously different environments, because we have certain characteristics/rules that is referenced to our internally stored auditory model of the world. This isn't as variable, as you suggest & maybe isn't as fixed in nature, as I suggest - although I believe that, after infancy, it only is added to or changes when we engage in auditory training, otherwise it is fixed & unmoveable. Where I think the gap between us lies is in how we think a mismatch to this internal model will be perceived - you, I suspect believe that this will be immediately noticeable & I maintain that a mismatch is not an all or nothing result - it is a sliding scale from great match to bad match. I'm saying that some mismatches will only become apparent over some time living with the pre-amp.

Here's an interesting paper that is relevant to something JackD posted - "Come on John. It's not like someone is blind testing blanks and live ammo in a revolver in a room full of people". This paper is from the Army Research Laboratory, titled "Feasibility of Audio Training for Identification of Auditory Signatures of Small Arms Fire". I reference this because it very much is a question of life & death - identifying the sound of enemy small arms is crucial in survival & is possible only because of the stored reference of the sound to which "live event" is compared to. This is happening in all sorts of different auditory surroundings.

After World War II and during the Korean War, Katzell et al. (1952) interviewed Combat Arms
veterans in an effort to identify the components that contributed to situational awareness and
target identification. According to surveys collected, the researchers found that the Soldiers
most frequently suggested training to recognize relevant enemy sounds, as visual cues were often
unavailable. Specifically of interest to the veterans were the sounds of enemy vehicles and
weapon firing noises. These Soldiers also indicated that the acoustic signatures of enemy rifles
were so distinct that they did not use captured weapons for fear of fratricide. The authors also
stated that the Soldiers “frequently recommended” having live demonstrations of enemy
equipment, and that these demonstrations should take place in a field setting.

In this case blind listening can be a matter of life & death

 

[esldude]

Now the majority of blind tests I see being run on forums make this same mistake - they think that the "night & day differences" reported by some subjectivist will be so evident in a blind A/B that they use it incorrectly - they use it as a test to REVEAL differences. Of course they get a null result which further reinforces their initial scepticism & ensures their expectation bias (no difference exists) is now set forever because they seldom ever return to the test having "proven" the issue to themselves.

I would laugh heartily at this if I didn't know you were serious.

Let me see, we are told the difference is NIGHT AND DAY, so then mistakenly believe a test designed to test for differences is appropriate. Yeah sure only that doesn't even make sense if you say it real fast without thinking.

So what we are supposed to do a translation of such reported effects.

NIGHT AND DAY a very minimal difference not objectively measrurable or verifiable blind. One only garnered after one week's listening.

QUITE SUBSTANTIAL DIFFERENCE- a difference only perceived after many weeks, and only under the most perfect conditions of personal calmness.

A BIT OF DIFFERENCE- one even your buddies won't hear, but will take your word for it.

NOT SURE WHAT I AM HEARING BUT FEEL I PREFER THIS ONE SLIGHTLY-a difference in quality you aren't even sure of.

Well sorry for the translations of subjective differences described. It is simply laughable to say a NIGHT AND DAY difference is one not revealed because we mistakenly believed it was a big difference. I mean there is hyperbole and there is hyperbole, but WOW>

 

[jkeny]

I would laugh heartily at this if I didn't know you were serious.

Let me see, we are told the difference is NIGHT AND DAY, so then mistakenly believe a test designed to test for differences is appropriate. Yeah sure only that doesn't even make sense if you say it real fast without thinking.

So what we are supposed to do a translation of such reported effects.

NIGHT AND DAY a very minimal difference not objectively measrurable or verifiable blind. One only garnered after one week's listening.

QUITE SUBSTANTIAL DIFFERENCE- a difference only perceived after many weeks, and only under the most perfect conditions of personal calmness.

A BIT OF DIFFERENCE- one even your buddies won't hear, but will take your word for it.

NOT SURE WHAT I AM HEARING BUT FEEL I PREFER THIS ONE SLIGHTLY-a difference in quality you aren't even sure of.

Well sorry for the translations of subjective differences described. It is simply laughable to say a NIGHT AND DAY difference is one not revealed because we mistakenly believed it was a big difference. I mean there is hyperbole and there is hyperbole, but WOW>

Yes, I've seen this sneering, condescending attitude before - one that often blinds to the learnings from psychoacoustics.

The concept of auditory stream formation is the area where "night & day" differences can arise. A small change can snap auditory streams into focus - this small change, such as a small drop in electrical noise can have widespread, perceptually important consequences (see the thread)

I'm open to being wrong on this but I suspect that all things are audibly identifiable in quick A/B listening is mistaken. One is usually focussed on identifying small differences between specific parts of the two A & B. This focus determines what is heard & excludes analysis of areas outside of this focus. You will probably say that quick A/B testing can be used in the manner I'm suggesting - just play A all through, then play B all through & decide if any differences can be sensed. Are you not then contorting the quick A/B test into something else - it's no longer quick & no longer using echoic auditory memory - so what's the point? Why not just say listen to A then listen to B - do this multiple times over weeks if necessary & decide if you hear a difference.

But would you then accept this as "evidence" or demand that the difference be "proven" in quick A/B testing?

 

[esldude]

Yes, I've seen this sneering, condescending attitude before - one that often blinds to the learnings from psychoacoustics.

The concept of auditory stream formation is the area where "night & day" differences can arise. A small change can snap auditory streams into focus - this small change, such as a small drop in electrical noise can have widespread, perceptually important consequences (see the thread)

I'm open to being wrong on this but I suspect that all things are audibly identifiable in quick A/B listening is mistaken. One is usually focussed on identifying small differences between specific parts of the two A & B. This focus determines what is heard & excludes analysis of areas outside of this focus. You will probably say that quick A/B testing can be used in the manner I'm suggesting - just play A all through, then play B all through & decide if any differences can be sensed. Are you not then contorting the quick A/B test into something else - it's no longer quick & no longer using echoic auditory memory - so what's the point? Why not just say listen to A then listen to B - do this multiple times over weeks if necessary & decide if you hear a difference.

But would you then accept this as "evidence" or demand that the difference be "proven" in quick A/B testing?

I'll ignore most of the thoughts you are thinking on my behalf.

Okay, sure play an entire hour of music followed by another hour. If you do that with enough samples and it gets positive results fine no argument from me. I would expect the same thing would get positive results in a short comparison too. But if it doesn't then it doesn't. Such a result would be of very high interest. If you did a long term and a short term test and got results long term and not short term I would not dismiss the long term results. Would be a very interesting thing to figure out why it was. But if the test is done well then no problem. I haven't said or implied the results have to be from short quickly switched AB comparisons. Only that such short comparisons seem to have more discrinatory power than longer ones. You keep insisting long term listening is better without evidence.

 

[Whatmore]

I'll ignore most of the thoughts you are thinking on my behalf.

Okay, sure play an entire hour of music followed by another hour. If you do that with enough samples and it gets positive results fine no argument from me. I would expect the same thing would get positive results in a short comparison too. But if it doesn't then it doesn't. Such a result would be of very high interest. If you did a long term and a short term test and got results long term and not short term I would not dismiss the long term results. Would be a very interesting thing to figure out why it was. But if the test is done well then no problem. I haven't said or implied the results have to be from short quickly switched AB comparisons. Only that such short comparisons seem to have more discrinatory power than longer ones. You keep insisting long term listening is better without evidence.

He has a "feeling", that's all the evidence required.
Blind tests on the other hand, require a completely different standard.

 

[jkeny]

He has a "feeling", that's all the evidence required.
Blind tests on the other hand, require a completely different standard.

I thought you understood that what is being talked about in the post you quoted, is quick Vs long-term A/B testing? Your statement "Blind tests on the other hand, require a completely different standard don't show this.
You certainly understood this before but have relapsed now

 

[jkeny]

As I said & I'll say it again with a bit more detail:
- with a bit of knowledge about psychoacoustics, we realise that what we perceive in audio is a signal stream moving through time & we continuously parse & organise this into auditory streams - these are what make sense of what we hear. Now when we enter a room it takes between 5-15 seconds to build to a final analysis & correctly group what we hear into audio streams. Now people here supporting quick A/B switching suggest that the speed of switching allows us to be more discerning in spotting differences because we are able to retain what we last heard for 3-5S in auditory echoic memory & therefore we can directly compare, the details of what was just heard with the detail of what is now being heard.
- This doesn't hold for auditory stream formation which can take up to 15s to build into a fully formed perceptual stream. In this 15S window the previous audio clip has long since disappeared from our echoic memory. So if there is a difference between the two samples A & B that cause differences in auditory stream formation we will not easily perceive it in quick A/B testing.

- The most effective way I know of sensing these sorts of differences is by using long term listening.

- does it not seem odd to anyone that the suggested test (quick A/B) is an audio snapshot for trying to identify differences in a constantly changing signal that we continuously parse into something that makes auditory sense - where signals separated by time are linked together to form auditory streams?

I believe that it's time to move beyond the simplistic tests & test signals that have served us well enough up to now. It's time to learn from what's known in psychoacoustics & adopt a more sophisticated view of both measurement & testing.

But somehow, based on the sneering & condescension coming from objectivists on this thread, it's obvious that they are not open to new information or learning (I predict another sneering remark in answer to this).

 

[Whatmore]

why can't you do long term listening blind?

 

[jkeny]

why can't you do long term listening blind?

You seem stuck on this - I said that wasn't the important issue - in fact I think it makes little to no difference if it's blind or sighted.
We already discussed this before, I think?

 

[Whatmore]

its hard to remember what we've discussed to be honest

so you think that long term blind tests will give the same results as long term sighted tests?

 

[treitz3]

why can't you do long term listening blind?

Hmmm, some members here shun on long term listening and the memory thereof.

Tom