Another playback software to argue over

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Tim,

What is the "audibility control" you want to add to Meyer and Moran? As far as I remember they already had one and were happy with it!

It was suggested here recently that M&M was invalid because it lacked a control testing the ability of the participants to hear subtle differences known to be audible. While I agree that would be a good thing to add, I seriously doubt the results were compromised by its omission. Lots of trials, variety of systems, listening material, a broad variety of well-screened listeners from civilians to audiophiles to pros...there was a lot of "control" in the study to avoid the hearing impaired and to compensate for individual weaknesses, just not that one in particular.

Tim
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
(...) Micro, was document BS1116 not prescribed as the procedure for such blind tests? But in general I agree with you, those who constantly cry DBT really don't want it, they just want an argument.

Yes, but I am sure that the supporters of near field and headphone listening will not like it ... Anyway, BS1116 does not have recipes, it is mainly a very good cooking book. ;) Even the statistical analysis of such data is not something you carry just reading a few posts in the net.
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
It was suggested here recently that M&M was invalid because it lacked a control testing the ability of the participants to hear subtle differences known to be audible. While I agree that would be a good thing to add, I seriously doubt the results were compromised by its omission. Lots of trials, variety of systems, listening material, a broad variety of well-screened listeners from civilians to audiophiles to pros...there was a lot of "control" in the study to avoid the hearing impaired and to compensate for individual weaknesses, just not that one in particular.

Tim

Tim,
IMHO, you should read the original M&M documentation and the extra details given later by the authors in a followup before commenting on it. I will not loose my time with what I consider a flawed test debating net suggestions or "control" between commas.
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
In previous talks about DBTs etc. I asked if any tests had been done which analysed the emotional effect of different sound reproduction rather than the usual focus on named differences. I directed this to JJ but got no reply.

I subsequently came across references to a German psychologist, Jürgen Ackermann, whose PhD thesis addressed this Jürgen Ackermann - PhD Thesis at the Frankfurt Hochschule für Musik und Darstellende Kunst (Music and Performing Arts University). I can't find his thesis on-line but a fair amount of details of the controlled tests are given here http://www.stereophile.com/content/god-nuances-page-3 which might act as a counterfoil & datapoint for the conduct of DBTs?
Ackermann found 53 people from all walks of life willing to participate in his experiment: hi-fi enthusiasts, musicians, and "normal" people with no special relation to music or its reproduction. The selection of participants was not truly stochastic, but the sample was large enough to give meaningful results.

Just a summary of some results
The participants began with a base tension level of 3.26; with the digital system this dropped to 2.35, and with the analog system to 1.75. Nervousness was raised from a base level of 1.8 to 2.2 by the digital system, but fell to 1.1 with the analog system. The need for relaxation fell from a base level of 2.6 to 1.9 with the analog system, but rose to 2.9 with the digital system. The ability to concentrate remained constant with the analog system at 4.3, but fell to 3.6 with the digital system. Relaxedness stayed constant with the digital system at 4.0, but rose to 4.6 with the analog system.

And some further results
Forty-seven participants said the music had improved their sense of well-being via the analog system, 31 via the digital.

Conversely, no participant said that the analog system had impaired their sense of well-being, but 16 participants said so of the digital system!
 
Last edited:

Occam

[Industry Expert]
Dec 15, 2010
117
1
0
NYC
Jkeny,

Thanks for the reference to the Stereophile article. It is truly a wonderful example of **** poor experimental design. I'd naively thought one should only change one thing in an experiment, but I guess not.
Heck, I could set up 2 systems with the same components, save for the source, and rig it easily so that the digital imparts that 'well being' and 'relaxedness' and the phono imparts 'stessedness'. Just pick components from Tim dPVC (EAR) and choose to populate with Mullard E88CCs or CBS specified Siemens E88CCs. I can easily configure for an opium dream or a meth induced nightmare.

I'm not disputing the truth of the conclusions, just that that experiment in any way confirms them.
FWIW
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
In previous talks about DBTs etc. I asked if any tests had been done which analysed the emotional effect of different sound reproduction rather than the usual focus on named differences. I directed this to JJ but got no reply.

I subsequently came across references to a German psychologist, Jürgen Ackermann, whose PhD thesis addressed this Jürgen Ackermann - PhD Thesis at the Frankfurt Hochschule für Musik und Darstellende Kunst (Music and Performing Arts University). I can't find his thesis on-line but a fair amount of details of the controlled tests are given here http://www.stereophile.com/content/god-nuances-page-3 which might act as a counterfoil & datapoint for the conduct of DBTs?


Just a summary of some results

And some further results

I'm not jj, but the speaker tests run by Toole, then Olive, at the Canadian research labs, then Harman, tested for and analyzed preference. That is emotional, perceptual, not analytical.

Tim
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Tim,
IMHO, you should read the original M&M documentation and the extra details given later by the authors in a followup before commenting on it. I will not loose my time with what I consider a flawed test debating net suggestions or "control" between commas.

And I'm not interested in debating it with you, micro. You asked for an example; I gave you one. I'm not at all surprised that you've already dismissed it, or that you've questoned the validity of my opinion of it without supplying anything substantive to support your own.

Tim
 

Gregadd

WBF Founding Member
Apr 20, 2010
10,517
1,774
1,850
Metro DC
Sometimes having multiple vatiables in a test is unavoidable.
 

FrantzM

Member Sponsor & WBF Founding Member
Apr 20, 2010
6,455
29
405
Jkeny,

Thanks for the reference to the Stereophile article. It is truly a wonderful example of **** poor experimental design. I'd naively thought one should only change one thing in an experiment, but I guess not.
Heck, I could set up 2 systems with the same components, save for the source, and rig it easily so that the digital imparts that 'well being' and 'relaxedness' and the phono imparts 'stessedness'. Just pick components from Tim dPVC (EAR) and choose to populate with Mullard E88CCs or CBS specified Siemens E88CCs. I can easily configure for an opium dream or a meth induced nightmare.

I'm not disputing the truth of the conclusions, just that that experiment in any way confirms them.
FWIW

+ 1
 

JackD201

WBF Founding Member
Apr 20, 2010
12,308
1,425
1,820
Manila, Philippines
I'm not jj, but the speaker tests run by Toole, then Olive, at the Canadian research labs, then Harman, tested for and analyzed preference. That is emotional, perceptual, not analytical.

Tim

Where did they even suggest that?
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
I know it's semantics Tim but "Expectation Bias" isn't always the right term to use. There are other biases that fit the varying situations. Google 'em. I'm trying to be nice too.
..
In the strictest sense, Mark is right here. I'm no stranger to new gear crush. Definitely not immune to it. In my experience, given time, any crush will fade and you're left with what you've got. I dare say that growing into nasty stuff is extremely rare. Not that that means one will swap that piece of gear out but rather steps are usually taken to mitigate the negative aspects of that piece. It could be as simple as speaker repositioning or as drastic as DSP. If something doesn't sound right no justification or rationalization will ever make it sound right. You have a choice of buyer remorse or taking it on the chin and set about finding proactive solutions.

And you're doing a fine job, Jack. I know there are other biases, but while I'm not an expert on the subjec I think what I'm talking about fits expectation bias pretty well -- based on cost, brand, looks, reputation, build quality...you expect it to sound good. And so it does.

Tim
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
And I'm not interested in debating it with you, micro. You asked for an example; I gave you one. I'm not at all surprised that you've already dismissed it, or that you've questoned the validity of my opinion of it without supplying anything substantive to support your own.

Tim

Tim,

You are trying to mix the control of the capabilities of test - what we call positive and negative tests - with the "controls" to check the hearing impaired and to compensate for individual weaknesses. It is not new - and was already argued in several forums. IMHO nothing can be debated in these conditions - as they say apples and oranges.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Tim,

You are trying to mix the control of the capabilities of test - what we call positive and negative tests - with the "controls" to check the hearing impaired and to compensate for individual weaknesses. It is not new - and was already argued in several forums. IMHO nothing can be debated in these conditions - as they say apples and oranges.

No, that's not what I'm trying to do, but I'm done trying to explain it to you.

Tim
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
I'm not jj, but the speaker tests run by Toole, then Olive, at the Canadian research labs, then Harman, tested for and analyzed preference. That is emotional, perceptual, not analytical.

Tim

The speaker tests carried by Toole, then Olive, at the Canadian research labs, then Harman, were of very diverse types and targeted at very diverse objectives - unless you specify what type you are specifically addressing this comment can not be understood and IMHO is confusing and meaningless.
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
And you're doing a fine job, Jack. I know there are other biases, but while I'm not an expert on the subjec I think what I'm talking about fits expectation bias pretty well -- based on cost, brand, looks, reputation, build quality...you expect it to sound good. And so it does.

Tim

Tim-Again I say that expectation bias that leads you to think that what is new that you bought based on cost, brand, reputation, and build quality sounds great either will sound great over the long-term or it won't. Those with decent ears will be honest enough to admit they made a mistake if over time they come to realize the new expensive brand name gizmo with great build quality doesn't sound as good as what it replaced.
 

JackD201

WBF Founding Member
Apr 20, 2010
12,308
1,425
1,820
Manila, Philippines
And you're doing a fine job, Jack. I know there are other biases, but while I'm not an expert on the subjec I think what I'm talking about fits expectation bias pretty well -- based on cost, brand, looks, reputation, build quality...you expect it to sound good. And so it does.

Tim

Sorry Tim. Cognitive biases are specific. Expectation biases deal with people performing experiments and affect the experiments eventual results by rejective data that is in conflict. There are biases that are more apt since audiophiles who buy gear for whatever reason aren't performing experiments. Choice Supportive Bias, Post-Purchase Rationalization, Hot Hand, Bandwagon, Belief Bias, etc. would better fit what your view is.

Thing is many folks use Expectation Bias as a catch all and it isn't. In this particular case where the focus is on duration of the evaluations, I think that Mark is right as the situation you describe are people undergoing Post Purchase Rationalization with the purchase not triggered by expectation bias but rather Choice Support, Hot Hand (last product was great, his newer product must be even better), Bandwagon (everybody at school has got one), etc.

So in my mind, there will always be room for all kinds of evaluation and tests. THE bias we all have to watch out for IMO is the Curse of Knowledge. ;)
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Sorry Tim. Cognitive biases are specific. Expectation biases deal with people performing experiments and affect the experiments eventual results by rejective data that is in conflict. There are biases that are more apt since audiophiles who buy gear for whatever reason aren't performing experiments. Choice Supportive Bias, Post-Purchase Rationalization, Hot Hand, Bandwagon, Belief Bias, etc. would better fit what your view is.

Thing is many folks use Expectation Bias as a catch all and it isn't. In this particular case where the focus is on duration of the evaluations, I think that Mark is right as the situation you describe are people undergoing Post Purchase Rationalization with the purchase not triggered by expectation bias but rather Choice Support, Hot Hand (last product was great, his newer product must be even better), Bandwagon (everybody at school has got one), etc.

So in my mind, there will always be room for all kinds of evaluation and tests. THE bias we all have to watch out for IMO is the Curse of Knowledge. ;)

Thanks for the information. I think I'll just refer to it all as "bias" then, if you don't mind. The relatively benign "expectation" has gotten me in enough trouble. Post purchase rationalization and bandwagon are liable to get me lynched.

Tim
 

JackD201

WBF Founding Member
Apr 20, 2010
12,308
1,425
1,820
Manila, Philippines
Hahahahahaha yeah, actually that would be better!
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing