Objectivist or Subjectivist? Give Me a Break

[terryj]

If you believe in the DBT's that say that MP3's sound the same as CD, or that a 1970's solid state amp is state of the art because DBT's can't show any difference, you are not an objectivist. You are a fool.

Your biases are showing.

MP3? All mp3 is the same? 98 then I guess all would agree with you. Higher rates not so much (or so they say). In any case you have not even defined the 'test'.

What I find more interesting is how this slipped past everybody without comment.

Blind testing does not overcome biases. There are some which can not be overcome by blind testing - e.g. that all amps sound the same. If you subject 1,000 people with this belief to a blind test, your result will be: all amps sound the same.

Why then would you test that group? After all it is not they making the claim of 'differences in sound' between amps. If you felt the need to test that group, how about sticking a honking great resistor and or cap on the output of one of the amps, something that would definitely change the sound. Would they still claim to hear no difference? (this would fall under the category of 'selecting the group', those that show an ability to hear differences.) I doubt people would deny a difference like that simply to maintain their position.

Successively reduce those components (to zero eventually), and BOTH groups would have to state the point at which no difference could be detected. (or find that even with those parts removed a difference was still heard)

Anyways, the point is to test the claimants surely?

 

[Whatmore]

Your biases are showing.

MP3? All mp3 is the same? 98 then I guess all would agree with you. Higher rates not so much (or so they say). In any case you have not even defined the 'test'.

What I find more interesting is how this slipped past everybody without comment.



Why then would you test that group? After all it is not they making the claim of 'differences in sound' between amps. If you felt the need to test that group, how about sticking a honking great resistor and or cap on the output of one of the amps, something that would definitely change the sound. Would they still claim to hear no difference? (this would fall under the category of 'selecting the group', those that show an ability to hear differences.) I doubt people would deny a difference like that simply to maintain their position.

Successively reduce those components (to zero eventually), and BOTH groups would have to state the point at which no difference could be detected. (or find that even with those parts removed a difference was still heard)

Anyways, the point is to test the claimants surely?

I imagine that many of these claimants may also feel that they would not pass a DBT. Wouldn't that skew the results in favor of "no difference"?

 

[jkeny]

Of course it can be overcome. All it takes is sound testing methodology. Two very simple ways to overcome the bias you're talking about are to screen the participants, and to not tell them what they're listening for. Give them a simple AB/X, with no more information. Problem solved. That doesn't mean another one won't appear, of course, but most of them can be pretty effectively addressed.
Tim

Tim, even though your answer wasn't for me I'll bite - I was talking about our own personal tests that we use to evaluate our systems, not formal, ABX testing which frankly I don't see being done too often, do you, especially in the manner that you suggest. So for the normal blind testing, negative bias that all amplifiers/DACs/etc. are the same is a trump card that is impossible to overturn. This fact has to be recognised for what it is. Normal sighted biases can easily be challenged by blind testing but this one is immovable.

 

[Phelonious Ponk]

Tim, even though your answer wasn't for me I'll bite - I was talking about our own personal tests that we use to evaluate our systems, not formal, ABX testing which frankly I don't see being done too often, do you, especially in the manner that you suggest. So for the normal blind testing, negative bias that all amplifiers/DACs/etc. are the same is a trump card that is impossible to overturn. This fact has to be recognised for what it is. Normal sighted biases can easily be challenged by blind testing but this one is immovable.

No, of course, personal testing will not have the scientific discipline. It won't have the numbers (trials) needed for statistical requirements, if nothing else. And yes, expectation bias could be negative. You could talk yourself into hearing nothing. I haven't been able to do that yet, but maybe I just haven't made a persuasive argument to myself.

Tim

 

[microstrip]

In my world there is no controversy to using listening tests and measurements. There are problems with using one or the other. I simply fail to see the issue. As I said in my article . . .

Jeff,

The big issue is correlating the measurements with the results of the listening tests for evaluation and specially for ranking purposes. IMHO there is nothing worst than a biased analysis of an incomplete suite of the measurements, specially because it usually is written in a technically authoritative style.

 

[Keith_W]

Your biases are showing.

Remind me who has the tertiary degree with the postgraduate qualifications and published articles in scientific journals, and who is the unemployed house-husband in the middle of country NSW again please?

 

[Steve Williams]

Remind me who has the tertiary degree with the postgraduate qualifications and published articles in scientific journals, and who is the unemployed house-husband in the middle of country NSW again please?

ouch

 

[Phelonious Ponk]

Jeff,

The big issue is correlating the measurements with the results of the listening tests for evaluation and specially for ranking purposes. IMHO there is nothing worst than a biased analysis of an incomplete suite of the measurements, specially because it usually is written in a technically authoritative style.

I hope you don't have the Harman study in mind, micro, because that one was very well designed and executed, and it correlated measurements to listening tests conducted specifically for preference ranking purposes. If that is what you have in mind, I'd love to hear about the "biased analysis."

Tim

 

[flez007]

Blind testing does not overcome biases. There are some which can not be overcome by blind testing - e.g. that all amps sound the same. If you subject 1,000 people with this belief to a blind test, your result will be: all amps sound the same.

+1

 

[Robh3606]

The big issue is correlating the measurements with the results of the listening tests for evaluation and specially for ranking purposes. IMHO there is nothing worst than a biased analysis of an incomplete suite of the measurements, specially because it usually is written in a technically authoritative style.

Hello Micro

What's a complete set of measurements? I would bet you that if you asked 10 different designers they would not all choose the the same ones and they would weigh them differently. A perfect example would be phase accuracy where one camp considers it paramount the other considers it irrelevant. That's just the way it is.

As far as listening tests as long as you have a standard to compare against along with the measurements I don't see an issue. I am with Jeff you need both measurements and listening tests and one is not a complete evaluation without the other.

Rob

 

[rbbert]

...
As far as listening tests as long as you have a standard to compare against along with the measurements I don't see an issue. I am with Jeff you need both measurements and listening tests and one is not a complete evaluation without the other.

Rob

Fortunately I think this is where most audiophiles (and nearly all, if not all manufacturers) live. I'm not so sure about most reviewers...

 

[microstrip]

Hello Micro

What's a complete set of measurements? I would bet you that if you asked 10 different designers they would not all choose the the same ones and they would weigh them differently. A perfect example would be phase accuracy where one camp considers it paramount the other considers it irrelevant. That's just the way it is.

As far as listening tests as long as you have a standard to compare against along with the measurements I don't see an issue. I am with Jeff you need both measurements and listening tests and one is not a complete evaluation without the other.

Rob

Rob,

Even worst - if you ask the best designers they will not answer. . Just think at top amplifiers being recently referred at WBF - D'Agostino Momentum, Dartzeel NHN458, Burmester 911 mk3 and Audio Research REF250 (yes, the list is not complete, apologies if I forgot your favorite . Can you get an answer of any of them?

Anyway I would love if any reviewer could correlate the measurements of these amplifiers with their subjective sound quality - but sorry I do not accept they all sound the same!

Sorry to be so negative, but we always have fantastic methodologies, but as far as I ask for solid printed examples of correlations in electronics I get no answers, either then the SET case!

 

[Soundproof]

There is much crazytalk to enjoy when Goodwin chooses to display her strange lack of knowledge of things HiFi. Her epic DAC nonsense comes to mind, but this article aims for the Olympos of Houyhnhnm chatter.

 

[Phelonious Ponk]

Anyway I would love if any reviewer could correlate the measurements of these amplifiers with their subjective sound quality - but sorry I do not accept they all sound the same!

I don't think anyone's askiing your to accept that all amplifiers sound the same, but if you're looking for measurement very closely correlating to listening, look no farther than the Harman speaker study discussed here for many days last month. If listeners can hear FR variations recorded in measurements and if their preferences correlate to those measurements, why would it be any different with audible differences in amplifiers, dacs, etc?

Tim

 

[jkeny]

I don't think anyone's askiing your to accept that all amplifiers sound the same, but if you're looking for measurement very closely correlating to listening, look no farther than the Harman speaker study discussed here for many days last month. If listeners can hear FR variations recorded in measurements and if their preferences correlate to those measurements, why would it be any different with audible differences in amplifiers, dacs, etc?

Tim

Because FR measurements aren't the full determinant of what we hear - you seem to think they are?

 

[FrantzM]

I don't think anyone's askiing your to accept that all amplifiers sound the same, but if you're looking for measurement very closely correlating to listening, look no farther than the Harman speaker study discussed here for many days last month. If listeners can hear FR variations recorded in measurements and if their preferences correlate to those measurements, why would it be any different with audible differences in amplifiers, dacs, etc?

Tim

Because FR measurements aren't the full determinant of what we hear - you seem to think they are?

Jkeny

Is that what you read in Tim's post ? That FR is the determinant?


Strawman argument comes to mind ....

 

[jkeny]

Jkeny

Is that what you read in Tim's post ? That FR is the determinant?


Strawman argument comes to mind ....

OK, let's see - he states "that if listeners can hear FR variations recorded in measurements" so we are talking about the frequency domain exclusively, right?
"If their preferences correlate to those measurement" so he now has a correlation between measurements & listener preferences, right?

Now take the next statement in his answer & tell me what the flaw is?

 

[Ron Party]

This thread is full of IT, i.e., straw men and gross generalizations, mischaracterizations and stereotypes, all of which began with the pathetic Goodwin article forming the basis for this thread. Frankly I'm stunned her article was published at all and ashamed that our membership gives it the slightest light of day. Regrettable, because it lowers the level of discourse and intellectual honesty.

And, BTW, with proper test implementation it is trivial to weed out those who are predisposed to claim to hear no difference, just as it is trivial to weed out those who will always claim to hear a difference. This is and has been done with regularity.

 

[jkeny]

Ron, are you saying my post is a strawman argument or just making a generalisation about the thread's contents?

 

[Robh3606]

If listeners can hear FR variations recorded in measurements and if their preferences correlate to those measurements, why would it be any different with audible differences in amplifiers, dacs, etc?

Hello Tim

Yes for sure the only difference is the magnitude of the frequncy response errors. If an amp ever measured like some of the speakers in Tooles study it would be laughed at and not taken seriously. Seems speakers are expected to be colored to some extent amps and dacs s/b within a .1db across the whole spectrum.

That could be why it is so hard to hear differences readily between them.

all of which began with the pathetic Goodwin article

Hello Ron

She seems to be really good at them.

Rob