Double Blind tests *did* show amplifiers to sound different

[Whatmore]

whats the big deal? is there any significance to different amps sounding different?

You know not what you ask

 

[MylesBAstor]

whats the big deal? is there any significance to different amps sounding different?

Is that a question or an answer?

 

[dingus]

Is that a question or an answer?

a question, though somewhat rhetorical. i dont get why it would be notable that different amps would sound different. to me it would be the norm. if the OP was in jest, i didnt get it.

 

[MylesBAstor]

a question, though somewhat rhetorical. i dont get why it would be notable that different amps would sound different. to me it would be the norm. if the OP was in jest, i didnt get it.

One needs to know the long history of DBX testing. Maybe to us all amps sound different but not to the ABX crew. All amps sound the same unless used outside their normal operating parameters.

 

[Orb]

A couple of points:

1. The test shows the importance of selecting the right clips that reveal the problem at hand. Two of the clips were highly revealing but the other was not at all. In this situation the selection was likely made at random. So it was a luck of the draw as the saying goes that two clips were revealing. They could have picked two other and arrived at the conclusion of no difference.

This is a problem. When we go through the trouble of setting up a test like this, we need to look at the hypothesis at hand. That hypothesis should not just be a layman one of "let's see if we can make this point or that." Instead one should think through what "1% clipping" could do and find content that makes it revealing. Without a hypothesis, you wind up using random files and getting potentially random results.

2. I think the protection circuits in amplifiers need to be examined more. They are complex beasts, performing dynamic current limiting for milliseconds at a time. The nature of negative feedback system is that it will fight this mechanism. The current limiter puts the brakes on the output stage of the amplifier. That creates distortion which the negative feedback sends to the input of the amp. That circuit then tries to force the output stage to do what it is demanding, i.e. push it harder. That in turn causes the protection circuit to current limit more.

As if that were not enough, majority of amplifiers use unregulated power supplies for the output stage of the amplifier. That voltage sags under load creating yet another indirect feedback loop. Now we have three feedback loops none of which is considering the other in the system. Such behavior simply cannot be simulated or predicted. Testing is also very hard since as noted, speaker loads are different and cause the current limiting to act differently. And of course that factor is also content dependent.

In the case of severe limiting, none of this matters. But when limiting is undetectable as such, then there can be coloration.

Yeah very good points Amir.
I am sure I read an article by J. Gordon Holt about his own experience and his conclusion was that the music must be very carefully selected to "engineer" abberations-traits-emphasised aspects in the recording (such as sibilance - my thoughts are exactly the same and many go into abx without said preparation and understanding-listening of the audio gear involved and selecting music to engineer selected variables as outlined in previous sentence.
In a way this "engineer abberations" makes the ABX more like the normal AFC in just noticable difference where the person has to select under a very specific factor-question (which has more noise/distortion in the picture/etc); this actually helps with bias selection in A/B (which posted research on quite a long time ago).

Point 2 I feel strongly highlights the implication of device dissipation/equivalent peak dissipation resistance (see earlier post in this thread for article link that also mention research done on said subject so has been known for many years), which does not occur with the usual clipping/loudness-watts.

Cheers
Orb

 

[dingus]

One needs to know the long history of DBX testing. Maybe to us all amps sound different but not to the ABX crew. All amps sound the same unless used outside their normal operating parameters.

maybe. 30 years ago i would like my chances at picking Yamaha, Pioneer, Marantz and Sansui from one another.

 

[DonH50]

So did I, and I was wrong, along with a lot of other folk. It was an astonishing and humbling revelation.

It is pretty easy to tell a tube amp from a SS amp with almost any speaker.

With some speakers it is easy to tell among various SS amps, especially difficult loads and a range of amplifiers.

With many speakers and a large range of amplifiers it is difficult if not impossible to pick them out in a double-blind (or just blind) or ABX test.

IMO/IME those who say all amps are alike and those who say all sound different are both at the extremes and have not heard enough amp/speaker combinations.

 

[dingus]

So did I, and I was wrong, along with a lot of other folk. It was an astonishing and humbling revelation.

It is pretty easy to tell a tube amp from a SS amp with almost any speaker.

With some speakers it is easy to tell among various SS amps, especially difficult loads and a range of amplifiers.

With many speakers and a large range of amplifiers it is difficult if not impossible to pick them out in a double-blind (or just blind) or ABX test.

IMO/IME those who say all amps are alike and those who say all sound different are both at the extremes and have not heard enough amp/speaker combinations.

and we know that its been demonstrated that ss can sound indistinguishable from tubes, and yes different speakers will reveal differences in amps as well. however, i dont think that those comparisons are meaningful, rather i think some ABX testing has shown that some amps can sound the same, or different, some of the time.

 

[MylesBAstor]

and we know that its been demonstrated that ss can sound indistinguishable from tubes, and yes different speakers will reveal differences in amps as well. however, i dont think that those comparisons are meaningful, rather i think some ABX testing has shown that some amps can sound the same, or different, some of the time.

I suggest you Google Arnie Krueger and ABX.

Or buy Ethan's book.

Or swing by Hydrogen Audio for S&Gs.

 

[Vincent Kars]

There is something to be said in doing things wrong the exact same way every time.

That's what is called experience

 

[Orb]

So did I, and I was wrong, along with a lot of other folk. It was an astonishing and humbling revelation.

It is pretty easy to tell a tube amp from a SS amp with almost any speaker.

With some speakers it is easy to tell among various SS amps, especially difficult loads and a range of amplifiers.

With many speakers and a large range of amplifiers it is difficult if not impossible to pick them out in a double-blind (or just blind) or ABX test.

IMO/IME those who say all amps are alike and those who say all sound different are both at the extremes and have not heard enough amp/speaker combinations.

The problem though Don is the approach IMO, to tell differences you need to have variables beyond timbre-tone-attack-decay of "normal" music/notes from a played track.
In reality one is creating identifiable cues from aspects that stand out, such as noise, possible behaviour of certain sibilance, badly mixed in terms of phase,etc (other possible traits-characteristics beyond just these) and using just a small snapshot of the music track (instead of most of the music track) around that selected piece for its behaviour with the amps.
And this is further compounded that those doing ABX then try to do this with 15+ different pieces of music.
The reality should be; if a listener can successfully A/B using one track repeatedly that has such a cue/trait as I mentioned then it needs to be investigated why two amps behave differently with that and then also investigate its measurement in relation to the standard spec ones, and then move on to next based upon findings.

Just my take from doing this myself, subtle audio ABX requires a very methodical approach and engineering because in reality the differences in general are not night and day (appreciate you and quite a few others here would agree in general differences are usually very subtle and not as large as they seem when done sighted).
That said preferences can make it seem larger due to tolerance/thresholds but these and preference are usually outside of the ABX audio tests that are bantered about on the various forums; which is ironic considering just how much of the extensive studies done by Harman involve blind A/B product preferences with objective investigation into why.

Cheers
Orb

 

[Phelonious Ponk]

I fail to understand how a non scientific test can be an humbling experience. IMHO those making incredible claims are as guilty as those using childish challenges to win bets ...

It's not hard to understand. You swear you hear an obvious difference. You trust your ears. But when you remove the visual reference, the difference is not only not obvious, it is difficult (or impossible) for you to discern which is which. Do you still trust your ears or do you need a controlled, scientific study? Been there done that. It was humbling. And it made me into a very different kind of audiophile. But I'm still not one who believes all amplifiers sound the same.

Tim

 

[dingus]

I suggest you Google Arnie Krueger and ABX.

Or buy Ethan's book.

Or swing by Hydrogen Audio for S&Gs.

to what end, more tail chasing? i'll grow bored of the subject long before i learn anything on my own. i plea for the forum shortcut where the experience of the community can inform the ignorant more efficiently than any personal research can, i really dont care enough to exert that much effort. if i can point out a couple of extremes...

ABX testing is definitely a useful tool, but if i cant walk away from one knowing which amp i "liked best" or which "sounded best", then its not much use to me. i want to be able to listen for myself, and be happy that i've made the best choice for me when i bring a component home. i dont need some demo ruining my happiness for me.

and without applying proven and accepted scientific testing and analysis, how can one know that they've made the best choice for them? we are all free to wallow in our ignorance, if thats what makes us happy.

...then,

imo ABX isnt adequate to be universally applied. they can show the presence of, or lack of discernible differences, but as an end-listener i havent found that to be particularly useful.

 

[Vincent Kars]

Do you still trust your ears or do you need a controlled, scientific study?

I’m inclined to say:
we can safely trust our ears. What else can we do?
We can safely mistrust our cognition. What else can we do?

 

[MylesBAstor]

to what end, more tail chasing? i'll grow bored of the subject long before i learn anything on my own. i plea for the forum shortcut where the experience of the community can inform the ignorant more efficiently than any personal research can, i really dont care enough to exert that much effort. if i can point out a couple of extremes...

ABX testing is definitely a useful tool, but if i cant walk away from one knowing which amp i "liked best" or which "sounded best", then its not much use to me. i want to be able to listen for myself, and be happy that i've made the best choice for me when i bring a component home. i dont need some demo ruining my happiness for me.

and without applying proven and accepted scientific testing and analysis, how can one know that they've made the best choice for them? we are all free to wallow in our ignorance, if thats what makes us happy.

...then,

imo ABX isnt adequate to be universally applied. they can show the presence of, or lack of discernible differences, but as an end-listener i havent found that to be particularly useful.

Why are you arguing with me? I'm the last person to defend blind testing and I've posted my comments about its issues on the board many times.

You wrote:

whats the big deal? is there any significance to different amps sounding different?

You wrote as if you weren't up on Krueger and crew's ABX work and all I was trying to do is provide some background.

So there's really no need to lecture me about ABX testing.

 

[microstrip]

It's not hard to understand. You swear you hear an obvious difference. You trust your ears. But when you remove the visual reference, the difference is not only not obvious, it is difficult (or impossible) for you to discern which is which. Do you still trust your ears or do you need a controlled, scientific study? Been there done that. It was humbling. And it made me into a very different kind of audiophile. But I'm still not one who believes all amplifiers sound the same.

Tim

Tim,

You swear you hear an obvious difference. Then you take a NON valid blind test that shows you are not able to confirm the difference in the blind condition. IMHO the test is humbling only if you accept that the NON valid test proves anything. Should we feel humiliated if we fail a Coca/Pepsi test ?

I know about the dangers of sighted listening. But I have listened to many excellent systems assembled using sighted listening and never heard about an excellent system assembled using only proper blind tests.

Perhaps if one day I want to leave this hobby I will take an humbling test. I can even imagine the scene - going to a meeting of the Audiophiles Anonymous and everyone sharing their humbling experiences.

 

[microstrip]

(...) I am sure I read an article by J. Gordon Holt about his own experience and his conclusion was that the music must be very carefully selected to "engineer" abberations-traits-emphasised aspects in the recording (such as sibilance - my thoughts are exactly the same and many go into abx without said preparation and understanding-listening of the audio gear involved and selecting music to engineer selected variables as outlined in previous sentence.
In a way this "engineer abberations" makes the ABX more like the normal AFC in just noticable difference where the person has to select under a very specific factor-question (which has more noise/distortion in the picture/etc); this actually helps with bias selection in A/B (which posted research on quite a long time ago). (...)

From ITU BS.1116-1 "METHODS FOR THE SUBJECTIVE ASSESSMENT OF SMALL IMPAIRMENTS IN AUDIO SYSTEMS INCLUDING MULTICHANNEL SOUND SYSTEMS"

6. Programme material
Only critical material is to be used in order to reveal differences among systems under test. Critical material is that which
stresses the systems under test. There is no universally “suitable” programme material that can be used to assess all
systems under all conditions. Accordingly, critical programme material must be sought explicitly for each system to be
tested in each experiment. The search for good material is usually time-consuming; however, unless truly critical
material is found for each system, experiments will fail to reveal differences among systems and will be inconclusive.
It must be empirically and statistically shown that any failure to find differences among systems is not due to
experimental insensitivity because of poor choices of audio material, or any other weak aspects of the experiment, before
a “null” finding can be accepted as valid. In the extreme case where several or all systems are found to be fully
transparent, then it may be necessary to program special trials with low or medium anchors for the explicit purpose of
examining subject expertise (see Appendix 1).
These anchors must be known, (e.g. from previous research), to be detectable to expert listeners but not to inexpert
listeners. These anchors are introduced as test items to check not only for listener expertise but also for the sensitivity of
all other aspects of the experimental situation.
If these anchors, either embedded unpredictably within the context of apparently transparent items or else in a separate
test, are correctly identified by all listeners in a standard test method (§ 3 of this Annex) by applying the statistical
considerations outlined in Appendix 1, this may be used as evidence that the listener’s expertise was acceptable and that
there were no sensitivity problems in other aspects of the experimental situation. In this case, then, findings of apparent
transparency by these listeners is evidence for “true transparency”, for items or systems where those listeners cannot
differentiate coded from uncoded versions.

 

[Phelonious Ponk]

Tim,

You swear you hear an obvious difference. Then you take a NON valid blind test that shows you are not able to confirm the difference in the blind condition. IMHO the test is humbling only if you accept that the NON valid test proves anything. Should we feel humiliated if we fail a Coca/Pepsi test ?

I know about the dangers of sighted listening. But I have listened to many excellent systems assembled using sighted listening and never heard about an excellent system assembled using only proper blind tests.

Perhaps if one day I want to leave this hobby I will take an humbling test. I can even imagine the scene - going to a meeting of the Audiophiles Anonymous and everyone sharing their humbling experiences.

Sighted listening and non-scientific unsighted listening are exactly the same thing: Casual listening. And neither proves anything. But if you readily accept what you hear with full knowledge of every component and cable in the system (trust your ears), why do you require scientific vigor to accept what you hear without that knowledge? Quite the double standard. And a very weak case for trusting your ears.

Tim

 

[Whatmore]

Tim,

<snip>

Should we feel humiliated if we fail a Coca/Pepsi test ?

<snip>

What do you think we can learn from the Pepsi challenge?

 

[microstrip]

Sighted listening and non-scientific unsighted listening are exactly the same thing: Casual listening. And neither proves anything. But if you readily accept what you hear with full knowledge of every component and cable in the system (trust your ears), why do you require scientific vigor to accept what you hear without that knowledge? Quite the double standard. And a very weak case for trusting your ears.

Tim

Tim,

No, sighted listening is not casual listening if you know how to listen and analyze it. BTW, the question is not that the blind tests are non-scientific, it is that IMHO all the humbling cases I have seen described by WBF members are flawed, either by statistics or method.