Double Blind tests *did* show amplifiers to sound different

[amirm]

In many online debates, position taken by some that when amplifiers are used that have flat frequency response and low distortion that no double blind tests have shown them to sound different.

Well, I managed to dig up a 32 year old test that says otherwise. What is fascinating is that one of the article/testers was no other than Arny Krueger, the most vocal champion of such arguments in forums!

The story is that it occurred to me one day that I had never, ever seen Arny document any blind tests he had run. Yet he said he had run 100,000 of these (no joke). After pressing him for an example, he said there was only one test that was ever documented(!) and it was in this 1982 Amateur Audio magazine. While that is pre-Internet era, one would think that someone, some place would have scanned a copy and put online. This has been done for other articles that said no difference was found but this article, was never, ever put on the net for anyone to read. Arny to his credit had made references to it here and elsewhere but never discussing the details of it.

My challenge then begun in trying to locate a copy of that magazine issue or another that was printed in UK. I came awfully close and missed it by one month on the UK publication. But came up empty other than that despite searching everywhere. I mentioned this to Arny and to my surprise he linked me to an auction site that I had never heard of with someone having that issue. With reluctance I signed up to their service and after paying my $11, the magazine arrived.

The article is actually quite short. What I am about to show you is about half of it and contains all pertinent portions. I am happy to field and answer questions that may be in the rest of the article.

So let's start at the top. Here is the title of the 2-page article:

What a headline, no? It says they have "proven" that amplifiers sound different! You almost don't need to read any more but for completeness let's go into it.

Here are the amplifiers in question:

So these are very powerful amplifiers. We are not talking a tube SET amplifier or some other amp that runs out of juice at 20 watts. We have very powerful amplifiers clocking at over 200 watts at 4 ohms.

This being the pre-CD era, a turntable was used. Here are the details of that and the speakers used:

So how did they do? Here are the votes for the three tracks tested:

We see why they say "prove" as the two tracks show statistically that the results are not due to chance.

So the conclusion is as expected:

Not the message we see broadcast on forums.

So what caused the difference? I think it is what I believe to be behind such audibility differences namely what happens at clipping point which occurs far more often than we may think:

No data is presented on what would happen if the volume was turned down.

So there you have it. "Proof" that amplifiers do sound different in double blind tests .

 

[MylesBAstor]

Yes but Amir their rebuttal would be that amplifiers sound the same when driven within their operating parameters. And the old Accoustats weren't an easy load. (But few are!)

 

[amirm]

And that has been their argument. But they have no answer for why they didn't present this test for 32 years. And that even 200 watts was not enough in that situation.

 

[Orb]

Amir,
you are the modern day Sherlock Homes
Well we know that nearly no speaker is actually flat impedance or necessarily always benign phase and if wanting a flat FR the amp must have output impedance of under 0.1ohms to be under the audible difference of 0.2db; the level matched argument of below 0.2db across FFR also comes from a few vocal abx testers.
I think Electrostatic would be a nightmare tbh.
Out of interest here is the Devialet with output impedance of under 0.05ohms and an example of one with 0.27ohms.

Devialet: http://www.stereophile.com/content/devialet-d-premier-da-integrated-amplifier-measurements


Ayre AX-5: http://www.stereophile.com/content/ayre-acoustics-ax-5-integrated-amplifier-measurements


Ignore the green as that is 2ohms, black is realistic simulated speaker load including at points 4ohms.
So a good modern amp with a reasonable 0.2ish ohms will have some interesting FR with a speaker that goes down to 4ohms.

Here is the chart showing the simulator speaker load charactistic:


Just an example of how there would be one measurable difference IMO; I tend to think how output impedance was measured in past compared to modern testing is different as one can see how a manufacturer in the distant past had very low output impedance but a more modern model actually has a notable measurement.
So that means one cannot rely upon the stated output impedance of historical products, in same way one really cannot rely upon spec of speakers back then as well (appreciate they would had tested speaker impedance across FR for test).

But strangely enough none of this type of variable-factors ever comes out or considered on ABX testing and when cheap amps sound the same *shrug*, should do from an objective standpoint.
Thanks Amir as it is interesting to see the approach-framework on how this all started.
Cheers
Orb

 

[microstrip]

Amir,

Unfortunately I have offered my vintage Hi-Fi News & Record Review collection to a good friend long ago, keeping only the few issues I considered of historical interest and do not have this article anymore. However I would like to understand what means exactly "clipping occurred 1% of the time". Why not tapping the amplifiers violently with an hammer 1% of the time?

 

[Whatmore]

surely if they were clipping then they don't have low distortion

 

[FrantzM]

Hi

I am not surprised by this finding. I try not to (but often do not succeed :b) cry "DOUBLE BLIND!!" every time I read some extraordinary claim by some of my fellow audiophiles. I have a few years ago subjected myself to some (nonscientific) double blinds and must say that these were very humbling experience , enough to elicit the type of denial people revert to when facts are threatening to blow their worldview to smithereens. Same unscientific tests showed me that amps (electronics in general) did sound very different and that in a variety of speakers in a a variety of rooms.
Thanks Amir! strange that this wasn't unearthed before ... See! Subjectivists you don't need to reject Double Blind tests (and Science) en masse, they can prove (some ) of your points too .

 

[microstrip]

I fail to understand how a non scientific test can be an humbling experience. IMHO those making incredible claims are as guilty as those using childish challenges to win bets ...

 

[FrantzM]

so microstrip .. nonscientific is childish... . They were however good enough for my purposes. YMMV.

 

[DonH50]

I posted simulated amplifier responses into simulated speaker loads showing differences some time ago (http://www.whatsbestforum.com/showt...ponses-into-Speaker-Loads&p=139289#post139289). Simplistic but shows there can be differences. I am not sure anyone credible has made the flat statement that all amplifiers sound alike, that is ludicrous. What is true, and has been shown by other studies I am sure Amir can dig up, is that for many if not most "average" audiophiles (which does not include most of those on WBF based on my read) using average amplifiers (in their AVR or out) and average speakers at reasonable levels there is no discernible difference. That is a whole lot of wishy-washy words to pick apart so have fun. I do not see any value (and a whole lot of grief) in this thread; sure to be another cat-fight.

 

[Orb]

I posted simulated amplifier responses into simulated speaker loads showing differences some time ago (http://www.whatsbestforum.com/showt...ponses-into-Speaker-Loads&p=139289#post139289). Simplistic but shows there can be differences. I am not sure anyone credible has made the flat statement that all amplifiers sound alike, that is ludicrous. What is true, and has been shown by other studies I am sure Amir can dig up, is that for many if not most "average" audiophiles (which does not include most of those on WBF based on my read) using average amplifiers (in their AVR or out) and average speakers at reasonable levels there is no discernible difference. That is a whole lot of wishy-washy words to pick apart so have fun. I do not see any value (and a whole lot of grief) in this thread; sure to be another cat-fight.

Might be of interest here is the article by Keith Howard (linked it in the past and worth reading like the additional link further down on device dissipation/EPDR) on the approach for speaker simulation: http://www.stereophile.com/reference/60/index.html
I appreciate what you are saying Don, but I am going by the various non-engineer audio-hifi forums that I lurk at now and again and from my experience most never consider the variable-factor I raised (well apart from you hehe, and a few others) and readers at those sites do end up with an over-reaching conclusion.
Many budget amps will fall into causing at least a 0.2db variation in the audio band even when one level matches at say 1khz, that is the crux of my point.
Furthermore many of those supporting budget well designed amps cannot be distinguished in ABX used to be directed to a site where they did ABX with various products, Arny and others have in the past directed readers to that site where they have fun presenting how budget equipment sounds the same as high end...
And aside this issue, there is also the device dissipation/equivalent peak dissipation resistance involving loudspeakers I indirectly raise now and again that occurs outside the traditional idea of amp clipping and loudness;
http://www.stereophile.com/content/heavy-load-how-loudspeakers-torture-amplifiers-page-2

Anyway this does not invalidate their or other ABX'ers argument, just that I think they are not always asking the right questions as part of their conclusion; such as a simple example is the FR notch-dip associated with the crossover audible for speakers where it goes over 0.5db with music and if not why (never seen anyone do a perception test for this and with music) and can this also include amp-speaker ABX setup that do have difference as I mentioned.
But I do feel the abx conclusions are usually too generalised or do not consider critical engineering variables such as the charts show *shrug*; btw the site where readers were directed to also never considered such parameters (it used to be referenced often from places such as AVForum/AVSForum/couple of others).
Unfortunately no I do not have the site nor engaged in its details as I always just lurked out of curiosity for what was being said on various other forums including the ABX one I visited a few times.

I could go on about weighted statistical significance for just noticable difference-weighting-bias selection control-AFC methodology as I have in the distant past on here that also included a fair amount of research papers, but that is even more of a potential grief debate than even this one
Got my virtual barge pole and now pushing this thread well away from me
Cheers
Orb

 

[microstrip]

so microstrip .. nonscientific is childish... . They were however good enough for my purposes. YMMV.

Frantz,

You should read posts with more care - I have never said such thing. But yes, MANY of the audio listening tests are childish challenges IMHO. YMMV.

 

[microstrip]

(...)
Thanks Amir as it is interesting to see the approach-framework on how this all started.
Cheers
Orb

Orb,

It was great to see this article, mostly because it brings a positive identification of differences. The inclusion of control tests giving positives in any listening test is a must and is systematically ignored.

As far as I remember there were other previous listening tests on amplifiers differences, but resulting in negative identification. One of the most famous is reported in Hi-Fi News & Record Review June 1978 - the famous Quad challenge. Curiously the same issue includes an interesting article by Stan Curtis on Amplifier Noise and Clipping. These were the days of the "Great Amplifier Debate"-

 

[Orb]

Micro,
you dig that out from a collection?
Yeah totally agree, and nice memory-find!
Cheers
Orb

 

[amirm]

Amir,

Unfortunately I have offered my vintage Hi-Fi News & Record Review collection to a good friend long ago, keeping only the few issues I considered of historical interest and do not have this article anymore. However I would like to understand what means exactly "clipping occurred 1% of the time". Why not tapping the amplifiers violently with an hammer 1% of the time?

. They don't say much more on that than this:

I don't know how when playing music on an analog scope they managed to detect 1% distortion. I know how to instrument the amp internally but even then, it would be hard to come up with a percentage as they did.

 

[amirm]

A couple of points:

1. The test shows the importance of selecting the right clips that reveal the problem at hand. Two of the clips were highly revealing but the other was not at all. In this situation the selection was likely made at random. So it was a luck of the draw as the saying goes that two clips were revealing. They could have picked two other and arrived at the conclusion of no difference.

This is a problem. When we go through the trouble of setting up a test like this, we need to look at the hypothesis at hand. That hypothesis should not just be a layman one of "let's see if we can make this point or that." Instead one should think through what "1% clipping" could do and find content that makes it revealing. Without a hypothesis, you wind up using random files and getting potentially random results.

2. I think the protection circuits in amplifiers need to be examined more. They are complex beasts, performing dynamic current limiting for milliseconds at a time. The nature of negative feedback system is that it will fight this mechanism. The current limiter puts the brakes on the output stage of the amplifier. That creates distortion which the negative feedback sends to the input of the amp. That circuit then tries to force the output stage to do what it is demanding, i.e. push it harder. That in turn causes the protection circuit to current limit more.

As if that were not enough, majority of amplifiers use unregulated power supplies for the output stage of the amplifier. That voltage sags under load creating yet another indirect feedback loop. Now we have three feedback loops none of which is considering the other in the system. Such behavior simply cannot be simulated or predicted. Testing is also very hard since as noted, speaker loads are different and cause the current limiting to act differently. And of course that factor is also content dependent.

In the case of severe limiting, none of this matters. But when limiting is undetectable as such, then there can be coloration.

 

[Whatmore]

I'm not sure what's surprising here. They drove a difficult load with amps that were not up to the task (they clipped). Unless we expect amps to clip identically then of course they would sound different.

 

[amirm]

I'm not sure what's surprising here. They drove a difficult load with amps that were not up to the task (they clipped). Unless we expect amps to clip identically then of course they would sound different.

They tested the amps and they produced over 200 watts/channel each. The argument has been that just about any amount of power is sufficient for everyone.

 

[Whatmore]

They tested the amps and they produced over 200 watts/channel each. The argument has been that just about any amount of power is sufficient for everyone.

The amps clipped, ergo they were not sufficiently powerful

 

[dingus]

whats the big deal? is there any significance to different amps sounding different?