Do blind tests really prove small differences don't exist?

garylkoh · May 28, 2011

Actually, Orb, how the amplifier designer achieves the low output impedance has as large a bearing to sound quality as the output impedance itself. If massive amounts of negative feedback is used, you can get theoretically zero output impedance, but then that amplifier may have huge damping, extremely low THD, and sound pretty bad.

There was an amplifier designer who told me that feedback is neither good nor bad - it's just that the wrong amount is often used.

Orb · May 28, 2011

Yeah good point gary, but I was trying to keep it simple

Negative feedback is still controversial as you have respected engineers on both sides of that fence, and yeah your point though is emphasised in one of those links where JA comments on the user setting that allows the defining of negative feedback used, and as you say shows an effect on the FR.
Still, I am not sure it can make it say under 0.1 for the whole of the audio frequency and also the various impedance loading from say 20ohm down to 3ohm, I could be very wrong so if anyone does have figures for one please share.
Ok was good to raise it Gary as it then reminded me to point out that JA shows and mentions it one of those links

This is why the Devialet D-Premier has raised a few eyebrows as its measurements are quite stunning of 0.004 to 0.006ohm output impedance across frequency and loading, might be why many (some do not) like this with a diverse range of speakers.
Edit:
It is the VTL MB-450 link (1st one in post 261).
Thanks
Orb

arnyk · May 28, 2011

mep said:
What school was that?

Oakland University, here in Michigan.

I have never heard of any engineering univerisity that gives out vanilla engineering degrees. How do you market that degree when you leave school when it can't be defined what type of engineer you are?

Never had any problem. Just told my story, sorted out the job offers and took one!

Normally you come out of school as an electrical engineer, mechanical engineer, chemical engineer, or civil engineer.

I see that going on around me.

Those degrees define who you are in the marketplace and the type jobs you will apply for.

The world appears to be tad more accomodating than that. At least that was true in my world. IME the only job where your degree really matters is your first one. After that they hire you based on what you did in your last job.

Ron Party · May 28, 2011

terryj said:
It seems to me that most objections are coming as a result of 'extreme' statements from either side.

(thanks JA for your answer to me earlier), but I still noted at the time in your response of why a particular DBT may have been faulty (something about output impedance of the amp), yet still maintained from that one example (which may have been truly faulty, I don't know) yet concluded therefore that every dbt is faulty.

Exactly. That there may be an exception to the rule does not mean the rule is incorrect. The doubters (not you, Terry) fail to recognize that the very nature of the scientific method requires an analysis of a particular test. If the blind test was faulty for some reason, that does not in any way, shape or form, mean that every blind test is faulty. It also is one of several reasons why even what appears to be a positive finding from a blind test must be repeatable. And, rest assured, if there is a new finding to be made, it will be made with the application of science, not faith which, in the case of cables, Terry perfectly sums up here:

Well, not all, there will be those that refuse dbt's because (to them) their cables have a truly audible difference, and refuse to do ANY sort of rigourous testing on it.

* * *

For me, it is sufficient to say 'in the overwhelming majority of cases when identity is blinded huge differences disappear'.

I try to not take absolute positions, after all there could be someone, somewhere, sometime that can hear interconnect differences (say). None-the-less the essential 'truth' (for me) of what dbt's show remains valid. (Emphasis added.)

Bingo. It is this concept that has seemingly escaped you, Amir. A positive or null finding in a blind test does not *prove* anything. 12 incorrect answers out of 12 trials does not *prove* no difference, likewise 12 correct answers out of 12 trials does not *prove* there is a difference.

As to Arny's statement, Amir, take it up with him. As to those who do not fully understand the meaning of the results of blind testing (which probably includes 90% of those posting at AVS), take it up with them.

* * *

That is the door thru which most dbt deniers will attempt to drive through, feeling they are somehow in that elect group. Yet, if we are honest, we cannot say otherwise.

Pure conjecture here, I think knowing that people will assume they too are capable of hearing these things is behind the overzealous positions of the dbt side...in other words *our* hands are tied by honesty and the *other* side take unfair advantage of that.

I mean because a few can run a hundred metres under ten seconds does not mean most can. In audio equivalent speak most audiophiles can run 8 second times.

This is a restatement of the dangerous statement you reiterate, Amir.

amirm · May 28, 2011

arnyk said:
The school I attended still does not give out differentiated degrees. You can be anything from a software engineer to a mechanical engineer and you get a BSE. My area of concentration was Systems Engineering which requires completing courses for both an EE and a ME plus software engineering. I spent almost 5 years in undergraduate school plus just under 2 years in graduate school.

Is this their web site Arny? http://www.oakland.edu/academics/

If so, this is what they list:

"School of Engineering and Computer Science

Departments
Computer Science and Engineering
Electrical and Computer Engineering
Industrial and Systems Engineering
Mechanical Engineering "

Drilling down into Engineering, it says: http://www.oakland.edu/?id=11167&sid=272

"The Department of Electrical and Computer Engineering offers a Bachelor of Science in Engineering (BSE) degree with separate majors in Electrical Engineering and Computer Engineering. "

So sounds like you didn't take the degree specific courses enough to get that certification? I did a google search and there are lots and lots of people saying they have BSEE from above so clearly they do hand them out.

My primary high end measurement tools are composed of audio interfaces + computers running analytical software, but I also have HP, Fluke, and Heath test gear.

As you recall, we had a healthy discussion relative to the accuracy of your measurements using above versus AP. Suffice it to say, as a professional tool, none of that qualifies to properly measure anything as I showed elsewhere with the large discrepancy between your results and that of AP. For DIY or hobby use though, they are fine to get you order magnitude data.

I was given an AP System One but I don't use it very much because I believe in using the best tool for the job. I typically provide my own test equipment, at least until the client likes what I use and buys his own.

I did a google search and didn't find any mention of you posting anywhere on the net about your AP measurements. All I could find is you saying your Lynx card was as good as AP and in one thread actually better. I suspect you got it recently and haven't even turned it on!

amirm · May 28, 2011

Ron Party said:
As to Arny's statement, Amir, take it up with him.

Why? You are worried about people running off with my argument, how come you don't want to comment if they should with his? After all, isn't that the heart of all of these debates? Either all equipment sounds the same in the manner he said it or not.

Look, we are not children. I can read between the lines. You know what he said is not supported by your point of view or mine. You could have 10000 blind tests and it still doesn't prove all equipment sounds the same. His weasel words of "practical matter" are just that: weasel words. To be fair, the more test we do, the more confidence we build about things we have not tested. But if we want to talk "science" the accuracy is important. And that calls for not declaring the unknown, known.

Seeing how this tiny concession is impossible to get from either one of you when it would not hurt your larger message one bit, speaks volumes to the fact these are NOT discussions about science but a fight to see who can prove the other wrong. The search for knowledge comes second. Arny doesn't want to be caught dead backing off from a position in front of me and you don't want to appear to dispute anyone form your camp.

Does the above apply to me? Of course. But note that I have done away with most of that issue by not taking an extreme position either way. I know if you do, you subject yourself to these dichotomies that are very hard to resolve.

As to those who do not fully understand the meaning of the results of blind testing (which probably includes 90% of those posting at AVS), take it up with them.

I will have to sadly because you won't engage in a deep or specific discussion even though you seem to have such strong feelings about the matter.

This is a restatement of the dangerous statement you reiterate, Amir.

You quote Terry and ask me about what he said? Why don't I do the same. I hope he does not mind

:

terry j said:
well, in that case let me thank you for your contributions. I KNOW I could not have kept my patience as you have, let alone maintained a sense of humour!

It's funny how hard *we* can go to maintain our rightness, and how quickly that line is crossed where we no longer wish to learn (despite our objections to the contrary) where we fight tooth and nail...usually because we know our position is so tenuous that the slightest 'loss' means the whole game is over.

FFS, Amir has sat here page after page and SHOWN how, and under what possible conditions jitter may be audible.

Hey, if it were a cable debate, and we showed with maths and sims that there could not possibly be a difference, well that would have proved it no?

So why the **** in an 'argument' where the shoe is on the other foot does it suddenly become irrelevant what the science says??

My take on what the fear might be is the worry of what might happen if we concede a point of argument. The 'other side' will drive a frickin lorry thru the door if we do.

I mean, there only has to be ONE person who hears a power cord (for sake of illustration) in what seems to be a proper test and the whole frickin lot of the rest of them will claim it as proof that they too can hear it.

No they can't, 'one in a million' means just that. But we KNOW every single one of them thinks they can hear it, using that person as proof, and even less urge to test the truth properly. After all it has been shown.

So, we had better clamp down HARD on the one ever coming out, if only to keep the lid on the rest.

So, move on to something far less controversial than PCs, but as long as it falls into audiofool territory we had better clamp down on that too. It is just safer that way, keep each and every genie in the bottle.

So the need to put amir in his place, and keep the lid hammered on tight. Because the ramifications of this little argument go waaaay past it's tiny borders.

""Oh, but amir has not given any evidence of audibilty"" (apart from the science you mean? The science that would be perfectly acceptable in a different argument, that the one we are talking about???).

Be totally honest here. If he told you that he had found, to his satisfaction, that turning the front panel on and off on his thingamabob had an audible difference, would you accept that?

What then his findings of jitter?

We know you would not accept his results, the genie is too terrifying to contemplate.

So don't come back at me with 'amir has yet to show audibility' ok? It is a definitional thing you know. Some things, by definition, are inaudible.

Bit like cancer, it cannot be cured hence any cure of cancer is untrue (why we are always then exhorted to donate to cancer research is beyond me).

All of you could be right, it may be completely inaudible.

But you sure as hell have not shown it by your arguments. Unless 'nanah nanah nah' counts as an argument.

Orb · May 28, 2011

Orb said:
Ron, whats your view about blind testing and validating the test process and results?
I ask this because would you say it is fair all blind tests should be validated otherwise to some extent it relies upon faith?
I am wondering if validation means doing subtly other test processes, capturing immense level of data that can show cognitive behaviour and heuristic pattern of the participant for perception relatated tests such as the ones we are discussing, a mixture of both, or something else.

This leads onto ABX exclusive tests and how they fit into the above statement; where we can then also come back to the different POVs from Arny and JA on that specific test.
While other blind test process do exist and have been thorough, nearly all relating to subtle differences between products come back to ABX (for me this does not constitute validation though).

Thanks
Orb

Ron any views on this post I asked you about awhile ago?
It would help if you provided some specifics on your views relating to such as the above.
I feel I needed to ask again after you said:

Ron Party said:
....The doubters (not you, Terry) fail to recognize that the very nature of the scientific method requires an analysis of a particular test. If the blind test was faulty for some reason, that does not in any way, shape or form, mean that every blind test is faulty. It also is one of several reasons why even what appears to be a positive finding from a blind test must be repeatable. And, rest assured, if there is a new finding to be made, it will be made with the application of science, not faith which, in the case of cables, Terry perfectly sums up here....

I have not seen anyone here comment about other blind tests being faulty, but as I said earlier the majority of tests relating to do audio equipments sound the same seem to all come back to ABX, so whats your take on it and the fact as you say must repeatable - should this not be more than just an ABX if no-one is capturing the cognitive behaviour and heuristic pattern or proved whether the results are comparable to other matching tests (specifically relating to audio).

Thanks
Orb

mep · May 28, 2011

Ron Party said:
This is a restatement of the dangerous statement you reiterate, Amir.

Ron-I find it comforting that you are keeping a close eye on Amir’s postings as he a dangerous man with dangerous words. We are all at peril when Amir strings dangerous words into dangerous statements. Seriously, I’m surprised at the overall negative tone of this thread. I’m not surprised in the direction that arnyk took things as a few clicks on google gives you an idea of what arnyk is all about. He is as polarizing as Ethan and he always comes with an agenda. Sounds like they (Ethan and Arny) share the same type of test equipment to make all of their claims with. It has been demonstrated several times that both of their test setups are woefully inadequate to perform the type of measurements that an AP is capable of.

The deep passion that some people have for DBTs is just amazing. I read that DBTs can’t prove a difference does exist and that DBTs can’t prove there is no difference. That tells me that DBTs are worthless at best and dangerous in the wrong hands.

Someone should have explored the statement that John Curl made about DBTs. John Curl is a man of science and has designed some legendary gear in his day and he is still designing gear now. John thinks there may be something to the process of the DBTs that actually shuts down the part of your brain that you need to hear differences in gear. Farfetched? Maybe, maybe not.

What I don’t understand is why those who love DBTs and those who don’t continue to argue with each other like there is going to be some type of Dr. Phil breakthrough and one side can “win.” Nobody is going to give an inch on either side of this argument. If you love DBTs and find them meaningful in your world, cool. Have fun with them, just don’t try and cram them down everyone else’s throat under the name of science. Don’t like DBTs? Cool. Keep on trucking brother-it’s all good.

I vote to close this thread and put this ugly mess to bed-dangerous statements and all.

Mark

Ron Party · May 28, 2011

Orb, I'll try to get back to you later (or more likely tomorrow). I'm at work now and all afternoon, then headed to a favorite sushi restaurant where they serve unlimited sake, then to the A's game. I for one appreciate the far more meaningful nature of dialogue in which you wish to engage, and it requires far more thought than what should be an extremely easy concept to grasp regarding the correct interpretation of the results of a blind test.

amirm said:
Why? You are worried about people running off with my argument, how come you don't want to comment if they should with his? After all, isn't that the heart of all of these debates? Either all equipment sounds the same in the manner he said it or not.

Take Arny's statement up with him. I never said all equipment sounds the same.

Look, we are not children. I can read between the lines. You know what he said is not supported by your point of view or mine. You could have 10000 blind tests and it still doesn't prove all equipment sounds the same. His weasel words of "practical matter" are just that: weasel words. To be fair, the more test we do, the more confidence we build about things we have not tested. But if we want to talk "science" the accuracy is important. And that calls for not declaring the unknown, known.

Exactly. We cannot declare the unknown, known. That is part of the reason why a blind test does not *prove* anything. I've stated this at least half a dozen times in this thread. I will not state it again.

Your 10,000 blind tests assertion is exactly correct. For the exact same reason, my previous statement about 12 incorrect or correct answers out of 12 trials is exactly correct.

But how you can at the same time state that "the more test we do [sic], the more confidence we build about things we have not tested" and then accuse someone about *weaseling* is baffling.

Look, you are exactly correct about interpreting the results of *testing* as confidence building. That includes the results of blind testing. Blind testing results are nothing more than probabilities. Everyone who truly understands blind testing understands this. The results are not *proof*.

Seeing how this tiny concession is impossible to get from either one of you when it would not hurt your larger message one bit, speaks volumes to the fact these are NOT discussions about science but a fight to see who can prove the other wrong. The search for knowledge comes second. Arny doesn't want to be caught dead backing off from a position in front of me and you don't want to appear to dispute anyone form your camp.

Your mistake is that you've lumped me in with someone's camp. You did this, not me. That may work in politics but it doesn't work here. Since you've failed to understand the proper meaning of the results of blind testing (at least so far), you are in no position to characterize my position whatsoever. Tiny concession? What statement is it that you're trying to exact from me this tiny concession? Is it that all equipment sounds the same? Well I never said that.

Honestly, I'm done here. I'm just repeating that which I've stated in this thread many times and it serves no further purpose. And as a moderator here I have to lead by example, otherwise I will be accused down the road of hypocrisy when I intervene in a tiresome thread. Either you grasp it or you don't. I strongly suspect many following this interchange between you and I do.

Ron Party · May 28, 2011

Mark, address the post, not the poster.

mep · May 28, 2011

You mean I can't do both like you have doing with Amir?? Are there two sets of standards that I don't know about Ron?

Ron Party · May 28, 2011

Pause.

amirm · May 28, 2011

Ron Party said:
But how you can at the same time state that "the more test we do [sic], the more confidence we build about things we have not tested" and then accuse someone about *weaseling* is baffling.

It is not baffling if you look at the sample size. In the entire thread that went on for a month on the topic of DAC fidelity with Arny and crew present, not one published double-blind test of DACs was cited. Not one! A sample size of zero tells you nothing. Yet Arny said what I quoted from him. Now, if you had tested 50 DACs, then we could talk. Without it, it is all assumptions, not confidence building.

On the topic of jitter audibility there is maybe three or four papers. That is it. Again, nothing to build up high confidence on given the vast landscape jitter covers.

Ask about high resolution formats vs CD, and you get one test. Not five, not ten. One. This time there was an army five times bigger including Ethan and everyone else known to man on the objectivity front. But again, only one report was cited. I was the one who cited the second one but of course, everyone cursed that one because four people did hear something different.

On my side, I have done blind tests of high resolution formats and found them to be different sounding. I have blind tested DACs and found them to sound different. So please forgive me for not taking a sampling of one or two from others as counter to may own experience. That is not enough to give me a high confidence counter to the data I personally have.

Steve Williams · May 29, 2011

seems like there is not much being discussed that is productive so I am going to close the thread to minimize any further vitriole here

Thanks everyone

Search

Search

Do blind tests really prove small differences don't exist?

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)

Orb

New Member

arnyk

New Member

Ron Party

WBF Founding Member

amirm

Banned

amirm

Banned

Orb

New Member

mep

Member Sponsor & WBF Founding Member

Ron Party

WBF Founding Member

Ron Party

WBF Founding Member

mep

Member Sponsor & WBF Founding Member

Ron Party

WBF Founding Member

amirm

Banned

Steve Williams

Site Founder, Site Owner, Administrator

Similar threads

Members online