Do blind tests really prove small differences don't exist?

[arnyk]

Yes, I believe they exist. And one of the reasons I am interested in WBF debates is learning about what causes them. I already know about the expectation bias, but I am also trying to learn abouth the other possibilities.

BTW, considering the DBTs, could you avoid nominating tons of them and referring one or two precise ones you consider of high relevance?

Why not do your own listening tests of these files, like hundreds of others already have:

http://www.ethanwiner.com/aes/

More specifically the ones related to "soundblaster generations".

 

[amirm]

That is me talking over at AVS, and they are basically the same statement twice. How can that possibly be confusing?

You can see why in my post. You said blind tests can never prove anything. Yet you said with absolute certainty that the results of a handful of blind tests apply to all people, all content and all equipment with *absolute certainty*. You could have said "it is highly likely based on what we know that such and such is true." But that is not remotely what you said:

"Once audio performance reaches a certain level, there can still be differences but nobody anywhere will ever hear the differences."

"Nobody will ever hear the difference" attempts to say the results of our few tests do apply to all other cases outside of those tested. Or else, it is a false statement. You can't have it both ways and hence my confusion as to what position you hold now. Actually, let me be blunt and say I am not confused . I firmly believe that you do think that you can take the results of these few tests and conclude that it applies to the entire universe. No other conclusion about your remarks is rational otherwise.

There are tons of DBTs that support those statements, but so does the rest of science.

"Support?" What blind test methodology extends the results of a blind test to all situations with absolute certainty as you said above?

Unless you believe that what is known about thresholds of hearing and masking is completely bogus, and that audible differences among good electronic equipment still abound, then the above should be like motherhood and apple pie. It's completely believable and not confusing at all!

Studies about what we can perceive can indeed be very helpful in how we design and evaluate equipment. So nothing about what I am saying implies what we know there and throw it away. I have used many of those principals to identify what differences matter more and what differences matter less.

Those studies are exceptionally helpful in assessing what the general public can or cannot hear or cares about.

As you know, the above data is also gathered using blind testing methodology. In that sense then, you have no more data point to add to your argument that results of blind tests apply with 100% confidence to all situations.

Tell me that all the "blind" tests in the world can tell us that this gift of hearing can be identified in the general public:

Now, I don't want to over-dramatize and say that audiophiles have such hearing. But hopefully we can learn to not over-generalize results of general population studies and say they apply to all people in all situations. Or else, that gifted young man wouldn't be able to "see" and identify a trash can by mere echoes of sound.

BTW, there are aspects of that hearing ability we use all the time. It is what tells us that a sound changes as it gets closer to a barrier such as a wall.

What does "really proven" mean as you ask it?

Philosophically, absence of proof, which is all that we have, is not the proof of absence.

It means whether we can say something with 100% certainty. What I quoted from you was precisely that. It left no ambiguity whatsoever. Therefore, you are still speaking from both halves of your mouth. You say in one breath that we can't prove a negative but in the other, you say we have proven inaudibility as quoted above. You can't have it both ways.

OTOH, it is very common to say as a practical matter that this or that is proven.

It is my turn to ask: what does "practical matter" mean? That the general public is not sensitive to it? If so, then we are in agreement. But you don't stop there. Your statement above was in reply to me in the other forum. I consider myself more sensitive to audio artifacts than vast number of general public as a matter of training and blind testing results. Is your statement supposed to apply to me too? If so, how is that as a matter of practicality?

For example, expert studies of thermodynamics and solid state physics prove that LED light bulbs will never exceed 40% efficiency. But is it really proven? Might there not be some quantum leap type finding about either thermodynamics or solid state physics down the road that changes everything?

That has nothing to do with the topic at hand. That is a question of current state of LEDs. There was a time when people thought they couldn't create white LEDs but here we are. So saying you agree that kind of thing is not proven is neither here, nor there.

What we are discussing here is whether we can take limited data and build up 100% confidence. That kind of statement can be judged based on data we have now and the definition of what a blind test means.

Of course everything can change in a heartbeat. But how much will you bet on that happening tommow, next week or in your lifetime?

How much will you bet that it can *never* happen? Agree that you are speaking of probabilities that someone can hear differences and not absolutes and we will be miles better than where we are now. Don't be the guy who predicted that white LEDs can't be manufactured. Or that submicron challenges would keep us from making semiconductors in those geometries as countless people did before every barrier was removed time and time again giving us our powerful yet power efficient tablets and phones.

 

[arnyk]

You can see why in my post. You said blind tests can never prove anything.

I never said any such thing Amir.

Please respond again when you can recreate some rough approximation of what I actually said.

 

[amirm]

I mever said any such thing Amir.

This is what you said:

The question "Do blind tests really prove small differences don't exist?" always has the answer "No" because proof that something does not exist is difficult or impossible.

You said the answer is "always no." Always no means they can never prove anything.

Please respond again when you can recreate some rough approximation of what I actually said.

I am quoting you word for word above.

 

[arnyk]

This is what you said:


You said the answer is "always no." Always no means they can never prove anything.


I am quoting you word for word above.

This time, kinda sorta in context. Big improvement.

I was speaking philosophically, as I have explained many times.

Is this a philosophical discussion or a real-world discussion?

Please remember that philosophically, we can't prove that 9-horned, 9-toed flying purple people eaters don't exist. ;-)

 

[garylkoh]

 

[Ron Party]

Amir, my friend, you continue to play it close to the vest. I cannot believe your claim you do not understand that blind tests don't prove anything. I cannot believe you've repeated the dangerous statement which opens the door to all of the highly suspicious claims of audibility. Last time you wrote I took your post out of context. I disagree. Now you've just repeated the same post in a different context. My posts ## 138 & 142 in this thread apply to this most recent interchange.

We cannot *prove* that sprinkling some magic pixie dust on a person's head never will cure that person from cancer. Likewise we cannot prove that placing a DAC upside down never will cause an improvement in, e.g., soundstage. Do we infer from that absence of cause and effect proof that the cause indeed can have the intended effect?

You either believe in the scientific method, i.e., that the perceptual world can be studied by humans, or you don't, i.e., you believe in faith. But please, pretty please, with magic pixie dust on top, please stop im- (if not ex-) plicitly stating that bias can go on holiday. Humans are absolutely expert at self-delusion. Extraordinary claims require extraordinary proof.

 

[arnyk]

Interesting - I get the words to that old song wrong, and someone still beat me to it! ;-)

 

[RUR]

Pffft! Any student of NFL history knows there are Purple People Eaters.

Those guys prolly thought a DBT was a Double Blind-Tackle.

 

[microstrip]

Why not do your own listening tests of these files, like hundreds of others already have:

http://www.ethanwiner.com/aes/

More specifically the ones related to "soundblaster generations".

Well, we have done it in WBF with some files supplied in Ethan site having variable amplitude low level signals buried in noise that we were supposed not to hear. Several of us reported we could listen to them. As far as I remember, as the signals were always buried in the same time position, someone wisely claimed that may be we were just expecting them ...

Anyway I am not interested in doing tests, just asked for othesr tests with results that prove (or disapprove) anything in this discussion. My main question was " BTW, considering the DBTs, could you avoid nominating tons of them and referring one or two precise ones you consider of high relevance? ".

 

[arnyk]

Anyway I am not interested in doing tests, just asked for othesr tests with results that prove (or disapprove) anything in this discussion. My main question was " BTW, considering the DBTs, could you avoid nominating tons of them and referring one or two precise ones you consider of high relevance? ".

The nominee is before you. One pass through Ethan's SoundBlaster measures differently than two passes, and that measures different than three passes, and so on. If memory serves, nobody hears any differences until the number of passes is > 5.

Another nominee. My LynxTWO is a much better measuring product than Ethan's SoundBlaster. Again, if memory serves it took more than 10 passes before anybody heard anything. Again, each pass through the LynxTWO imprinted a different set of distortions on the audio signal.

Another nominee - the CD-standard A/D/A loop that was used in the Meyers and Moran High Resolution listening test (of JAES fame) was compared to a piece of wire. Obviously, the two measured differently, but were indistinguishable to a large number of listeners. The CD-standard A/D/A loop was composed of components of HHB CDR-850 professional CD recorder.

 

[amirm]

Amir, my friend, you continue to play it close to the vest. I cannot believe your claim you do not understand that blind tests don't prove anything.

You are taking my position and asking me if I believe in it??? It is very difficult to follow your arguments in this thread Ron. I almost don’t want to answer them for that reason. So here it is in my words: what I am asking is whether *your camp* believes in it. Both you and Arny use the above words but then go on to say the opposite as I quoted verbatim for him. Here it is again:

"Modern gear can be so close to being perfect that two devices with excellent but vastly different technical performance will still sound the same.
[...]
The scientific facts tell a completely different story than the post I'm replying to. Once audio performance reaches a certain level, there can still be differences but nobody anywhere will ever hear the differences."

If you are in agreement that blind tests don't prove lack of audibility in all equipment, how are the above statements true? Clearly they were not based on measurements or design – both of which Arny presupposes as being different. What other area of science then gave those conclusive data in his post?

I cannot believe you've repeated the dangerous statement which opens the door to all of the highly suspicious claims of audibility.

“Dangerous?” I am supposed to tiptoe around a topic because you are worried about someone else running with them? How about the countless people who run wild with conclusions from your camp leaders? Recall the jitter thread with Ethan where I showed on a case by case basis how there were no jitter audibility data in AES that proved them to be inaudible? Why don’t you think those reports were dangerous? I have had to counter them in more threads than I can count with people who don’t even know what the word jitter means, but read the conclusion in that paper that it was not audible at certain amounts. The mark of AES and “bind test” was good enough for them. Damn the details of what was tested and how.

We cannot *prove* that sprinkling some magic pixie dust on a person's head never will cure that person from cancer. Likewise we cannot prove that placing a DAC upside down never will cause an improvement in, e.g., soundstage. Do we infer from that absence of cause and effect proof that the cause indeed can have the intended effect?

How many times do I have to correct you Ron on the position I have taken on this thread? I am specifically talking about cases where we can with *100% scientific certainty show measurement differences.* Upside down DACs and pixie dust may not apply unless they do show a measured difference in which case, I would love for you to explain that away .

You either believe in the scientific method, i.e., that the perceptual world can be studied by humans, or you don't, i.e., you believe in faith. But please, pretty please, with magic pixie dust on top, please stop im- (if not ex-) plicitly stating that bias can go on holiday. Humans are absolutely expert at self-delusion. Extraordinary claims require extraordinary proof.

“Scientific *method*” translates into a tool. As with any tool, you better know its limitations or else, you get yourself in trouble and deep. Take video measurements. You can buy a spectrometer for $70, or get the Minolta unit I have for $20,000. Both follow “scientific method” of measuring colors on your display. If you knew nothing different, you would likely buy the $70 unit thinking all is well. Then you go and talk to an expert and he tells you that as the luminance level (brightness) becomes lower, the cheap meter gets swamped with noise and loses all accuracy. And unfortunately for our pocketbooks, those levels are 100% visible to the eye. Without the more accurate meter, you would have a color shift in your monitor as the video became darker. Not a good thing if you care about fidelity and correctness.

I personally use both of above solutions. I use the cheap unit to quickly calibrate my computer monitors around the house. Its software is fully automated and anyone in two minutes can have a far more accurate display. Would I use that on our $60,000 projector at work? Heck no. I use the above Minolta meter because people are sitting in the dark watching a 17 foot image with a high price tag and expect it to have the right tones.

By the same token, I have lived through so many blind tests professionally that I have formed an opinion of where they are solid as a rock and where they might produce faulty data. Here is an example I have given elsewhere. We once paid an independent and major testing company to evaluate the quality of our audio codec at 64 Kbps against the CD. To my pleasant surprise, the people conducting the test said they wanted to follow the ITU measurement standards to the letter and that they wanted to use reasonably high quality headphones and such to make sure users were not fooled one way or the other. We said fine.

After $25,000 and 2-3 weeks with > 100 people recruited for the test, do you know what they found? That 90%+ of the people could not tell the 64kbps version from the original. Let me repeat. They followed textbook methodology of double-blind testing as standards by International Telecommunications Union and said that across a pretty large population, folks could not detect that we had thrown out 96% of the signal. So our marketing folks broke out the Champaign and declared to everyone how good our technology was.

Of course, the truth was nowhere near that. The testing company couldn’t see the forest from the trees. It didn’t understand how compressed music worked and what content would be difficult and what would be easy. They thought for example the classical music was hard because audiophiles use them as reference tracks. Well, classical music is harmonic and hence, can be easier represented than something that suddenly changes like a transient. Had they picked MPEG reference clips for example, I doubt that 30% of those people would say the clips sounded the same as the CD. So clearly there is more to the “scientific method” than the fact that it is double blind and follows textbook proceedures.

Another example that might resonate with you and others better. Barry Scheck was the lead defense expert for OJ Simpson. His work prior to that trial was with the Innocence Project where he would try to get wrongly convicted prisoners freed by using DNA testing. So I found it quite hypocritical when he was called to testify that DNA testing was too unreliable as a scientific method to convict OJ Simpson. Later though, I saw an interview of him where he said that DNA tests can conclusively rule out someone being connected with the scene of a crime. But the reverse, is a game of probability and not absolute. And that even if the probabilities are high, we still cannot know for sure if someone’s DNA was left at the scene of the crime. He followed by saying that when we are talking about putting someone in prison for life, we better be sure.

So here is a “scientific method” that seems to be valid in one direction but not the other. To the layman like me, that distinction was not apparent and hence, I thought the guy who would say anything for money. I was wrong. Or at least I think I was. Someone with more expertise would have a more informed opinion than I. Fact is that if you are not schooled in the science, you may not know all that there is to know to understand whether it is being applied correctly or not.

This week I spent two wonderful days at Harman, including hours of presentation and data on blind testing. Yet on the simple question of how you would treat the side wall by a speaker, there were 2.5 to 3 answers from three world-renowned experts in speakers and acoustics! Are there three versions of the truth or are two of them wrong? The logical conclusion is that the scientific method is not complete enough to lead us to a single answer. We have not researched the topic well enough, or the topic itself is too challenging to yield itself to amount of research we have performed to give us one answer across the full range of audio topics.

On a personal level I have to say, your position seems to be that if I disagree with you, I must be delusional. I find that illogical to put it politely. If I tried to give you lessons on the meaning of the US constitution or what contract law means, you would probably laugh at me. I have practically lived with attorneys for the last 10 years so much so that my attorney lets me argue with the opposing one without him being present half the time. Yet, there is not an instant that I would try to teach him how the law works. How come you feel so confident in being right and me being wrong in matters related to audio here? Why isn’t there door ever so ajar that I might, just might, know something you don’t?

I realize folks want one-liners that come out of fortune cookies such as Arny used and you mention above. I appreciate how simple that can make our lives. But please don’t look to me to provide them. I am bewildered by some audio topics, even though my signature says I like to avoid it as best as I can . The nature of the science is quite complex at times.

 

[arnyk]

"Modern gear can be so close to being perfect that two devices with excellent but vastly different technical performance will still sound the same.
[...]
The scientific facts tell a completely different story than the post I'm replying to. Once audio performance reaches a certain level, there can still be differences but nobody anywhere will ever hear the differences."

If you are in agreement that blind tests don't prove lack of audibility in all equipment, how are the above statements true?

Here we see Amir for the who-knows how manyth time demand absolute proof that nobody will ever hear a difference, when he constantly overlooks the fact that absolute proof is generally lacking in this world?

If Amir demands absolute certainty, does he offer his customer's absolute certainty?

I'd like Amir to offer double the money back for every component he sells on the grounds that it sounds better, if the purchaser fails to hear the purported difference in a DBT. Thing is, that level of proof pales in comparison with the absolute proof that Amir demands of others.

I can say that there is less than X% chance that a listener will hear a difference, and X can be so small that nobody would ever bet even $1 on it. But absolute proof is far tougher than that.

 

[amirm]

Here we see Amir for the who-knows how manyth time demand absolute proof that nobody will ever hear a difference, when he constantly overlooks the fact that absolute proof is generally lacking in this world?

Not true at all. If I can show something mathematically to be true, it is absolute. For example, 16 bit audio will have maximum signal to noise ratio of 98 db. If you claimed that it is 80, I can absolutely prove you wrong. Ditto if you claimed it is 120db.

I am telling you that you cannot with certainty claim what you did.

Then don't make absolute claims. You say such things don't generally exist in this world but you are so comfortable minting one every day of the week on any audio topic.

If Amir demands absolute certainty, does he offer his customer's absolute certainty?

I'd like Amir to offer double the money back for every component he sells on the grounds that it sounds better, if the purchaser fails to hear the purported difference in a DBT. Thing is, that level of proof pales in comparison with the absolute proof that Amir demands of others.

We do one better. We let people take high-end equipment home for free and test them however they like. Blind or otherwise. If they like it, they can buy it. If they don't, they can return it. We even deliver them and pick the up as we just did on our Revel Salon 2s which are a pain to carry without the case.

As to paying them on top of that to the tune of 2X the purchase price, how about you paying me for the blind tests I have run with DACs that show a difference? If my judgement is sufficient enough for the validity of the test, how about coughing that money right now? I take paypal .

I can say that there is less than X% chance that a listener will hear a difference, and X can be so small that nobody would ever bet even $1 on it. But absolute proof is far tougher than that.

Then say it that way. That is worlds better than what you said. You might think the difference is small but leaving the possibility, however small, is very different than saying there is none.

When asked about small differences, I usually say that 99% if not more percentage of the population cannot hear the differences. Is 1% the right number? I don't know. I do know that I can't prove it is zero. If you can't prove either, just say it and we are done.

 

[Ron Party]

Amir, your post is so long I don't know where to begin. There is so much deflection that I need the debate equivalent of proper room treatment to sift through it. I'm sorry, but the noise to signal ratio is just too high. I already read your spectrometer anecdote in the AVS thread. It adds absolutely nothing to the discussion. Neither does your OJ nor any of the other anecdotes.

If you do not understand or simply refuse to admit that one cannot prove a negative, then there is no purpose for any further discussion.

You are hanging your hat on the math in your incomplete hypothetical. You did that same thing in the AVS thread and A.J. (while a complete jerk) repeatedly asked you whether the jitter was audible. The math is the math. As I posted before, the bottom line question is and always is: is it audible? This is precisely the question where the two camps converge.

So how do we go about answering that question? Using the scientific method or using faith?

 

[Orb]

The nominee is before you. One pass through Ethan's SoundBlaster measures differently than two passes, and that measures different than three passes, and so on. If memory serves, nobody hears any differences until the number of passes is > 5.

Another nominee. My LynxTWO is a much better measuring product than Ethan's SoundBlaster. Again, if memory serves it took more than 10 passes before anybody heard anything. Again, each pass through the LynxTWO imprinted a different set of distortions on the audio signal.

Another nominee - the CD-standard A/D/A loop that was used in the Meyers and Moran High Resolution listening test (of JAES fame) was compared to a piece of wire. Obviously, the two measured differently, but were indistinguishable to a large number of listeners. The CD-standard A/D/A loop was composed of components of HHB CDR-850 professional CD recorder.

Are all of these purely ABX or other type of blind test process?

Thanks
Orb

 

[terryj]

It seems to me that most objections are coming as a result of 'extreme' statements from either side.

(thanks JA for your answer to me earlier), but I still noted at the time in your response of why a particular DBT may have been faulty (something about output impedance of the amp), yet still maintained from that one example (which may have been truly faulty, I don't know) yet concluded therefore that every dbt is faulty.

that is as extreme as the current argument about arny's statements.

Whilst not exactly addressing the title of the thread, I now wonder what the formulation regarding DBT's might be that we can ALL agree on?

Well, not all, there will be those that refuse dbt's because (to them) their cables have a truly audible difference, and refuse to do ANY sort of rigourous testing on it.

Leaving them aside (no insult intended) what statement can we agree on?

For me, it is sufficient to say 'in the overwhelming majority of cases when identity is blinded huge differences disappear'.

I try to not take absolute positions, after all there could be someone, somewhere, sometime that can hear interconnect differences (say). None-the-less the essential 'truth' (for me) of what dbt's show remains valid.

That is the door thru which most dbt deniers will attempt to drive through, feeling they are somehow in that elect group. Yet, if we are honest, we cannot say otherwise.

Pure conjecture here, I think knowing that people will assume they too are capable of hearing these things is behind the overzealous positions of the dbt side...in other words *our* hands are tied by honesty and the *other* side take unfair advantage of that.

I mean because a few can run a hundred metres under ten seconds does not mean most can. In audio equivalent speak most audiophiles can run 8 second times.

 

[Orb]

Amir, your post is so long I don't know where to begin. There is so much deflection that I need the debate equivalent of proper room treatment to sift through it. I'm sorry, but the noise to signal ratio is just too high. I already read your spectrometer anecdote in the AVS thread. It adds absolutely nothing to the discussion. Neither does your OJ nor any of the other anecdotes.

If you do not understand or simply refuse to admit that one cannot prove a negative, then there is no purpose for any further discussion.

You are hanging your hat on the math in your incomplete hypothetical. You did that same thing in the AVS thread and A.J. (while a complete jerk) repeatedly asked you whether the jitter was audible. The math is the math. As I posted before, the bottom line question is and always is: is it audible? This is precisely the question where the two camps converge.

So how do we go about answering that question? Using the scientific method or using faith?

Ron, whats your view about blind testing and validating the test process and results?
I ask this because would you say it is fair all blind tests should be validated otherwise to some extent it relies upon faith?
I am wondering if validation means doing subtly other test processes, capturing immense level of data that can show cognitive behaviour and heuristic pattern of the participant for perception relatated tests such as the ones we are discussing, a mixture of both, or something else.

This leads onto ABX exclusive tests and how they fit into the above statement; where we can then also come back to the different POVs from Arny and JA on that specific test.
While other blind test process do exist and have been thorough, nearly all relating to subtle differences between products come back to ABX (for me this does not constitute validation though).

Thanks
Orb

 

[amirm]

Amir, your post is so long I don't know where to begin.

Simple: see my quote of Arny? Tell me if you agree or disagree. One line: no "noise." Just one question. Let's see if an answer comes back or this broken record which takes my position and yet again pretends I don't believe it:

If you do not understand or simply refuse to admit that one cannot prove a negative, then there is no purpose for any further discussion.

I not only understand but for the tenth time, you are repeating *my* position and not answering how that is consistent with Arny's position I quoted.

You are hanging your hat on the math in your incomplete hypothetical. You did that same thing in the AVS thread and A.J. (while a complete jerk) repeatedly asked you whether the jitter was audible. The math is the math. As I posted before, the bottom line question is and always is: is it audible? This is precisely the question where the two camps converge.

How green is grass Ron? Impossible to answer, no? Jitter is a 3-dimensional distortion metric. It has amplitude, spectrum and frequency. The jitter numbers are amplitude, they do not describe the other two parameters. Jitter values can be high and inaudible or low but audible. Worse yet, its effect is content dependent. In that sense, the question has no general answer.

That said, there are examples that show audibility and one was in the very thread you said you were following but seems like, ignoring key areas relative to this question:

Chu, there's an interesting discussion of this problem (buffer underrun and overrun in DACs and the design of PLLs to combat that) in this article by Hitoshi Kondoh of BB/TI.

Thanks so much for posting that link Rock. In 20+ years I have been reading EE Times, I don’t recall a more human story related to electronic design! What wonderful wit Kondoh-san has, especially considering that English must be his second language. I would gladly give up two thirds of what I know to be able to write so well!

Anyway, for those who didn’t read it, it is a story of a DAC designer working for Burr-Brown/TI in Japan, integrating one of their DAC cores with a USB interface to be used for PC “speakers.” After a bug fix unrelated to our topic of interest, the first silicon starts to work and he sends it to US headquarters for evaluation. This is how he puts it:

“At such a time it is human nature to want various people to see (hear) the result, so we demonstrated it to all of those purported to be 'Golden Ears.' The audio signal came through the PCM1716, a DAC with an industry-wide reputation, and the PLL as the PLL1700, which has excellent C/N performance.”

Oh wait a minute. Did you catch that? Gold Ears? At an IC company? You know, the place where guys with protractors hang out? They use their ears for evaluation? Horror of horrors!

“When the guys in charge listened to the prototype I saw dubious faces and was asked a variety of questions such as "Is the source coming from the PC corrupted?" In the end I was told to measure the audio performance. When I announced the results in a subsequent meeting I was told the distortion was an order of magnitude too high; the THD+N was 0.03%.”

More horrors. Their trained listeners were able to hear artifacts prior to measurements. And measurements showed 0.03% distortion. Conventional wisdom in this thread says they shouldn’t have heard anything but seems like they did. He goes on:

“I went into this thinking "Since we are processing digital signals, we can expect good sound as a matter of course, and from here on we are dealing with digital!" So this experience was a real shock.”

So we have the engineer, as with many here, thinking prior to this experience digital had to perform well as long as it worked. Yet that wasn’t that case. The story keeps going:

“Upon Raising the FFT Resolution . . . A 100 Hz Monster Appeared!
Next, in order to investigate the skirt around the fundamental, I decided to increase the FFT resolution to a higher setting than I usually use. Naturally it took longer to make the measurement. After a wait time that would best be measured in a fractions of an hour, I was amazed at the FFT analyzer's output graph. The measured FFT is shown in Figure 9.”

This is Figure 9.

He of course figures out that this is the infamous USB frame-induced jitter biting him in the behind (and hence the reason I have cautioned multiple times that not all USB interfaces sound good):

“Even for a sample rate of 44.1 kHz, the USB isochronous mode packets have a period of 1 ms (1 kHz). In order to distribute 44.1 kHz across 1 ms intervals, one 45-sample packet is sent for every nine 44-sample packets. The tracking pulse (as we will call it here) for every 45 sample packet occurs once every 10 packets, or with a frequency of 100 Hz. Since the PLL loop filter, a so-called low pass filter, has its corner in the tens of kHz range, this 100 Hz tracking pulse goes right on through and shows up on the PLL's VCO control voltage. It appears as frequency jitter.

From the graph it is seen that the PLL frequency fluctuates impulsively right at 10 ms intervals. As a test I changed the sampling frequency to 48 kHz and measured the same 1 kHz signal.”

Here is a guy with a PhD in engineering and stumped by the classic mistake of assuming any old PLL design does the job. Not! Making a long story short, he contacts his professor from 20 years back at the University who had done research in PLLs and takes one of his ideas and modifies it. The new PLL works and everyone lives happily after. But not before he coins this clever phrase: “Since it is useless to be continually bewildered….” Boy, if there ever was a line that you wanted to steal for your signature, this would be it!

The article is somewhat technical but I hope more people read it and realize the points I made earlier. That engineers assume digital audio is more perfect than it is, and that it does take effort and good set of ears to get right. You also will realize the cost constraints involved.

So there you have a great example that is not from me so hopefully it doesn't get dismissed because I wrote the words.

So how do we go about answering that question? Using the scientific method or using faith?

Faith? So my science is faith but yours as a person not schooled in the science is well, science?

You want an answer? It starts by not demanding that it fit fit the fortune cookie paper. I advance my learnings by interacting with others and listening to what they have to say. Look at what Arny is throwing at me. They are often half incomprehensible yet I still read and respond to them. I write pages and pages to him. Why? Because I think others read and benefit from them. I have no hope that he will change. And have no hope that you change either. The hope then is that in a *debate* forum, we do just that: debate. We don't declare the other person as crazy, practicing faith instead of science, add a few debating phrases for good measure and call it done. The only thing that makes me feel good right this minute is the belief that you wouldn't use any of this in your everyday law practice!

 

[Orb]

It seems to me that most objections are coming as a result of 'extreme' statements from either side.

(thanks JA for your answer to me earlier), but I still noted at the time in your response of why a particular DBT may have been faulty (something about output impedance of the amp), yet still maintained from that one example (which may have been truly faulty, I don't know) yet concluded therefore that every dbt is faulty.

that is as extreme as the current argument about arny's statements.

Whilst not exactly addressing the title of the thread, I now wonder what the formulation regarding DBT's might be that we can ALL agree on?

Well, not all, there will be those that refuse dbt's because (to them) their cables have a truly audible difference, and refuse to do ANY sort of rigourous testing on it.

Leaving them aside (no insult intended) what statement can we agree on?

For me, it is sufficient to say 'in the overwhelming majority of cases when identity is blinded huge differences disappear'.

I try to not take absolute positions, after all there could be someone, somewhere, sometime that can hear interconnect differences (say). None-the-less the essential 'truth' (for me) of what dbt's show remains valid.

That is the door thru which most dbt deniers will attempt to drive through, feeling they are somehow in that elect group. Yet, if we are honest, we cannot say otherwise.

Pure conjecture here, I think knowing that people will assume they too are capable of hearing these things is behind the overzealous positions of the dbt side...in other words *our* hands are tied by honesty and the *other* side take unfair advantage of that.

I mean because a few can run a hundred metres under ten seconds does not mean most can. In audio equivalent speak most audiophiles can run 8 second times.

Terryj IMO you touch on something that is not normally discussed in the forums about blind test matching amps.
And that is output impedance with loading from speaker and level matching.
I understand level matching is critical to remove cues and the usual discussion online is that this needs to be 0.2db or better.
However in the real world, amps have pretty different output impedance against frequency range-speaker loading, which can also be exacerbated by its type and design of the output stage.
It is rare for moderate priced amps to have an output impedance of 0.005ohms across the whole frequency range, and quite possible comparing two different amps one will be say 0.01ohms and the other up to an uncommon 0.7ohms across frequency for SS amps, in the past the differences would had been much greater and the effect.
Anyway using real speakers and music you are talking easily 0.5db differences and even with today's SS it is possible but not common up to 1.7db differences between two amps in parts of the FR.
This is something I cannot see how one level matches against without changing the characteristic of the output stage-amp, as level matching is done at a specific frequency-tone.

However if the above is correct this leads to 2 possibilities; listeners are even worse with actual music when it comes to descriminating-masking even more problematic-etc, or something strange is happening in ABX tests.
I really would like to see further types of blind test processes involved with ABX to be sure.

Cheers
Orb