Amir, my friend, you continue to play it close to the vest. I cannot believe your claim you do not understand that blind tests don't prove anything.
You are taking my position and asking me if I believe in it??? It is very difficult to follow your arguments in this thread Ron. I almost don’t want to answer them for that reason. So here it is in my words: what I am asking is whether *your camp* believes in it. Both you and Arny use the above words but then go on to say the opposite as I quoted verbatim for him. Here it is again:
"Modern gear can be so close to being perfect that two devices with excellent but vastly different technical performance will still sound the same.
[...]
The scientific facts tell a completely different story than the post I'm replying to. Once audio performance reaches a certain level, there can still be differences but
nobody anywhere will ever hear the differences."
If you are in agreement that blind tests don't prove lack of audibility in all equipment, how are the above statements true? Clearly they were not based on measurements or design – both of which Arny presupposes as being different. What other area of science then gave those conclusive data in his post?
I cannot believe you've repeated the dangerous statement which opens the door to all of the highly suspicious claims of audibility.
“Dangerous?” I am supposed to tiptoe around a topic because you are worried about someone else running with them? How about the countless people who run wild with conclusions from your camp leaders? Recall the jitter thread with Ethan where I showed on a case by case basis how there were no jitter audibility data in AES that proved them to be inaudible? Why don’t you think those reports were dangerous? I have had to counter them in more threads than I can count with people who don’t even know what the word jitter means, but read the conclusion in that paper that it was not audible at certain amounts. The mark of AES and “bind test” was good enough for them. Damn the details of what was tested and how.
We cannot *prove* that sprinkling some magic pixie dust on a person's head never will cure that person from cancer. Likewise we cannot prove that placing a DAC upside down never will cause an improvement in, e.g., soundstage. Do we infer from that absence of cause and effect proof that the cause indeed can have the intended effect?
How many times do I have to correct you Ron on the position I have taken on this thread? I am specifically talking about cases where we can with *100% scientific certainty show measurement differences.* Upside down DACs and pixie dust may not apply unless they do show a measured difference in which case, I would love for you to explain that away
.
You either believe in the scientific method, i.e., that the perceptual world can be studied by humans, or you don't, i.e., you believe in faith. But please, pretty please, with magic pixie dust on top, please stop im- (if not ex-) plicitly stating that bias can go on holiday. Humans are absolutely expert at self-delusion. Extraordinary claims require extraordinary proof.
“Scientific *method*” translates into a tool. As with any tool, you better know its limitations or else, you get yourself in trouble and deep. Take video measurements. You can buy a spectrometer for $70, or get the Minolta unit I have for $20,000. Both follow “scientific method” of measuring colors on your display. If you knew nothing different, you would likely buy the $70 unit thinking all is well. Then you go and talk to an expert and he tells you that as the luminance level (brightness) becomes lower, the cheap meter gets swamped with noise and loses all accuracy. And unfortunately for our pocketbooks, those levels are 100% visible to the eye. Without the more accurate meter, you would have a color shift in your monitor as the video became darker. Not a good thing if you care about fidelity and correctness.
I personally use both of above solutions. I use the cheap unit to quickly calibrate my computer monitors around the house. Its software is fully automated and anyone in two minutes can have a far more accurate display. Would I use that on our $60,000 projector at work? Heck no. I use the above Minolta meter because people are sitting in the dark watching a 17 foot image with a high price tag and expect it to have the right tones.
By the same token, I have lived through so many blind tests professionally that I have formed an opinion of where they are solid as a rock and where they might produce faulty data. Here is an example I have given elsewhere. We once paid an independent and major testing company to evaluate the quality of our audio codec at 64 Kbps against the CD. To my pleasant surprise, the people conducting the test said they wanted to follow the ITU measurement standards to the letter and that they wanted to use reasonably high quality headphones and such to make sure users were not fooled one way or the other. We said fine.
After $25,000 and 2-3 weeks with > 100 people recruited for the test, do you know what they found? That 90%+ of the people could not tell the 64kbps version from the original. Let me repeat. They followed textbook methodology of double-blind testing as standards by International Telecommunications Union and said that across a pretty large population, folks could not detect that we had thrown out 96% of the signal. So our marketing folks broke out the Champaign and declared to everyone how good our technology was.
Of course, the truth was nowhere near that. The testing company couldn’t see the forest from the trees. It didn’t understand how compressed music worked and what content would be difficult and what would be easy. They thought for example the classical music was hard because audiophiles use them as reference tracks. Well, classical music is harmonic and hence, can be easier represented than something that suddenly changes like a transient. Had they picked MPEG reference clips for example, I doubt that 30% of those people would say the clips sounded the same as the CD. So clearly there is more to the “scientific method” than the fact that it is double blind and follows textbook proceedures.
Another example that might resonate with you and others better. Barry Scheck was the lead defense expert for OJ Simpson. His work prior to that trial was with the Innocence Project where he would try to get wrongly convicted prisoners freed by using DNA testing. So I found it quite hypocritical when he was called to testify that DNA testing was too unreliable as a scientific method to convict OJ Simpson. Later though, I saw an interview of him where he said that DNA tests can conclusively rule out someone being connected with the scene of a crime. But the reverse, is a game of probability and not absolute. And that even if the probabilities are high, we still cannot know for sure if someone’s DNA was left at the scene of the crime. He followed by saying that when we are talking about putting someone in prison for life, we better be sure.
So here is a “scientific method” that seems to be valid in one direction but not the other. To the layman like me, that distinction was not apparent and hence, I thought the guy who would say anything for money. I was wrong. Or at least I think I was. Someone with more expertise would have a more informed opinion than I. Fact is that if you are not schooled in the science, you may not know all that there is to know to understand whether it is being applied correctly or not.
This week I spent two wonderful days at Harman, including hours of presentation and data on blind testing. Yet on the simple question of how you would treat the side wall by a speaker, there were 2.5 to 3 answers from three world-renowned experts in speakers and acoustics! Are there three versions of the truth or are two of them wrong? The logical conclusion is that the scientific method is not complete enough to lead us to a single answer. We have not researched the topic well enough, or the topic itself is too challenging to yield itself to amount of research we have performed to give us one answer across the full range of audio topics.
On a personal level I have to say, your position seems to be that if I disagree with you, I must be delusional. I find that illogical to put it politely. If I tried to give you lessons on the meaning of the US constitution or what contract law means, you would probably laugh at me. I have practically lived with attorneys for the last 10 years so much so that my attorney lets me argue with the opposing one without him being present half the time. Yet, there is not an instant that I would try to teach him how the law works. How come you feel so confident in being right and me being wrong in matters related to audio here? Why isn’t there door ever so ajar that I might, just might, know something you don’t?
I realize folks want one-liners that come out of fortune cookies such as Arny used and you mention above. I appreciate how simple that can make our lives. But please don’t look to me to provide them. I am bewildered by some audio topics, even though my signature says I like to avoid it as best as I can
. The nature of the science is quite complex at times.