Truth and Tonality: can they co-exist?

[Gregadd]

I can't speak for Mr. Fremer. You guys can google the info jut like I did.

The point is Mr. Fremer and Mr. Atkinson with their credibility publicly on the line took the DBT set up by the AES. The argument was there is no difference between amps with matched levels under double blind conditions. That the subjectivist reviewers were afraid to take the test because they knew they would be unable to detect a difference. Mr. Atkinson and Mr. Fremer proved

1. If they were afraid they did it anyway. 2. They were able to detect a difference.

For them at least the notion ,that subjective reviewers are afraid of DBT because they can't identify the differences they claim to hear under DB condition is forever settled.

You guys are going to have to abandon your DBT argument and return to arguing on the merits.

 

[Phelonious Ponk]

I can't speak for Mr. Fremer. You guys can google the info jut like I did.

The point is Mr. Fremer and Mr. Atkinson with their credibility publicly on the line took the DBT set up by the AES. The argument was there is no difference between amps with matched levels under double blind conditions. That the subjectivist reviewers were afraid to take the test because they knew they would be unable to detect a difference. Mr. Atkinson and Mr. Fremer proved

1. If they were afraid they did it anyway. 2. They were able to detect a difference.

For them at least the notion ,that subjective reviewers are afraid of DBT because they can't identify the differences they claim to hear under DB condition is forever settled.

You guys are going to have to abandon your DBT argument and return to arguing on the merits.

Wrong. Let's give this example a HUGE benefit of the doubt. Let's assume that Mr. Fremer would have continued to get it right, over and over again, all the way up to the point where he had exceeded the margin for error and proven, statistically, that the difference between those two amps was audible. Surely you understand that doesn't mean he's right about everything else he has ever "heard?" Or that every other audiophile, which includes guys who hear massive expansions of sound stages when they sit pyramids on top of their preamps, have been right about everything else they ever heard? I know you get logic better than that. If anything, this offers evidence (nothing has been proven here but Fremer's and Atkinson's confidence) that blind listening is a good idea, even for the most subjectivist audiophiles. Confirmation is good, huh? Vindication might be even better. It doesn't compel anyone to return to whatever is your personal vision of arguing "on the merits." If anything, it provides a bit of evidence that blind listening might help us understand what the merits are and are not.

By the way, I don't personally think the differences between amplifiers are inaudible, though I find that beyond a point, they are insignificant.

Tim

 

[Gregadd]

Tim I don't see an ounce of proof in your post. It is so much ipse dixit. I recall that when I previously told you that Fremer had been able to discern a difference you doubted that it even occurred.

Let me say once again and hopefully for the last time. POINTING TO THE ABSENCE OF DBT IS NOT PROOF. I NEED THE RESULTS OF A PROPERLY CONDUCTED DOUBLE BLIND PROTOCOL COMPLETE WITH PROPER STATISCAL ANALYSIS. Bring me that and I'll consider it. Don't tell me it's too hard, too expensive, or that the method you use is "good enough."


Someone can correct me if wrong from memory. I beleive amplifier was the solid sate Adcom GFA -555 vs the VTL 300. Tube vs. Solid . The AES had already taken the position that all amps operating within thier capabilities sound the same. How do we know then that thier failure to detect a difference was not the often stated by propenets of DBT "expectation bias." Moreover I agree with Mr. Fremer. The same AES that claims tube amps are so colored, was completely unable to detect those colorations under blind conditions! Now that's news!

 

[amirm]

Guys, it would be good to not go down the DBT arguments again. The part of the story I thought was relevant was in regards to the question I asked. I said measurements are most reliable, followed by double blind and finally subjective sighted test. To the extent we have two reviewers get it right in DBT supports the notion that DBTs can produce reliable results. It for sure is not in support of the opposite view that DBTs cause so much disturbance that no good results can come out of it and hence, should be ranked below sighted tests.

We were all taught the basics of Mathematic Induction in school. It says for something to be true, we need to prove it for N=1, N and N+1. So let's see if we can apply at least part of that here.

Let's say I have a CD ripped to my hard disk. I then make a copy of it. This is what I call N=1. Let's see how the three methods would do in this:

1. Measurement. Raise your hand if you think the measurements would be different for these. It would not including the simplest of subtracting them from each other but going as far as looking at spectrum, response, etc. It is also very repeatable.

2. Double-blind tests. We routinely run such a "control" in double-blind tests to see if a) our equipment and test set up is working and b) our listeners are not deaf . And in every test I have seen, better than 90% of the people get this right. Why not 100%? Well, some people think since they are being tested, there MUST be a difference so even though they hear none, they vote there is anyway. The percentage is not zero either because we usually have a lot of people who are disinterested in the outcome so they vote more truthfully than otherwise.

3. Sighted. I can speak from personal experience that then 10% effect above is quite a bit more magnified here. We have plenty of evidence for this. Here is Floyd & Sean Olive's observations in this section of his excellent book titled, “Loudspeaker and Headphone Handbook”:

” In practice, the principal difficulty with subjective evaluations is to control what the listeners are responding to. Listeners who, because of room acoustics, hear different sounds from the same loudspeakers cannot possibly agree. Listeners who see the products they are evaluating cannot completely distance themselves from the biasing influences of style, price, size, brand reputation, and so on. Listeners who hear sounds that are unreasonably altered by atypical acoustical surroundings cannot express a view that represents that of a larger population. Music that reflects only a narrow spectrum of what customers play cannot be assured of revealing a balanced perspective of performance. We need to get serious about the conduct of listening tests – so that ‘opinions’ can take on the attributes of ‘facts’, to create a system of subjective evaluation that is a proper companion to technical measurements.

It is well known, in subjective evaluations, that humans are susceptible to influences other than the parameter or device under test. We simply find it very difficult to ignore the evidence of brand, size, price, and so on.

...In loudspeaker evaluations, there is no argument that there are audible differences between products, and clearly audible differences traceable to interactions with programme and rooms. In comparisons of evaluations done in both blind and sighted conditions, it has been observed that listeners substantially altered their ratings of products when they were in view, following biases suggested by visual cues. "

So again, without digging deep into each bucket, do we not rank the accuracy of what is measured in the order I listed?

 

[amirm]

Note also how they make another point I made earlier: the dangers of picking a sample of music and expecting it to a) be representative of all music out there and b) be revealing.

I can't tell you how many audio codec shoot outs we won because the person walked in with music that we knew would show no difference yet fell in the category of "audiophile" music. You could always tell because I had a smile on my face before the test started . I wonder how often a manufacturer at a show has a smile on their face when they hear your selection of music .

 

[Phelonious Ponk]

Tim I don't see an ounce of proof in your post.

Greg, I don't see an ounce of proof in this thread! All I'm objecting to at this point is this statement:

You guys are going to have to abandon your DBT argument and return to arguing on the merits.

Wow. OK. Here's the deal: I come up with one example of one guy, in one informal blind listening test, who got it wrong five times in a row and you guys abandon your "I hear what cannot be measured" argument and return to arguing on the merits. Deal?

Tim

 

[Gregadd]

Greg, I don't see an ounce of proof in this thread! All I'm objecting to at this point is this statement:



Wow. OK. Here's the deal: I come up with one example of one guy, in one informal blind listening test, who got it wrong five times in a row and you guys abandon your "I hear what cannot be measured" argument and return to arguing on the merits. Deal?

Tim

I've always argued on the merits based on what I hear. I readily admit that is my opinion. Or even better my interpretation of what I heard. You can't rebut an opinion with an opinion. Nor can you rebut an opinion with what a DBT might show.

 

[amirm]

So here is how I would net out one's ability to know the audio "truth:"

1. How well do your ears work? Have you verified their evaluation ability? I am not talking about audiology tests of frequency response. I am talking if I changed the feedback level in an amp, can you hear the difference? How sure are you that you could?

2. How well do you understand the design of a system? If you don't know the difference between two different engines in your car or terminology used to describe them, how would you choose one vs the other?

3. How well do you understand third-party reviews of systems? Do you understand all the charts, measurements and words used to describe the performance of the unit?

4. How much experience do you have in listening -- truly listening -- for lack of fidelity? I am not talking about how much you listen to music or how many concerts you have gone to. Little of that matters. To use your ear as an instrument, you need to teach your brain what to hear. And that only happens in controlled situation where only one variable is changed and you can then correlate what you are hearing against the change that was made. See Fremer's point about looking at the measurement to match his observation and see if he can connect the dots.

Give yourself a numeric score from 1 to 5 and then add them up and see what you get!

 

[Gregadd]

Amir it was you yourself who said that DBT is not necessary for significant differences.

 

[Ron Party]

Greg, it is not only Amir but everyone who believes in the validity of DBTs. A rhetorical example might be: do we need a DBT to determine whether there is an audible difference between Steve's speakers and a pair of Bose. There is every reason, both objective and subjective, to conclude there is a significant audible difference between these speakers.

Now there are at least 2 qualifications to this: (1) what if there is not an objective reason to believe there is an audible difference (or, related to this, the purported objective reason is not accepted by all, which is where a lot of the food fights occur); and, (2) DBTs still can be a useful tool to determine preference.

 

[Gregadd]

So here is how I would net out one's ability to know the audio "truth:"

1. How well do your ears work? Have you verified their evaluation ability? I am not talking about audiology tests of frequency response. I am talking if I changed the feedback level in an amp, can you hear the difference? How sure are you that you could?

2. How well do you understand the design of a system? If you don't know the difference between two different engines in your car or terminology used to describe them, how would you choose one vs the other?

3. How well do you understand third-party reviews of systems? Do you understand all the charts, measurements and words used to describe the performance of the unit?

4. How much experience do you have in listening -- truly listening -- for lack of fidelity? I am not talking about how much you listen to music or how many concerts you have gone to. Little of that matters. To use your ear as an instrument, you need to teach your brain what to hear. And that only happens in controlled situation where only one variable is changed and you can then correlate what you are hearing against the change that was made. See Fremer's point about looking at the measurement to match his observation and see if he can connect the dots.

Give yourself a numeric score from 1 to 5 and then add them up and see what you get!

I guess it does not matter what I say.

Let me make the car racing analogy. The only thing that matters is who crosses the line first. That's the truth I'm looking for. Mr. Femer and Mr. Atkinson crossed the line first. They followed all the rules. They were disqualified.

 

[amirm]

I guess it does not matter what I say.

I can honestly say the feeling is mutual .

Let me make the car racing analogy. The only thing that matters is who crosses the line first. That's the truth I'm looking for. Mr. Femer and Mr. Atkinson crossed the line first. They followed all the rules. They were disqualified.

How do you know which car crosses first if they are awfully close? You use cameras, instruments, etc. If you are standing sideways, you might think the wrong car won! Besides, there is no simple truth like that in audio. That is what the thread is all about. For you to be right about your analogy, audio evaluation should be that easy, that objective and that authoritative. If I ask the opinion of a tube amp, I get 10 different answers. In what way is that anything close to your finish line business?

Your angle there that your ears tell you that is what I disputed before. There is no evidence that your ears speak the truth as the instruments in that finish line do. Not even close. We have ample evidence that all of our ears misfire and misfire drastically. I have outlined my own failings here: http://www.whatsbestforum.com/showthread.php?1014-Your-most-embarrassing-audio-moments

If for a moment, all of us including Mr. Fremer accepted that what we hold as strong belief is an educated guess when we subjectively evaluate equipment, then all is well. Look at CES reports. It is all over the map on what was best. I heard someone say one room was worst while another said it was the best. We have that as real data. These are people who claim to know all about audio and are avid audiophiles or why would they go to CES? Yet they don't even remotely agree with each other.

It is hard for us to accept criticism. As males, we have far more problem with this. We can't admit failing easily. But if we want to learn and be better, that is the first thing we need to accept. You see me volunteering the above thread. Read what followed. Hardly anyone admitted the same embarrassing faults with audio. Think about that! Learn something about yourself in this interchange. That is something we can accomplish. The truth about audio? That is very hard .

 

[Phelonious Ponk]

I've always argued on the merits based on what I hear. I readily admit that is my opinion. Or even better my interpretation of what I heard. You can't rebut an opinion with an opinion. Nor can you rebut an opinion with what a DBT might show.

I've always argued the merits of what I hear, my subjective opinion, and what can be measured, the objective evidence, including the rare DBTs of audio that have sound methodology and statistically valid results. I don't think you'll ever find me arguing that casual blind listening is proof of anything. You've argued the merits of your own opinion and pretty consistently rejected any objective data that disagrees with it. And you're not only doing it again now, you're attempting to get it eliminated from future discussions.

Tim

 

[mep]

There is no evidence that your ears speak the truth as the instruments in that finish line do. Not even close. We have ample evidence that all of our ears misfire and misfire drastically. I have outlined my own failings here: http://www.whatsbestforum.com/showthread.php?1014-Your-most-embarrassing-audio-moments

Amir-I just went back and read your post where you documented your "failings." You came to the following conclusions:

1. your brother's tube amp sounded better than your new SS amp except in the bass when turned up loud
2. LP sounded better than CD by a wide margin on the day you had your shoot-out
3. You couldn't tell one digital file from another

Many would argue that all 3 points are still valid today (except drop off the part about it being your brother's tube amp and just say tube amps). At least you admitted you were wrong in your judgements at the time. In the first two cases you presented, measurements then and now would tell us it simply couldn't be so right? And I'm sure there are some on this forum that would argue that you weren't right, you didn't hear what you thought you heard. Your brother's amp did not sound better than your new SS amp and the CDs certainly sounded better than the LPs you played on that day. The measurements said so.

 

[amirm]

Correct. I find faults with all three methods. Combined though, I find them more powerful than any single tool.

 

[Gregadd]

I can honestly say the feeling is mutual .


How do you know which car crosses first if they are awfully close? You use cameras, instruments, etc. If you are standing sideways, you might think the wrong car won! Besides, there is no simple truth like that in audio. That is what the thread is all about. For you to be right about your analogy, audio evaluation should be that easy, that objective and that authoritative. If I ask the opinion of a tube amp, I get 10 different answers. In what way is that anything close to your finish line business?

Your angle there that your ears tell you that is what I disputed before. There is no evidence that your ears speak the truth as the instruments in that finish line do. Not even close. We have ample evidence that all of our ears misfire and misfire drastically. I have outlined my own failings here: http://www.whatsbestforum.com/showthread.php?1014-Your-most-embarrassing-audio-moments

If for a moment, all of us including Mr. Fremer accepted that what we hold as strong belief is an educated guess when we subjectively evaluate equipment, then all is well. Look at CES reports. It is all over the map on what was best. I heard someone say one room was worst while another said it was the best. We have that as real data. These are people who claim to know all about audio and are avid audiophiles or why would they go to CES? Yet they don't even remotely agree with each other.

It is hard for us to accept criticism. As males, we have far more problem with this. We can't admit failing easily. But if we want to learn and be better, that is the first thing we need to accept. You see me volunteering the above thread. Read what followed. Hardly anyone admitted the same embarrassing faults with audio. Think about that! Learn something about yourself in this interchange. That is something we can accomplish. The truth about audio? That is very hard .

In determining who won the race we shure don't do it blind. Ultimately no matter what aids we use it comes down to human perception.

Mr. Fremer was minding his business when he was attacked by the AES DBT crowd. He is a critic not a scientist. He went to AES because they attacked him. As smart as they are they should have known he had figured out a way to defeat thier test.

Listening does not count? That's like saying Jimmy Stewart's experience with NASCAR does not count. I would take his advice over a scientist any day of the week.

Truth? Just whose truth is that? Learn something? Everybody wants you to measure using their yardstick and usually when you do they win. If they lose they want to change the yardstick.

Getting back to an earleir example I made. When my smoke alarm goes off I ain't running out of the house until I smell smoke or see fire.

I am a lawyer and I assure you I can argue anything. So maybe we ought to just agree to disagree.

 

[Gregadd]

Greg, it is not only Amir but everyone who believes in the validity of DBTs. A rhetorical example might be: do we need a DBT to determine whether there is an audible difference between Steve's speakers and a pair of Bose. There is every reason, both objective and subjective, to conclude there is a significant audible difference between these speakers.

Now there are at least 2 qualifications to this: (1) what if there is not an objective reason to believe there is an audible difference (or, related to this, the purported objective reason is not accepted by all, which is where a lot of the food fights occur); and, (2) DBTs still can be a useful tool to determine preference.

Everyone believes in DBT? There is a sentence in serious need of a qualifier.

 

[mep]

Everyone believes in DBT? There is a sentence in serious need of a qualifier.

You left out a qualifier: Everyone WHO believes in DBTs.

 

[Gregadd]

I've always argued the merits of what I hear, my subjective opinion, and what can be measured, the objective evidence, including the rare DBTs of audio that have sound methodology and statistically valid results. I don't think you'll ever find me arguing that casual blind listening is proof of anything. You've argued the merits of your own opinion and pretty consistently rejected any objective data that disagrees with it. And you're not only doing it again now, you're attempting to get it eliminated from future discussions.

Tim

I agree that you stay on the merits when pressing your points. It's when you challenge others you resort to the "lack of DBT"

Just as you find it frustrating when others imply "you do not listen" I find it equally frustrating when others imply "I ignore measurements."

The only argument I am trying to eliminate is that subjective reviwers are afraid DBT because they will be unable to idnetify the differences they claim to hear.

 

[Gregadd]

You left out a qualifier: Everyone WHO believes in DBTs.

Thanks. I need to clean the glasses. My apology to Ron.