Blind Listening Comparison

[Gregadd]

I think they call that projection.

 

[henrich3]

Please just cite one recognized authority accepting a single blind study as a recognized standard.

Single blind test are allowed but only when that is the only option. It is irrelevant here because it is so easy to achieve double blind. When you use single blind they are less reliable.. Impossibility is the rationale for a single blind test. Not inconvenience or informality.

I think that you're confusing the degree of scientific rigor that's appropriate for a peer-reviewed scientific journal versus what's reasonable to offer up as evidence in a WBF thread.

 

[Gregadd]

I think that you're confusing the degree of scientific rigor that's appropriate for a peer-reviewed scientific journal versus what's reasonable to offer up as evidence in a WBF thread.

Thank you.
I understand the difference. I do sighted evaluations. I think someone else is being disingenuous about how "what's reasonable to offer up as evidence in a WBF thread" can be applied. I might add though has that not always been there thing? We are wrong because we are unscientific.
Fortunately no one needs my permission.

 

[Gregadd]

I think that you're confusing the degree of scientific rigor that's appropriate for a peer-reviewed scientific journal versus what's reasonable to offer up as evidence in a WBF thread.

So seriously take the results and methodology of the test being discussed and tell me what you could reasonably conclude based on the results. Don't worry there is no wrong answer.

 

[microstrip]

I think that you're confusing the degree of scientific rigor that's appropriate for a peer-reviewed scientific journal versus what's reasonable to offer up as evidence in a WBF thread.

We can easily know pretty well what is a proper listening blind test, unfortunately no one knows what is a blind test that " is reasonable to offer up as evidence in a WBF thread" .

 

[henrich3]

So seriously take the results and methodology of the test being discussed and tell me what you could reasonably conclude based on the results. Don't worry there is no wrong answer.

You spilled considerable digital ink railing against single blind test results. My post was just pointing out that the rigors of double blind testing & other controls shouldn't be a requirement if an objectivist audiophile simply wants to enlighten folks on a forum about some single blind test results. They're without a doubt far better than sighted tests if one wishes to divine whether A is better than B.

 

[Gregadd]

You spilled considerable digital ink railing against single blind test results. My post was just pointing out that the rigors of double blind testing & other controls shouldn't be a requirement if an objectivist audiophile simply wants to enlighten folks on a forum about some single blind test results. They're without a doubt far better than sighted tests if one wishes to divine whether A is better than B.

Ok. you would generalize from the specific. Fair enough.
Thank you for your response..

 

[thedudeabides]

k, you've tried to play every card to avoid you've been wrong and haven't addressed my real arguments.

Youl are a brave soul to argue science on WBF. I admire your moky. There are very few folks on this forum (myself included) who would agree with your cited studies because most folks here choose, for better or worse, a subjective approach versus objective criteria. It is simply a difference of opinion.

In the end, this type of discussion is a black hole. No one is right or wrong.

 

[microstrip]

Youl are a brave soul to argue science on WBF. I admire your moky. There are very few folks on this forum (myself included) who would agree with your cited studies because most folks here choose, for better or worse, a subjective approach versus objective criteria. It is simply a difference of opinion.

In the end, this type of discussion is a black hole. No one is right or wrong.

We can't forget that once properly expressed and analyzed subjective findings can become objective data.
And we can use objective criteria to guide our subjective findings.

Science is also accepting the limitations of the experiments due to restrictions needed to carry them.

 

[Gregadd]

Basic factors not discussed
Truly double blind- a test examiner who does not anything about cables. their ignorance would tecnnicaly make them "blind" to the choices.
matched levels- people tend to pick the louder one.
music or test tones. tones tend to be preferred because you know it assures you are comparing apples to apples. Many use a 15 second clip
rapid switching- short term memory of what you heard is not very good.
multiple test subjects-very easy for one person to make a mistake or just be wrong.
number of trials-- you have to flip that coin a certain number of times before you can say anything about the outcome.
control -just to have a benchmark so to speak.
ratio of positive /to negative outcomes -informal confidence level. easy to calculate 8 of 10 would be 80%.
stake in the outcome. What was the method of comparison. Did the subject endeavor to match a to b in a binary choice? Did the listen to sevral cables and try to pick a preference? There are options.
preexisting bias against the efficacy of the subject under test. If you already believe the product is snake oil or incompetent you are not a good test subject.
Which of these would you be willing to sacrifice and still have confidence in the results?
That's just off the top of my head . Things I have learned from 10 years of debating the most vociferous proponents of double blind testing. Those issues should be addressed even for "evidence on WBF". IMO YMV.

 

[henrich3]

Basic factors not discussed
...
preexisting bias against the efficacy of the subject under test. If you already believe the product is snake oil or incompetent you are not a good test subject.
Which of these would you be willing to sacrifice and still have confidence in the results?

Good list, but if the test participants are blinded, then the "preexisting bias" factor is eliminated and you don't have to screen for it in your participants. That's why it's important to do blind testing.

 

[Tangram]

Basic factors not discussed
Truly double blind- a test examiner who does not anything about cables. their ignorance would tecnnicaly make them "blind" to the choices.
matched levels- people tend to pick the louder one.
music or test tones. tones tend to be preferred because you know it assures you are comparing apples to apples. Many use a 15 second clip
rapid switching- short term memory of what you heard is not very good.
multiple test subjects-very easy for one person to make a mistake or just be wrong.
number of trials-- you have to flip that coin a certain number of times before you can say anything about the outcome.
control -just to have a benchmark so to speak.
ratio of positive /to negative outcomes -informal confidence level. easy to calculate 8 of 10 would be 80%.
stake in the outcome. What was the method of comparison. Did the subject endeavor to match a to b in a binary choice? Did the listen to sevral cables and try to pick a preference? There are options.
preexisting bias against the efficacy of the subject under test. If you already believe the product is snake oil or incompetent you are not a good test subject.
Which of these would you be willing to sacrifice and still have confidence in the results?
That's just off the top of my head . Things I have learned from 10 years of debating the most vociferous proponents of double blind testing. Those issues should be addressed even for "evidence on WBF". IMO YMV.

Your points don’t invalidate double-blind testing - they just emphasize that it’s difficult to do it well. This trope is often trotted out to discredit double-blind testing in defence of the typical, subjective tests that most audiophiles, including reviewers with (apparently) superhero caliber aural memories conduct.

In my opinion, it’s in the best interests of the industry to downplay double-blind tests.

 

[Gregadd]

Your points don’t invalidate double-blind testing - they just emphasize that it’s difficult to do it well. This trope is often trotted out to discredit double-blind testing in defence of the typical, subjective tests that most audiophiles, including reviewers with (apparently) superhero caliber aural memories conduct.

In my opinion, it’s in the best interests of the industry to downplay double-blind tests.

I is not a trope. It is a summary of double blind testing protocol. I have no desire to invalidate double blind testting. But if they are to have any credibility the must be done properly. "It is difficult tp do well." Science is hard sometimes.

 

[Gregadd]

Good list, but if the test participants are blinded, then the "preexisting bias" factor is eliminated and you don't have to screen for it in your participants. That's why it's important to do blind testing.

Thank you for being open minded.
I don't think the bias is eliminated. Would you hear a difference if you don't want to and already decided there is no difference. Bias works both ways.

 

[Tangram]

I is not a trope. It is a summary of double blind testing protocol. I have no desire to invalidate double blind testting. But if they are to have any credibility the must be done properly. "It is difficult tp do well." Science is hard sometimes.

As a former scientist, I agree.

The trope is that unless done to absolutely exacting, unrealistic standards (for example, you must know nothing about audio equipment to be a participant), it is of no value.

Ergo, don’t bother.

I recently conducted a blind test with six different record weights and concluded I wasn’t able to hear any difference, including between having a weight on and having a weight off. If there was any bias, it was the other way - that the differences would be easily discerned and repeatable. Someone else’s ears may be able to hear the differences under the same test conditions, but I couldn’t. My 59 yr-old ears are fine (tested annually) and I’ve got a system that should be up to the task of hearing something caused by a mechanical change to the conditions a transducer is operating under. But no, that wasn’t the case.

The “funny” thing about double-blind testing is that it, too, is subjective.

 

[andromedaaudio]

Does clueless triple blind testing fit the scientific protocol.

Hearing is completely subjective just like people who visually judge a piece of art
Some modern art is completely rediculous in my view .
A van Gogh however is fenomenal art .

Beauty lies in the eyes / ears of the beholder.
Discussing it is merely a distraction of the beauty

 

[henrich3]

Thank you for being open minded.
I don't think the bias is eliminated. Would you hear a difference if you don't want to and already decided there is no difference. Bias works both ways.

I am predisposed to believe, for example, that power cord upgrades do not make an audible difference. That said, if I were a blinded test participant in a power cord test and I did hear a difference, I would certainly change my tune. I would, however, want to follow up that revelation with measurement tests of both power cords to determine why I heard a difference. Are the resistance, capacitance, or inductance values of the two power cords significantly different? If so, what would account for that? If the RCL values weren't significantly different, then why would I hear a difference? That would certainly be baffling.

It would then be interesting to perform frequency response measuments of the system being tested using the different power cords. Do they measure differently? If I hear a difference, then I should also see a difference in the FR measurements. If there were any significant FR differences, then those would actually be nulled out if an owner of those fancy power cords re-ran their room EQ setup which would set the frequency response back to its target curve.

 

[Gregadd]

"The trope is that unless done to absolutely exacting, unrealistic standards (for example, you must know nothing about audio equipment to be a participant), it is of no value."

Ergo, don’t bother."

I am not saying that at all. My point all along is the relative credibility of the test given dependent on the methodology. If I were saying what you suggest, I myself would be disqualified because I have strong opinions. Ideally the better studies I have seen have trained and amateur listeners. They do share one trait. They have no stake in the outcome. You may think that a medicine that a guy took on his own was of interest, but you would not take that to the FDA.

 

[Gregadd]

I am predisposed to believe, for example, that power cord upgrades do not make an audible difference. That said, if I were a blinded test participant in a power cord test and I did hear a difference, I would certainly change my tune. I would, however, want to follow up that revelation with measurement tests of both power cords to determine why I heard a difference. Are the resistance, capacitance, or inductance values of the two power cords significantly different? If so, what would account for that? If the RCL values weren't significantly different, then why would I hear a difference? That would certainly be baffling.

Fair enough, but expectation bias is a double edged sword. I guess we could "blind" you to the fact that it was the power cord that is under consideration.

 

[henrich3]

Fair enough, but expectation bias is a double edged sword. I guess we could "blind" you to the fact that it was the power cord that is under consideration.

That would be fine too.