DIscussion of ABX results of Winer's Loopback files

[maxflinn]

Conclusion - that sightedness brings a bias to what we hear. It doesn't prove that there is actually no difference

Agreed, though it can be an (whispers) indication

What you have to grasp is that testing for perceptually subtle differences is difficult.

Agreed again.

The usual argument is that declarations of night & day differences is hyperbole, Absolutely, agreed - these are subtle differences.

And you get the hat-trick - Agreed for a third time, there's hope for us yet

 

[jkeny]

Agreed, though it can be a strong indication.

Agreed again.

And you get the hat-trick - Agreed for a third time, there's hope for us yet

Wow, yes, I'm in awe at our agreement - maybe we can have a pint sometime?

If you simply left out the "strong indication" we would have a complete agreement. Will ya, ah go on now, you will, you will, (in best Ms Doyle voice)

 

[maxflinn]

Wow, yes, I'm in awe at our agreement - maybe we can have a pint sometime?

Absolutely

Just goes to show how when nobody is being put under undue pressure, common ground can be found

 

[FrantzM]

Conclusion - that sightedness brings a bias to what we hear. It doesn't prove that there is actually no difference
What you have to grasp is that testing for perceptually subtle differences is difficult.

The usual argument is that declarations of night & day differences is hyperbole, Absolutely, agreed - these are subtle differences.

John

One of the few rare times I agree with you.

Although I am a proponent of blind testing method or what I call "Knowledge removed", I don't take it as definitive and admit that rigorous ABX is not trivial. I see it however as less flawed than any other method. If anything blind tests should serve as a baseline to weed out our audiophile penchant for hyperbole. To keep up honest if you will.

 

[jkeny]

Seeing as we are getting somewhere, I'll do a bit of the work for you

If you search on this forum for J_J's posts - he being one of the leading experts in codec testing & design (I'm sure I'm overlooking his many other accolades) - you will find this post outlining his very quickly posted, off the top of his head criteria for a proper blind test

A short, and undoubtedly insufficient list (since I'm writing this off the cuff) would be:

1) listener training
2) quiet, single-listener situation, with equipment, acoustics, etc of appropriate quality
3) negative and positive controls, and stimulus repetition for evaluation of consistency
4) perfect time alignment and level alignment (either of those off by much at all will absolutely result in a positive result)
5) feedback during training and after each individual trial
6) consistent A and B stimuli, which the subject is permitted to know, and who can refresh their recollection at any time. This is also an element that can easily cause any test to be positive by mistake.
7) transientless, quiet switching between the signals, with extremely low latency. Switch transients can cause either lower sensitivity or unblind a test, depending on how they arise.
8) the ability to loop the test material under user control
9) of course the setup must be double-blind, ordering must be varied, etc. All standard test confusion issues must be satisfied.


That's just few, that's not even close to a full set, but just that much shows how it isn't easy to run a good test.

The biggest flaw as I see it, in any tests I've seen run on forums, is the lack of controls - the "testing of the test" itself. This would answer a lot of possible objections

 

[jkeny]

John

One of the few rare times I agree with you.

Although I am a proponent of blind testing method or what I call "Knowledge removed", I don't take it as definitive and admit that rigorous ABX is not trivial. I see it however as less flawed than any other method. If anything blind tests should serve as a baseline to weed out our audiophile penchant for hyperbole. To keep up honest if you will.

Thanks, Franz, it really is a red letter day then

 

[maxflinn]

Thanks Max. The band is a bit quiet at the moment, but that's a good thing. It gives us time to work on new material and work in the new bass player. Besides. It takes ungodly powerful air-conditioning to make rock-n-roll comfortable in North Carolina in August, and outdoor gigs, even on relatively cool summer nights are still sticky. July and August just aren't pleasant here, period. The winters, on the other hand, are Godsend. And Spring and Fall are both just gorgeous.

Tim

I'm thirsty now, Tim, and yearning for Budweiser and Southern Comfort

 

[microstrip]

Can I use and edit that part of your post ever so slightly?

Differences and gross differences at that with knowledge of the components under test
Remove the knowledge.
Differences gone or too often differences perceived when there aren't any (same component playing but mistaken for different ones)

Conclude???

1. There was no difference and it was all bias.
2. The methodology being used for the blind tests obfuscates the differences.

Anyone is free to pick 1 or 2. When I read references to same component playing but mistaken for different ones , but not a single reference to positive control tests I know I am wasting my time.

 

[microstrip]

(...) The usual argument is that declarations of night & day differences is hyperbole, Absolutely, agreed - these are subtle differences.

Probably many times. But very often these "subtle" differences make the difference between a competent and a great high-end system.

IMHO just because there are many situations where one can scent hyperbole everywhere does not mean that there are no situations of "night and day" due to small changes,

 

[Phelonious Ponk]

Wow, yes, I'm in awe at our agreement - maybe we can have a pint sometime?

If you simply left out the "strong indication" we would have a complete agreement. Will ya, ah go on now, you will, you will, (in best Ms Doyle voice)

Hey...I agree with all of that too. When you get a pint, link me in online and I'll drink one with you.

Tim

 

[Phelonious Ponk]

I'm thirsty now, Tim, and yearning for Budweiser and Southern Comfort

You could go that way, Max. Or you could choose to drink actual beer and real whisky, both of which are available to you locally. I have great respect for the irish. Not exactly known for their cuisine, their beers and whiskies are exemplary. That, my friend, is an understanding of priorities. Budweiser? Watery swill. Southern Comfort? Cough syrup. Here in the states, give me an Arrogant Bastard Ale and a Woodford Reserve. Unless you're buying....

Tim

 

[jkeny]

You could go that way, Max. Or you could choose to drink actual beer and real whisky, both of which are available to you locally. I have great respect for the irish. Not exactly known for their cuisine, their beers and whiskies are exemplary. That, my friend, is an understanding of priorities. Budweiser? Watery swill. Southern Comfort? Cough syrup. Here in the states, give me an Arrogant Bastard Ale and a Woodford Reserve. Unless you're buying....

Tim

Hey, Tim, I completely agree with you - now that wasn't as difficult as I thought it might be - good beer & whiskey show that we know about life - Uisce beatha (the Irish name for Whiskey meaning water of life!)

 

[Phelonious Ponk]

Good subject matter. I may not believe in premium power cables, but I've got religion when it comes to whisky. Any idea what the best Irish whiskies are that make it across the pond?

Tim

 

[Audio_Karma]

I'm thirsty now, Tim, and yearning for Budweiser and Southern Comfort [/QUOTE

Budweiser when chased with red wine always does the trick

 

[jkeny]

Jameson & Bushmills are the best well known Irish Whiskeys. I don't like Paddy or Powers & can't remember if I tried Mideleton or Redbreast.
Jameson probably has the biggest range & 12 year old is probably the best but it really depends on what sort of taste you like? It's all preference
Unfortunately, I'm no expert but I know what I like - I don't like peaty whiskey so Irish is better than scotch as a lot of scotch has that peaty taste to it - tastes like plastic to me. However, there are also some lovely scotch whiskeys, Glanmorangie, Glenfiddich