Conclusive "Proof" that higher resolution audio sounds different

[Phelonious Ponk]

Some pertinent quotes from J_J

"ABX tests (or any kind of scientific test) should not be presented as personal challenges, either. They should be presented with as much support as possible given to those taking the test, including (but not limited to) training, feedback, comfort, appropriate equipment, audio material, and so on."

"And, yes, you have to be comfortable, relaxed, and extremely familiar with everything involved in order to get a sensitive result. One does not just jump into the test willy-nilly."

I don't know why the idea of pressure in a test is being rejected as an influence? Just because Amir was able to overcome it does not mean that it's therefore of no significance. Just because Max couldn't differentiate the two samples sighted & at a later stage could do so in an ABX doesn't show us anything about pressure - it could very well be that at the time Max first tried the sighted test as when no one on the PFM forum (his peers) had returned a positive result, he felt pressure to conform. Later when positive results were returned by some of his peers - who knows, probably not even Max, knows - motivations & pressures can remain hidden to the individual.

These particular anecdotes do not show any evidence that contradicts my statement that "psychological pressure is an influencer in such tests". Do you really feel that it doesn't have an influence?

It's not being rejected as a possible influence, but it clearly was not much of an influence in this example. I personally doubt that it is a particularly strong influence in most situations, certainly not enough to negate everything else, but that's just personal opinion, just like the opinions you quoted from JJ above. A point of view, with no supporting evidence presented.

Tim

 

[Phelonious Ponk]

From RECOMMENDATION ITU-R BS.1116-1

METHODS FOR THE SUBJECTIVE ASSESSMENT OF SMALL IMPAIRMENTS IN AUDIO SYSTEMS INCLUDING MULTICHANNEL SOUND SYSTEMS

It must be empirically and statistically shown that any failure to find differences among systems is not due to experimental insensitivity because of poor choices of audio material, or any other weak aspects of the experiment, before a “null” finding can be accepted as valid. I

Who is this scientist who insists on proof of a negative?

Tim

 

[amirm]

Who is this scientist who insists on proof of a negative?

Tim

They are simply asking for double and triple checking of the test if the outcome is negative.

 

[Phelonious Ponk]

They are simply asking for double and triple checking of the test if the outcome is negative.

Ok. That's not what it says, but ok.

Tim

 

[Don Hills]

Ultimately I knew this was fruitless but still I tried - just shows you how irrational human beings can be! ...

... You're position is full of holes, john, and in its relentless defense, you've been contradicting yourself for awhile now. it's way past time to give this one up. ...

... just shows you how irrational human beings can be.

 

[jkeny]

It's not being rejected as a possible influence, but it clearly was not much of an influence in this example. I personally doubt that it is a particularly strong influence in most situations, certainly not enough to negate everything else, but that's just personal opinion, just like the opinions you quoted from JJ above. A point of view, with no supporting evidence presented.

Tim

I see, Tim - you consider J_J's as just another personal opinion? OK. You are now equating the view of someone who has expertise & worked extensively in this field as having the same weight as your or my opinion? It's not like he's talking about economics - he's talking about his area of expertise!!

Even with this equivocation it still is a long way from your statements, yesterday:

Actually I've done the opposite of that John. I've acknowledged the "pressure" in Amir's methods, and noted that the pressure didn't return a negative result in this case, while casual listening has. Which is the opposite of what you're saying the great "pressure" (which could be easily avoided, by the way) of unsighted listening should cause. You're position is full of holes, john, and in its relentless defense, you've been contradicting yourself for awhile now. it's way past time to give this one up.

Tim

Methinks the man doth protest too much!

I see you noticed too, Don but you missed this one "Who is this scientist who insists on proof of a negative?"

 

[jkeny]

It's not being rejected as a possible influence, but it clearly was not much of an influence in this example. I personally doubt that it is a particularly strong influence in most situations, certainly not enough to negate everything else, but that's just personal opinion, just like the opinions you quoted from JJ above. A point of view, with no supporting evidence presented.

Tim

So now you are arguing over the degree of influence of pressure, not that it has no influence?
Previously you agreed with Max's list of the only things to look after for a "valid" test:
- knowledge
- volume level
But you would now add?
- pressure

Is that now the full list or is there more you want to add to ensure a "sensitive test" as J_J calls it (I use the word valid)

 

[jkeny]

Who is this scientist who insists on proof of a negative?

Tim

Tim, by your statement "Who is this scientist who insists on proof of a negative?" it's obvious you have never read the document you are criticising "Methods for the subjective assessment of small impairments in audio systems" Recommendation ITU-R BS.1116-2 (06/2014)


Please do so before digging yourself a bigger hole. Max should also do the same - although he will probably reject it as he has previously rejected (or forgotten) J_J's list of criteria for such tests.

(BTW, J_J had an input into drawing up the contents of this document but if it's considered just an opinion piece then I guess that's not so important?)

 

[jkeny]

BTW, & here's an angle you guys should have considered - long term listening may well indicate that these ABX positive results of small differences are not of much concern when one is listening normally. Now wouldn't that be a kicker, eh? (it's what I already said Ron Party was interested in answering).

I thought you would be all over that one, Tim & Max

 

[Whatmore]

What about the pressure that participants on drug trials feel?
Does that invalidate the results ?

 

[maxflinn]

So Tim and I are now digging holes, forgetting things, being irrational, etc?

I'm really not sure what to say on this thread anymore. I'll go walk my dogs and see if I return inspired

 

[Orb]

If the differences are large then they would be picked up in short term testing too, so what does long term testing add?

Tolerances-threshold to anomalies/traits, and a person's behaviour (in this context would be deviation of a defined personal baseline).
But this is not done using blind AB comparison in same way as discussing ABX, due to using defined metrics and variables that are monitored-captured-weighed.
Cheers
Orb

 

[Stereoeditor]

A great point, to which the answer seems dependent on preference and politics rather than absolute objectivity.

Why would a hobby dominated by subjectivists suddenly become so ridged in this one, small area? People are bending over backwards to find a reason why a comparative listening method -- let's not call it a test; the casual listening being held up as the benchmark certainly doesn't rise to that level -- which eliminates many opportunities for bias is no better than one that begs for them. It makes no logical sense. The position is not "valid."

You're missing the point. Just because a test is blind doesn't in itself mean that it is a valid test. If interfering variables haven't been accounted for -- see Recommendation ITU-R BS.1116-2 (06/2014) -- then it is a flawed test. Now you and others have argued that sighted listing is also flawed, so as I was saying, the choice then becomes between two flawed test protocols.

One, the poorly designed blind test has the propensity for throwing up false negatives, ie, there is a null result when a real audible difference exists. The other, sighted listening has the propensity for throwing up false positives, ie, the listeners report hearing a difference where none exists. Audiophiles seem to prefer the risk of false positives, skeptics the risk of false negatives. But those choices are, as I said, due to preference and politics rather than to an inherent superiority of one choice over the other.

John Atkinson
Editor, Stereophile

 

[maxflinn]

IMO, there is a far, far greater chance of getting false positives in a sighted test than false negatives in a non-sighted test.

Knowledge, and all the powerful biases that goes with it will see to that.

 

[Phelonious Ponk]

So now you are arguing over the degree of influence of pressure, not that it has no influence?
Previously you agreed with Max's list of the only things to look after for a "valid" test:
- knowledge
- volume level
But you would now add?
- pressure

Is that now the full list or is there more you want to add to ensure a "sensitive test" as J_J calls it (I use the word valid)

The list of possible influences on any test, in any endeavor, is longer than you, I, Max or your experts have considered. It is not only long, it is highly variable and unpredictable. If they all had the kind of invalidating influence you so desperately want this one to have, there would be no science.

The list of influences likely to have an impact strong enough to allow you to reduce blind listening to the same level as casual, long-term listening with full knowledge of what is being compared? That one is pretty short. This is getting boring, John. You're circling and dodging. Making minor changes of subject in lieu of answering questions. Your position here has long been gutted and I suspect you're only hanging on because you are so keen to broadly discredit a testing methodology the often delivers results that you don't like. I've already said this at least twice -- if pressure is such a big factor, release the pressure, John. Give yourself as much time as you like. Set up the switch, balance the levels, relax, have a glass of wine and compare for hours, days, weeks, months. Record the results, but they won't change, because with the knowledge removed, no matter how much time you give it, it won't be enough to talk yourself into hearing what you want to hear. But it'll be ok. I'm confident you'll come up with an excuse to "invalidate" the results.

Tim

 

[Phelonious Ponk]

Tim, by your statement "Who is this scientist who insists on proof of a negative?" it's obvious you have never read the document you are criticising "Methods for the subjective assessment of small impairments in audio systems" Recommendation ITU-R BS.1116-2 (06/2014)


Please do so before digging yourself a bigger hole. Max should also do the same - although he will probably reject it as he has previously rejected (or forgotten) J_J's list of criteria for such tests.

(BTW, J_J had an input into drawing up the contents of this document but if it's considered just an opinion piece then I guess that's not so important?)

My statement was a response to a statement, John, not an entire document. As quoted, it is a call to prove a negative. Is that not what the author meant? Is there a more complete context I'm missing? Expand the context; expand the quote. Read what you post before you put it up, and understand what it is saying when standing on its own. If this statement is representative of the document, I'm not interested in investing time in the rest of it. If it is not representative, I await clarification.

Tim

 

[Orb]

So anyone yet done the test (after passing) with noise shaped and TPDF regarding the downsampled-decimated comparison
Smiling because this is one aspect raised awhile ago but enthusiasms seems to be in other areas.

Just mentioning as was a potential avenue of interest going back to page 40-50 or so with Amir, and probably 10 pages ago with another.

Thanks
Orb

 

[thedudeabides]

I'm really not sure what to say on this thread anymore.

How about the dog chasing the tail.

 

[jkeny]

So anyone yet done the test (after passing) with noise shaped and TPDF regarding the downsampled-decimated comparison
Smiling because this is one aspect raised awhile ago but enthusiasms seems to be in other areas.

Just mentioning as was a potential avenue of interest going back to page 40-50 or so with Amir, and probably 10 pages ago with another.

Thanks
Orb

Yes, this is a more interesting avenue than the recent discussions - I too would be interested in the investigation of the audibility of different resamplers & different types of dithering.
I know Barry Diament (soundkeeper Recordings), states that all software resamplers, other than Alexey Lukin's 64-bit SRC, marketed by iZotope, audibly brightens and hardens the results

 

[maxflinn]

How about the dog chasing the tail.

That's as valid as anything else