Another Foobar ABX positive result on High res

I just came across this interesting 2013 Gearslutz thread - Foobar 2000 ABX Test - Redbook vs 192/24 showing positive ABX results & wanted to post it here as a sort of repository for ABX testing

What I found interesting in it were the descriptions of the listener, Ultmusicsnob (who seems to have recording & mixing experience) & his honest reporting of the ABX process.

What come across is the difficulty & dedication required to find the "difference" even though he already heard a difference in sighted listening.
Things like:

"Keeping my attention focused for a proper aural listening posture is brutal. It is VERY easy to drift into listening for frequency domains--which is usually the most productive approach when recording and mixing. Instead I try to focus on depth of the soundstage, the sound picture I think I can hear. The more 3D it seems, the better. "

Click to expand...

"Caveats--Program material is crucial. Anything that did not pass through the air on the way to the recording material, like ITB synth tracks, I'm completely unable to detect; only live acoustic sources give me anything to work with. So for lots of published material, sample rates really don't matter--and they surely don't matter to me for that material. However, this result is also strong support for a claim that I'm detecting a phenomenon of pure sample rate/word length difference, and not just incidental coloration induced by processing. The latter should be detectable on all program material with sufficient freq content.
Also, these differences ARE small, and hard to detect. I did note that I was able to speed up my decision process as time went on, but only gradually. It's a difference that's analogous to the difference between a picture just barely out of focus, and one that's sharp focused throughout--a holistic impression. For casual purposes, a picture that focused "enough" will do--in Marketing, that's 'satisficing'. But of course I always want more."

Click to expand...

"I tried to listen for soundstage depth and accurate detail. It took a lot of training repetitions, and remains a holistic impression, not any single feature I can easily point to. It seems to me that the 192 files have the aural analogue of better focus. To train, I would try to hear *precisely* where in front of me particular sound features were located, in two dimensions: left-to-right, and closer-to-further away--the foobar tool would then allow me to match up which two were easier to precisely locate. I know it muddies the waters, but I also had a very holistic impression of sound (uhhhhhh) 'texture'??--in which the 192 file was smoother/silkier/richer. The 192 is easier on the ears (just slightly) over time; with good sound reproduction through quality headphones (DT 770) through quality interface (RME Babyface) I can listen for quite a while without ear fatigue, even on material that would normally be considered pretty harsh (capsule's 'Starry Sky', for example), and which *does* wear me out over time when heard via Redbook audio."

Click to expand...

"Practice improves performance. To reach 99.8% statistical reliability, and to do so more quickly (this new one was done in about 1/3 the time required for the trials listed above in the thread), I mainly have to train my concentration.

It is *very* easy to get off on a tangent, listening for a certain brightness or darkness, for the timbre balance in one part, several parts, or all--this immediately introduces errors, even though this type of listening is much more likely to be what I am and need to be doing when recording and mixing a new track.

Once I am able to repeatedly focus just on spatial focus/accuracy--4 times in a row, for X & Y, and A & B--then I can hit the target. Get lazy even one time, miss the target."

Click to expand...

It took me a **lot** of training. I listened for a dozen wrong things before I settled on the aspects below.

I try to visualize the point source of every single instrument in the mix--that's why I picked a complex mix for this trial. I pinpoint precisely where each instrument is, and especially its distance from the listener. Problem is, both versions already have *some* spatial depth and placement, it's only a matter of deciding which one is deeper, and more precise. I've tried making determinations off of a particular part, like a guitar vamp or hi-hat pattern, but can't get above about 2/3 correct that way.
The better approach is just to ask myself which version is easier to precisely visualize, as a holistic judgment of all the pieces together. Equally effective, or rather equally contributing to the choice, is asking which version holistically gives me a sense of a physically larger soundstage, especially in the dimension extending directly away from me--thus the idea of listening to reverb characteristics.
Having to listen to four playbacks (A/B, X/Y, for one choice) gives rise to the problem of desensitization. Neurons naturally give decreased response to repetitions, so I've found I can target my answer more easily if I pause 5-10 seconds between an A/B (or an X/Y). Otherwise, A/B is always easier than X/Y.
I have rather junky monitors, KRK Rokit 6's, so I'm kind of surprised I can get a result out of them. To get down into low single digits I shifted to my headphones pushed by a nice Schiit Asgard2 amp, which I just acquired--if your headphones are good, I'd recommend using them for the testing. This is more for isolation than anything else.

Click to expand...

Summary to this point:

The effect being heard in foobar ABX testing here has been robustly detected:
1) In popular music (2 different songs), dense textures
2) In classical music, more transparent textures
3) With default iZotope SRC values (32 filter, 175 alias suppression)
4) With "highest quality" SRC values (150 filter, 200 alias suppression)
5) Using high quality headphones through high quality headphone amp out of high quality interface (RME Babyface)
6) Using cheap earphone plugs driven by generic motherboard audio chips from garden variety [Dell] desktop PC

NOTE: The above results contribute ***nothing*** to the established science of 192/24 versus 44.1/16 and human hearing.

They do establish that whatever I am detecting in these file pairs is robust to a wide range of conditions: equipment, program content, and SRC algorithm settings.

They do establish that when I say, "It sounds better to me", I am reporting a provable reality, not a placebo effect.

Click to expand...

I did a search for him - with a name like ultmusicsnob I figured it might be easy & found his 2013 posts on Head-Fi, this time on jitter & audibility where he again posts Foobar ABX positive results showing his discernment of the test files used.

Again some interesting comments on ABX testing

Well, I will insist on the caveat that *all* ABX testing is of a sort pretty much wholly removed from how one would normally listen to music. The protocol can't be completed otherwise. The *only* time I ever listened like that in real life was when I was trying to hear John Lennon say "I bury Paul" at the end of "Strawberry Fields". :biggrin: That said,

Yes, my first research question is usually "Is differentiation possible at all???", and so I use the tools available to hunt for the differences.
It was particularly difficult in this case, as I don't have a good sense of what problematic jitter *ought* to sound like, and it matters what testers are listening for.

Since I can pick out a difference on one snare hit, a further refinement would be to listen more 'casually', and see if the drum set sounds different throughout.

I'm guessing that the added jitter track would have been indistinguishable for this particular music, but it's faintly conceivable that interested listeners could learn to hear the difference without the procedures I described.

Click to expand...

And later in the thread as he learns what to listen for:

Of course, this time I had the enormous benefit of knowing what to listen for (snare), that I could apply across the track. No hunting around for artifacts.

I hear the jittered drum set as *slightly* less precise. The jittered version thuds instead of snapping tightly. It's a *tiny* difference, but you can find it anyplace that the cymbal-crazy drummer isn't covering it up. If I had to reverse-engineer from the treatment, I would hypothesize that the applied jittering is smearing/obscuring the attacks slightly.

Click to expand...

amirm said:

Thanks John. Had no idea others online had generated such results. Would have saved me fair amount of work in the argumentative threads . His comments mirror my experience pretty well.

So one of the problems seems to be the fallacy of "I have not seen x, y and z." As in, "I have not seen anyone pass Foobar blind test of this and that." They haven't seen it because they have not looked!

Click to expand...

I had no idea that such results already existed, either.

I find it worthwhile collecting these examples - particularly the descriptions of the ABX "procedure" because the challenge usually made is "if you can hear it sighted then why can't you hear it blind" - therefore it must be imaginary, delusional, etc. Fact of the matter is that ABX testing is extremely difficult for subtle differences.

Here's one of his last posts on the Head-Fi Jitter audibility thread & I'm not sure what level of jitter is in the generated file that he is able to differentiate but Nick_Charles says this of his Foobar result copied below:

Can somebody else please attempt to DBT these tracks, I've tried and failed miserably (predictably) , what UMS can do should not be possible (based on the differences between the tracks at different frequencies) I do not know if something else is at play here, in any case we really need others to try this.

Click to expand...

Replication.

foo_abx 1.3.4 report
foobar2000 v1.2.8
2013/09/27 16:48:19
File A: C:\Users\jhughes1\Downloads\path30n.flac
File B: C:\Users\jhughes1\Downloads\path30jr.flac
16:48:19 : Test started.
16:48:45 : 01/01 50.0%
16:48:53 : 02/02 25.0%
16:49:06 : 03/03 12.5%
16:49:49 : 04/04 6.3%
16:50:17 : 05/05 3.1%
16:50:34 : 05/06 10.9%
16:51:03 : 06/07 6.3%
16:52:00 : 07/08 3.5%
16:52:25 : 08/09 2.0%
16:53:09 : 09/10 1.1%
16:53:20 : Test finished.
----------
Total: 9/10 (1.1%)

Earlier I was listening for quiet chords near the end. These last two reports, I used quiet chords before the big sound at ~21 secs.

Hate to muddy the waters, and I appreciate all the responses above, but this report here was generated on the "cheap" earphones with motherboard chipset DAC.

I have to think it's not about fidelity of the equipment, it's figuring out what to listen for. Listening for jitter is *unlike* other ABX comparisons I've done before. If it helps, I try to imagine the sharpest focus of sound in terms of how "narrow" I can hear the piano attack, as though it were a spatial measure. The narrower attack is 'n'. It is difficult because I'm continually tempted to chase mirages of differences in other details. If I stick to "focus" and "narrow" I get a result.

Click to expand...

I may have referenced this before but on PinkFishMedia Forum 4f listening sessions were run over the course of a year or so. These were hosted by Vital who was a sceptic that DACs sounded different (as he had not heard any differences in the past) but was open-minded enough to wonder why others reported such differences. In other words a proper sceptic - not an ideological, ascerbic, "it's all your delusion" type. At these gatherings volumes were matched & DACs were listened to sighted & blind by a number of people - probably 20 -30 in all over the 4 sessions. In the first 2 sessions no differences were heard, sighted or blind, by all present. In the 3rd session I think some people said they heard differences but I think they couldn't do so blind. A 4th session was organised to definitively put this to bed.

Here's what Vital said after he had reported the results of the session in which differences were heard sighted & blind - I asked him this

Hi Vital
Glad it went well & you seem to have gained some further insights.
One thing - did you not already have both the Sonus & Leema in your home? Just wondering how come you never compared them before this?

Click to expand...

John, yes, and all my blind testing came up with 'null' results.

Andrew helped quite a lot in drawing my attention to the differences to look out for, which once noted became more obvious to me, and when we did a few simple blind ABs, first him, then me, found it relatively easy to pick out the Sonos vs. the Quiet PC into the Pyxis DAC.

I'm now of the opinion that blind tests are part of the issue (and this has been an evolving feeling over the last year), and that DAC differences are still of relatively minor significance over a certain level.

As stated above, the Mirus even sighted was, to me, only a slight improvement* over my Pyxis. If I'm honest, if I had money to burn I'd listen to it some more, and if initial impressions remained, likely buy it, but it's not worth the investment to me at this time. Put more simply, I can't afford it, and it's not enough of an improvement for me to hanker after it!

*or am I telling myself this because I can't afford it?!

Anyway, I now believe that DACs don't all sound the same. They sound more the same than speakers do, but that's not the point of the four DBOs. It's been a long journey, and to 99.99% of the population would be seen as a waste of time and effort. For me, it's been fun, and perhaps most importantly I've made some new friends through it, who I know I'll continue to meet up with to share music (and little HiFi nonsense I'm sure!).

Click to expand...

My interpretation of this is that continued null results from blind testing led him to the conclusion that there were no differences & de-motivated him from hearing any differences, sighted or blind. Only when "trained" in what the differences were did he hear it.

It's the audio equivalent of change blindness

BTW, here's a little experiment & something fun to listen to - try it but don't say anything about it until others have also tried it
https://youtu.be/zGKADgFCoeU

I will pm the website details to anyone who PMs me - later I will post them on this thread

Yes but it is such a simplistic & believable mantra "If you hear a difference sighted then why can't you do so blind"
Unfortunately, explaining the "why" isn't simple because perception isn't a simple black & white affair.
That's why I believe that posting listener's description of doing ABX tests is a way of exposing the lie of this mantra

The video/audio track I linked to above shows that we are as blind to obvious auditory changes as we are to visual changes when attention is focussed elsewhere.
This audio is the auditory equivalent of the well known visual blindness video in which a gorilla moves through the video scene - instead of "change blindness" we have "Inattentional Deafness"
From the study only about 30% of participants, focussed on the women's conversation, noticed the gorilla in the audio scene.
https://futurscientist.wordpress.co...ed-inattentional-deafness-for-dynamic-events/

Some further reading can be done here "Auditory Attention"
The summary concludes:


This, to me, seems to be the great mistake in blind testing - unless a difference can be consciously & specifically identified before doing a blind test then it is meaningless to continue to a blind test. But I see many examples of people saying that in sighted listening they could hear differences (without being specific) but this disappeared when they did a blind test & concluding that there is therefore no audible difference.

There is no point in hoping that some difference will suddenly make itself known during a blind test!