Conclusive "Proof" that higher resolution audio sounds different

[jkeny]

Even if it's all at -95db or less?

Amir, shows -72dB so I don't know why you keep using this figure?

So your primary objective, as an audiophile DAC designer, is to produce a product with better performance than the average DAC in the average AVR, right? A single measurement, like Amir presented here, doesn't give you nearly enough information, I get that. So in order to develop a better than average product, you must measure a defined signal at at many points up and down the audible range, yes?

Tim

Tim, it's not about how I design or measure DACs -it's a discussion using logic - which you seem to be deflecting from?

 

[Phelonious Ponk]

Amir, shows -72dB so I don't know why you keep using this figure?



Tim, it's not about how I design or measure DACs -it's a discussion using logic - which you seem to be deflecting from?

I'm using the -95db figure because that's the figure we started our discussion with and...sorry for the thread drift but I thought we'd moved on - maybe it's just me - to how designers measure the performance of DACs to decide where to work on improvements and know when they've actually made them. You've indicated that measuring the jitter created by a sine wave at one frequency is not useful. I'm very interested in what is useful and how you use it.

If I start a new thread just for that discussion, will you answer the question there?

Tim

 

[jkeny]

I'm using the -95db figure because that's the figure we started our discussion with

No WE didn't - Sasuly brought over/continued this audibility of HDMI jitter rubbish from AVS forum & he quoted -95dB (quickly corrected by Amir

and...sorry for the thread drift but I thought we'd moved on - maybe it's just me - to how designers measure the performance of DACs to decide where to work on improvements and know when they've actually made them. You've indicated that measuring the jitter created by a sine wave at one frequency is not useful. I'm very interested in what is useful and how you use it.

No, WE had not moved on to "how designers measure the performance of DACS - you might have tried but failed to introduce this canard

If I start a new thread just for that discussion, will you answer the question there?

Tim

No, Tim, I won't - it's simply divisive & s**t stirring in my opinion. You & Ashley James, Sasuly have been given answers to your contention that "these audible differences are insignificant". Yes, to many they are - be happy with that answer & number yourself among the many. Accept that others are more discerning, discriminating & afford them this. Are you being forced to listen to ArnyK's or Scott's files to check for differences? No. So let others discuss what they consider important without you trying to impose your value system on them.

 

[Phelonious Ponk]

No WE didn't - Sasuly brought over/continued this audibility of HDMI jitter rubbish from AVS forum & he quoted -95dB (quickly corrected by AmirNo, WE had not moved on to "how designers measure the performance of DACS - you might have tried but failed to introduce this canard


No, Tim, I won't - it's simply divisive & s**t stirring in my opinion. You & Ashley James, Sasuly have been given answers to your contention that "these audible differences are insignificant". Yes, to many they are - be happy with that answer & number yourself among the many. Accept that others are more discerning, discriminating & afford them this. Are you being forced to listen to ArnyK's or Scott's files to check for differences? No. So let others discuss what they consider important without you trying to impose your value system on them.

OK.

I accepted a long time ago that some can hear subtle differences that I cannot John. Again, sorry for the thread drift, but I moved on to question how, if measuring for jitter using music is not practical, the audible spectrum is tested so you know where the problems are and when you've solved them. As a DAC designer/supplier (and the guy who told me music won't work), you seemed like an appropriate guy to ask. I offered to move the discussion to a separate thread to end the drift and not confuse this question with the rest of the issues in this thread. Not at all sure how, being told what won't work, asking a follow-up question about what will constitutes stirring the (*&*, but OK. You won't answer the question. All you had to do was say so.

Tim

 

[FrantzM]

No WE didn't - Sasuly brought over/continued this audibility of HDMI jitter rubbish from AVS forum & he quoted -95dB (quickly corrected by AmirNo, WE had not moved on to "how designers measure the performance of DACS - you might have tried but failed to introduce this canard


No, Tim, I won't - it's simply divisive & s**t stirring in my opinion. You & Ashley James, Sasuly have been given answers to your contention that "these audible differences are insignificant". Yes, to many they are - be happy with that answer & number yourself among the many. Accept that others are more discerning, discriminating & afford them this. Are you being forced to listen to ArnyK's or Scott's files to check for differences? No. So let others discuss what they consider important without you trying to impose your value system on them.

SO what are we discussing about?

 

[Phelonious Ponk]

SO what are we discussing about?

See my post above, Frantz. I drifted. My bad.

Tim

 

[jkeny]

OK.
I accepted a long time ago that some can hear subtle differences that I cannot John.
Tim

You mean, you can't or you're not interested or you know your equipment is not capable or......? You have stated that you don't intend to try the tests so how do you know if these differences are subtle to you or not?

 

[Phelonious Ponk]

You mean, you can't or you're not interested or you know your equipment is not capable or......? You have stated that you don't intend to try the tests so how do you know if these differences are subtle to you or not?

How I know what's subtle varies from case to case. In the case of Amir's recent discovery of his ability to hear differences between RB and hi-res, the fact that these differences are extremely subtle is obvious by what was required to hear them and the small number of people who have, so far, been able to do so. And no, if there's any chance that training myself to hear these differences will mean that, once I know how to recognize them, I will then hear them even when listening for pleasure, no, I don't want to learn how to hear them. The overwhelming majority of music in the world is RB, I'll not deliberately diminish the experience of listening to it. YMMV.

Tim

 

[jkeny]

How I know what's subtle varies from case to case. In the case of Amir's recent discovery of his ability to hear differences between RB and hi-res, the fact that these differences are extremely subtle is obvious by what was required to hear them and the small number of people who have, so far, been able to do so. And no, if there's any chance that training myself to hear these differences will mean that, once I know how to recognize them, I will then hear them even when listening for pleasure, no, I don't want to learn how to hear them. The overwhelming majority of music in the world is RB, I'll not deliberately diminish the experience of listening to it. YMMV.

Tim

Ah, I see irrational fear.
It's the usual irrationality that is posted by some & goes like this - " I pity those who can hear these differences - their enjoyment of music must be destroyed"

 

[Orb]

I'm using the -95db figure because that's the figure we started our discussion with and...sorry for the thread drift but I thought we'd moved on - maybe it's just me - to how designers measure the performance of DACs to decide where to work on improvements and know when they've actually made them. You've indicated that measuring the jitter created by a sine wave at one frequency is not useful. I'm very interested in what is useful and how you use it.

If I start a new thread just for that discussion, will you answer the question there?

Tim

Try treating correlated/deterministic jitter as "phase noise and amplitude noise" rather than like normal noise-distortion.
This means when you reference that sideband say at -90db, you must include all associated correlated jitter component sidebands to the carrier signal (11.025khz when using J-test) to derive the true value for jitter IF interpreting from a spectrum analyser.
Also it is interesting some products in the past would have minimal jitter at 48khz/24bit and then fail at 96khz/24bit with many correlated side bands; Denon/Pioneer also suffered this in the past.
Anyway as I mentioned most very modern day DACs are consistent with having great low jitter measurements (specifically talking about correlated/deterministic) under 130psec for 16bit and much lower for 24bit, but consider the caveats mentioned before regarding interface used and the added challenge it may pose for some including mass producing manufacturers.
This was more of an issue going back 5-10 years for traditional interfaces (HDMI may still be challenging), but you will find some of the well known digital engineers did their own testing to evaluate correlated jitter closer to what happens in the real world (Guido Tent was one, Bruno Putzeys another,etc) and it is not possible to compare its behaviour and audibility or more important effect to that of noise/distortion/IM/etc.

So what Amir shows does have credible merits, especially when considering the context and how to interpret jitter; unfortunately Stereophile I think does not show the Total correlated jitter values for the J-test, but tbh as mentioned before the issue these days is more specific.
Thanks
Orb

 

[Phelonious Ponk]

Try treating correlated/deterministic jitter as "phase noise and amplitude noise" rather than like normal noise-distortion.
This means when you reference that sideband say at -90db, you must include all associated correlated jitter component sidebands to the carrier signal (11.025khz when using J-test) to derive the true value for jitter IF interpreting from a spectrum analyser.
Also it is interesting some products in the past would have minimal jitter at 48khz/24bit and then fail at 96khz/24bit with many correlated side bands; Denon/Pioneer also suffered this in the past.
Anyway as I mentioned most very modern day DACs are consistent with having great low jitter measurements (specifically talking about correlated/deterministic) under 130psec for 16bit and much lower for 24bit, but consider the caveats mentioned before regarding interface used and the added challenge it may pose for some including mass producing manufacturers.
This was more of an issue going back 5-10 years for traditional interfaces (HDMI may still be challenging), but you will find some of the well known digital engineers did their own testing to evaluate correlated jitter closer to what happens in the real world (Guido Tent was one, Bruno Putzeys another,etc) and it is not possible to compare its behaviour and audibility or more important effect to that of noise/distortion/IM/etc.

So what Amir shows does have credible merits, especially when considering the context and how to interpret jitter; unfortunately Stereophile I think does not show the Total correlated jitter values for the J-test, but tbh as mentioned before the issue these days is more specific.
Thanks
Orb

Thank you, Orb.

Tim

 

[Phelonious Ponk]

Ah, I see irrational fear.
It's the usual irrationality that is posted by some & goes like this - " I pity those who can hear these differences - their enjoyment of music must be destroyed"

I wouldn't go so far as to say their enjoyment of music is destroyed, John. But I think there's a pretty good chance that they will be hearing irritants or limitations when listening. I'd just as soon not join them in that, given I have a hard drive with several thousand songs ripped from RBCD on it. And YMMV, but I think that's quite rational.

Tim

 

[amirm]

Gentlemen, please dial down the emotion/personal remarks.

 

[maxflinn]

So has anyone ever successfully differentiated two (or more) DACs in a blind test before?

 

[Orb]

So has anyone ever successfully differentiated two (or more) DACs in a blind test before?

Complex test to do; critically needs to consider digital filter design (same or different and how such as fast-slow rolloff/linear-minimum phase/stopband rejection) that then has a potential influence on some of the following that also needs consideration; source music native sampling rate,CD or hires (and checked to ensure the hirez music "transparency" is not messed up due to upsampling/not DSD-PCM with ultrasonic swell/other anomalies).
Another potential "depends" can be transmission used such as USB/SPDIF/etc: some products still have maximum sampling rate/bit depth, possible interraction with one interface spec.
And then consider setting up in a way that a trained listener can use their methodology-approach to attempt the blind ABX; this adds further complexity in terms of level matching, being able for listener to isolate and focus on "identified" trait within the segment consistently between the test products,etc.

I appreciate I am taking this a bit far and too brief with the last two sentences, but there will be those that will argue the test is invalid and only anecdotal at best if the variables are not understood,measured and shown whether the interraction is part of the test scope and its context, furthermore this is compounded real world components connected do have a fair few "it depends" compared to a more limited scope-context controlled environment.
And to top it off, how does one decide which settings to use on the DAC when it comes to filters/modes as their is no true ideal filter due to very subtle compromises between time and frequency domain and stopband rejection tbh; some will say this deliberately skews results for positive outcome while deliberately forcing choice to be the "traditional" brickwall filter that is embedded to chips could also be argued of also skewing any result.
Just worth noting that the provided on chip filters are limited in terms of hardware resources/computational power/etc and therefore not necessarily as ideal as dedicated hardware that provides greater flexibility as we see these days (along with greater expertise in developing filters on commercial DACs); this subject has been covered by both audio manufacturers and chip manufacturers.
Thanks
Orb

 

[maxflinn]

Orb, thanks for the reply. Personally I don't see the issue as being so complex.

I think it's simply a case of finding two DACs that, when played in a certain system using certain music, some people are certain sound different.

Then, using all the same equipment/room/people, just remove the knowledge of which DAC is in use and ask them to determine which is which, or which is X, under these conditions.

IMO, a failure to reliably do so, having being able to do so sighted, eludes to only one conclusion.

 

[Orb]

Orb, thanks for the reply. Personally I don't see the issue as being so complex.

I think it's simply a case of finding two DACs that, when played in a certain system using certain music, some people are certain sound different.

Then, using all the same equipment/room/people, just remove the knowledge of which DAC is in use and ask them to determine which is which, or which is X, under these conditions.

IMO, a failure to reliably do so, having being able to do so sighted, eludes to only one conclusion.

But then that ignores DACs that may have different filter designs, so you could end up with a false conclusion that DACs cannot be differentiated because they both use same integrated digital functions.
Same could be said using hirez if one does not ensure it is a true native hirez with right sampling right without meddling or other issues.

As an example you could take 2 Class A amps with same topology-spec and say amps with a trained listener cannot be differentiated even with ideal source content-music; but then this ignores Class D and Class A in a test that may provide differences in such a test.
Appreciate this is a very simple crude example just emphasise importance of this aspect and scope-understanding required for DAC ABX.
Thanks
Orb

 

[jkeny]

Max,
Did we not already cover this on the Winer's thread & some agreement reached?

The agreed answer to the question "IMO, a failure to reliably do so, having being able to do so sighted, eludes to only one conclusion." with which you partially agreed being "Conclusion - that sightedness brings a bias to what we hear. It doesn't prove that there is actually no difference"

Please don't start the same discussion on another thread as if the first one hadn't happened.

Just to reiterate (i posted this for you on the other thread) - here's JJs brief list of requirements for how to properly set up a blind test

A short, and undoubtedly insufficient list (since I'm writing this off the cuff) would be:

1) listener training
2) quiet, single-listener situation, with equipment, acoustics, etc of appropriate quality
3) negative and positive controls, and stimulus repetition for evaluation of consistency
4) perfect time alignment and level alignment (either of those off by much at all will absolutely result in a positive result)
5) feedback during training and after each individual trial
6) consistent A and B stimuli, which the subject is permitted to know, and who can refresh their recollection at any time. This is also an element that can easily cause any test to be positive by mistake.
7) transientless, quiet switching between the signals, with extremely low latency. Switch transients can cause either lower sensitivity or unblind a test, depending on how they arise.
8) the ability to loop the test material under user control
9) of course the setup must be double-blind, ordering must be varied, etc. All standard test confusion issues must be satisfied.


That's just few, that's not even close to a full set, but just that much shows how it isn't easy to run a good test.

Edit: Oh, & BTW - a quick search for "blind DAC tests" reveals this Head-fi thread in which some people used ABX to test for differences in recordings of 6 different DAC's playing back the same song (not a great test, I know)

 

[jkeny]

Reading through that Head-Fi thread is another interesting study in psychology - just how we all are prone to accepting evidence that favours our position & rejecting evidence that doesn't (this seems to be a bigger bias than sightedness ). The thread starts off by the o/p in a very bullish way

"Objectivists claim that all DACs that aren't severely broken sound the same.
Audiophiles claim that different DACs sound different, more or less noticeably. I've been told, for instance, that a Clip+ and an iPod Classic sound completely different.
......
You may try to identify a particular piece of gear, or which sample is the original, or rank the samples in descending order of preference (the one you like the most first, the one you like the least, last), or ABX any two samples… whatever you're most comfortable with.

P.S.: I don't expect that many people will venture a guess or two, but at least the opportunity is here.

And ends up after some positive ABX results are posted saying

"Knowing what to listen for gave you an advantage that you wouldn't have had, if you hadn't analyzed the files first. And for the same reason, you didn't choose which files to ABX by accident. Not exactly 100% blind, is it.

Interesting, isn't it, as this is repeated on nearly every forum where "objectivists" post. I even started a thread on PFM called "sorting out evidence-based from faith-based" & even though the thread's objective was in the title people showed their "belief system" by their very posts trying to argue that the ABX results were "faked", "needed proctoring", "needed repeating", "result of IMD", "result of bad downsampling", etc.

Thanks, Max for making me do this search - I wouldn't have read that Head-fi thread otherwise

 

[FrantzM]

Reading through that Head-Fi thread is another interesting study in psychology - just how we all are prone to accepting evidence that favours our position & rejecting evidence that doesn't (this seems to be a bigger bias than sightedness ). The thread starts off by the o/p in a very bullish way

And ends up after some positive ABX results are posted saying

Interesting, isn't it, as this is repeated on nearly every forum where "objectivists" post. I even started a thread on PFM called "sorting out evidence-based from faith-based" & even though the thread's objective was in the title people showed their "belief system" by their very posts trying to argue that the ABX results were "faked", "needed proctoring", "needed repeating", "result of IMD", "result of bad downsampling", etc.

Thanks, Max for making me do this search - I wouldn't have read that Head-fi thread otherwise

John

I would think you are aware this works both ways. re your insistence on trying to find flaws in ABX while conveniently sidestepping the more serious, numerous and known flaws of sighted testings ..

For the record. I don't believe that all DACs sound the same. Having compared a BADA DAC to a multitude of others .. The results are clear enough for me , The BADA is a a consistent winner to my ears.