Conclusive "Proof" that higher resolution audio sounds different

[amirm]

First set of results from Scott are available now. It is broken into people who had systems with ultrasonic reproduction (called HRA) and not. Here is the results of the former:

Among those who have HRA-capable systems, all six identified the high-res versions perfectly—there wasn't a single incorrect determination. Obviously, both results support my contention that high-res audio requires a high-res audio system to reliably discern the difference that HRA can make to the sound of a recording.

http://www.avsforum.com/forum/91-au...igh-resolution-audio-test-results-so-far.html

 

[Orb]

Arny, you do appreciate that those tones at that level can push many budget components into clipping/distortion?
Case in point look back to the AP measurement analysis JA mentioned and how the dbfs level of those provided tones was enough to cause distortion/clipping, lowering the tones to -10dbfs that is still high resolved this.
TBH it is not surprising, and one reason many using ultrasonic tones do incremental db sweeps so can correlate to level of distortion/stress/clipping (David Griesinger did this and so do other test procedures including one other I linked to much earlier in this thread).

You can see that with the 39+41kHz signal at 0dBFS (above), there are audio-band products visible as high as -50dBFS.
However, the oscilloscope reveals that the amplifier is starting to clip with this maximum-level signal. Reducing the level to -10dBFS (below), which is still above the level of the jangling keys in Arny Krueger's file, results in any audio-band products dropping to below -100dB and the higher-order products above the audio-band disappearing.

And lets put that into perspective; 2 different products critically close to USB pen size that are also integrated USB DAC-Pre-Amp for headphone.
If people have products that behave worse than these with ultrasonic tones at -10dbfs, then they really should change their products or check they are designed/spec to handle hirez; especially when one considers its use with ABX tests relating to this thread (that no-one it seemed had passed until recently anyway)

BTW JA using 19+20 at near 0dbfs to deliberately stress test and where applicable levels specific to each product, furthermore this is a defined standards test.
For me I just cannot see what else there is to discuss around IM, so going to focus back on other actual probable causes-considerations as all this is seriously distracting what has been done-posted to date even includes proving IM not the cause (in general terms of product performance unless using a very poor modern designed-engineered product, and we also have listeners proving their system did not have IM issues) and part of the challenge is the listening approach and potentially compounded by expected fail biases (as noticed by those taking test on AVSF who initially failed then tried a more methodical listening approach and passed).

Thanks
Orb

 

[amirm]

One very serious problem with your analysis is that you are apparently unknowingly comparing apples and tomatoes. Music is an incoherent signal and naturally shows up on a FFT as a large number of significantly lower data points.

Test signals are coherent signals and show up on a FFT as a small number of data points at far higher amplitude(s) when both signals contain the same amount of energy. Of course they don't have the same amount of energy in this situation for other reasons such as the vast difference in crest factors.

I suspect that my pure sine waves are actually too small to catch all monitoring systems that are causing audible distortion during these tests.

Too small? We are not talking about fishing where small fish can escape through the net Arny. You have ultrasonic tones that the system will see solidly. The system does not have "eyes" to look at the spectrum and say, "oh, those are too small so I am not going to care about them."

In the other thread, Mark talked about how his AVR shut down twice on your ultrasonic tones but did not on the key jingling. That demonstrates that the total energy of those spikes is way above the level of the track itself.

At high level, you are walking into an audio paradox. Neither you nor I can hear those ultrasonic tones. As such, we cannot use our ears to say how loud they are. Hence the paradox of discussing their loudness level. Instrumentation is our only choice for objective analysis but you are dismissing that tool with "trust me." If you don't like my analysis, please demonstrate using your own analysis what the actual levels are.

It appears that it is not unusual for headphone-based systems have fewer dynamic range reserves than loudspeaker-based systems.

Don't know where this comes from. As I have said, I have run your tests and during the ultrasonic tones playing I hear nothing. So no issue with my headphones.

 

[Ashley James]

I've sort of looked at this thread, decided that I can't add anything other than on the subject of ABX testing. I think I was first involved in them when I was with ATC in the eighties and since then, whenever we need to know if differences exist, especially subtle ones, we have found they're totally reliable. They always identify differences and which is best. Absolutely no doubt whatsoever or we wouldn't keep using them for ours and no one else's benefit.

They also prove to anyone who has proper experience of them, that lots of differences they think they can hear they can't and that protracted listening periods are not as reliable. Not surprisingly audiophiles don't like ABX or anything else that finds them out.

Another thing ABXs reveal is that younger people can hear differences that those over say thirty five can't. This is probably because by that age you've lost about 10dB of hearing sensitivity at 3kHz and above by then. This is the equivalent of half the volume on the bit of the spectrum that gives clarity or intelligibility. It may explain why thirty five is the age that many used to consider hi fi, no longer though because headphones have more or less killed it.

Most of our comparisons are between speakers where differences are way to hear and ABX difficult to apply, but for people who believe op-amps or DACs sound different, they're perfect.

 

[Orb]

Welcome
TBH Ashley ABX only reliable for subtle differences with trained listeners (can be informal but not as ideal), as Amir has shown where some started only started hear the differences (context this thread and AVSF) when they did a more methodical approach required for ABX; the approach has been outlined by a few of us.
I would put my money on a fail when anyone tries it more adhoc in terms of statistical significance pass by a listener with such subtle differences; which unfortunately (not their fault) is how many ABX are done by audio hobbyists.

Cheers
Orb

 

[Ashley James]

Orb

It's certainly true that if people don't know what to listen to, they won't necessarily be able to hear differences that are audible, but we've tested lots customers/visitors over the years and always get the right answers.

However if I was doing the same with a bunch of visitors to a hi if show, there probably would be a failure rate.

AB testing is infallible, people aren't and enthusiasts aren't usually capable of setting them up properly. It needs someone like us who may use it as part of the development process of our products.

 

[Phelonious Ponk]

Welcome
TBH Ashley ABX only reliable for subtle differences with trained listeners (can be informal but not as ideal), as Amir has shown where some started only started hear the differences (context this thread and AVSF) when they did a more methodical approach required for ABX; the approach has been outlined by a few of us.
I would put my money on a fail when anyone tries it more adhoc in terms of statistical significance pass by a listener with such subtle differences; which unfortunately (not their fault) is how many ABX are done by audio hobbyists.

Cheers
Orb

Orb, surely you're not serious? Amir's results have shown that ABX is reliable for subtle differences with trained listeners; they say absolutely nothing about ABX only being reliable under those circumstances. Given less subtle differences, ABX works successfully all the time.

Perhaps I misunderstood what you were trying to say.

Tim

 

[Orb]

Yeah Ashley, first time I was involved in ABX was relating to audio transmission many years ago, Amir has done a lot more ABX in controlled environments and much more recently than many IMO including me.

Yeah Tim misunderstanding me
I think I have explained it quite a few times now the specific methodical/analytical approach required and so has Amir
If you look back one forum member here tried this approach we mentioned earlier and passed albeit stating it was tough (sure is as it is not how we "normally" critically listen).
Seems this thread keeps going round circles so going to spend much less time in this thread tbh.
Amir I stand by my point from much ealier on; nothing is going to change
Well apart from the few who passed using said approach and now appreciating ABX test in a different way.

Cheers
Orb

 

[amirm]

Hi Orb. You are right that there are no wholesale view changes. But I think this is an inflection point. More and more people are now coming out with positive outcomes and taking listening tests seriously. No longer do they believe it is impossible to pass them and give up instantly.

 

[Phelonious Ponk]

Yeah Ashley, first time I was involved in ABX was relating to audio transmission many years ago, Amir has done a lot more ABX in controlled environments and much more recently than many IMO including me.

Yeah Tim misunderstanding me
I think I have explained it quite a few times now the specific methodical/analytical approach required and so has Amir
If you look back one forum member here tried this approach we mentioned earlier and passed albeit stating it was tough (sure is as it is not how we "normally" critically listen).
Seems this thread keeps going round circles so going to spend much less time in this thread tbh.
Amir I stand by my point from much ealier on; nothing is going to change
Well apart from the few who passed using said approach and now appreciating ABX test in a different way.

Cheers
Orb

I understand Amir's method, what I don't understand is this statement --

ABX only reliable for subtle differences with trained listeners (can be informal but not as ideal), as Amir has shown where some started only started hear the differences (context this thread and AVSF) when they did a more methodical approach required for ABX

Are you not saying that ABX is "only reliable for subtle differences with trained listeners?"

Tim

 

[JackD201]

Taking a more methodical approach reduces stress. Stress is the enemy of useful results. Unless stressful listening is the norm of course. LOL.

 

[Orb]

I understand Amir's method, what I don't understand is this statement --



Are you not saying that ABX is "only reliable for subtle differences with trained listeners?"

Tim

Tim,
sorry but getting a bit irritated-frustrated here now.
How many times have I explained this and yet you just focus on the last post-sentence rather whole context of all my posts in this thread?
This is why these threads never achieve much in the long run.
Please do not bother responding, I will start to become really frustrated as it seems I am wasting my time
If you understand what Amir has been saying and myself in MANY earlier posts, you will understand why some kind of formal or informal as given in posts but not the most ideal training (already explained several times in this thread and in the past in other threads and why I am starting to get raaaargh tbh ) is required for subtle differences with blind ABX; someone else managed to take this on board in this thread and then passed the current ABX test.
Leaving it at that.
Orb

 

[Ashley James]

Tim,
sorry but getting a bit irritated-frustrated here now.
How many times have I explained this and yet you just focus on the last post-sentence rather whole context of all my posts in this thread?
This is why these threads never achieve much in the long run.
Please do not bother responding, I will start to become really frustrated as it seems I am wasting my time
If you understand what Amir has been saying and myself in MANY earlier posts, you will understand why some kind of formal or informal as given in posts but not the most ideal training (already explained several times in this thread and in the past in other threads and why I am starting to get raaaargh tbh ) is required for subtle differences with blind ABX; someone else managed to take this on board in this thread and then passed the current ABX test.
Leaving it at that.
Orb

TBF Orb, that's not been my experience. Anyone can do an ABX test successfully because we're all expert listeners and all looking for the same thing. Clarity or intelligibility. I've invited all sorts of people to do them and with reliable results.

Rather tongue in cheek I suggested that audiophiles might be the ones who had the most problems with them because they'd be panicking lest they were shown up and also, because of the flowery language they use, they could be unsure of what to listen for. Being an audiophile doesn't bestow any special talents on anyone any more than being a car enthusiast makes you a good driver.

 

[Orb]

TBF Orb, that's not been my experience. Anyone can do an ABX test successfully because we're all expert listeners and all looking for the same thing. Clarity or intelligibility. I've invited all sorts of people to do them and with reliable results.

Rather tongue in cheek I suggested that audiophiles might be the ones who had the most problems with them because they'd be panicking lest they were shown up and also, because of the flowery language they use, they could be unsure of what to listen for. Being an audiophile doesn't bestow any special talents on anyone any more than being a car enthusiast makes you a good driver.

Try digital filters (these is rather broad as some will be easier than others depending upon variables involved), amps more similar than different,etc; we are talking subtle differences.
Please note I am not suggesting special talent; it is the approach-methodology-understanding to the listening that is critical for ABX, and plenty of practice-hard work (compared to normal listening)/formal-informal (such as the web) training.
In the past hobbyist sites doing ABX between amps/DACs/etc found no differences even for models that vary a lot in both price-spec-performance-design; just a simple example.

"Audiophiles"/hobbyists would fail or have in the past because they are confident sighted using a critical listening approach that does not necessarily work for ABX because of biases/high number of correct-consistent results required by listener/etc; do not isolate and identify a consistent audible trait at specific points in a short segment but rather approach it more from a "whole"/broader/adhoc approach with complete track or long segment without focusing on specific points.
Again oversimplifying it but Amir and myself went into more detail much earlier and thankfully those who originally failed redid the test with a different approach in line with what we both suggested and then passed.
Just to re-mention, source content can also be important as part of the scope-focus of any subtle ABX test and ideally understanding the hardware-solutions and their performance behaviour/traits; again oversimplifying but we are seriously digressing so I suggest if anyone has a problem maybe a new thread on critical listening and how it is done may be of interest; such as critical listening by musicians listening to other musicians, critically listening to music and its quality, critical listening applied to testing-training (usually depends upon scope-context of test).
Thanks
Orb

 

[Ashley James]

Try digital filters (these is rather broad as some will be easier than others depending upon variables involved), amps more similar than different,etc; we are talking subtle differences.
Please note I am not suggesting special talent; it is the approach-methodology-understanding to the listening that is critical for ABX, and plenty of practice-hard work (compared to normal listening)/formal-informal (such as the web) training.
In the past hobbyist sites doing ABX between amps/DACs/etc found no differences even for models that vary a lot in both price-spec-performance-design; just a simple example.

"Audiophiles"/hobbyists would fail or have in the past because they are confident sighted using a critical listening approach that does not necessarily work for ABX because of biases/high number of correct-consistent results required by listener/etc; do not isolate and identify a consistent audible trait at specific points in a short segment but rather approach it more from a "whole"/broader/adhoc approach with complete track or long segment without focusing on specific points.
Again oversimplifying it but Amir and myself went into more detail much earlier and thankfully those who originally failed redid the test with a different approach in line with what we both suggested and then passed.
Just to re-mention, source content can also be important as part of the scope-focus of any subtle ABX test and ideally understanding the hardware-solutions and their performance behaviour/traits; again oversimplifying but we are seriously digressing so I suggest if anyone has a problem maybe a new thread on critical listening and how it is done may be of interest; such as critical listening by musicians listening to other musicians, critically listening to music and its quality, critical listening applied to testing-training (usually depends upon scope-context of test).
Thanks
Orb

I'm not an engineer and my business partner is away right now, but surely digital filters are chips used with multi-bit DACs. Modern mostly Delta Sigma convertors use analogue filters and the chipset manufacturers tell you how to implement them. They give you a choice of implementations......

I think you're making all this much more complicated than it is. DACs are simple and self contained so there is no longer the scope for incompetents to get them wrong. Critical listening is a human skill, we all do it every day, all of us, not just audiophiles, so why suddenly do some need training. Hang on! I don't need that one answered.......

 

[jkeny]

I'm not an engineer and my business partner is away right now, but surely digital filters are chips used with multi-bit DACs. Modern mostly Delta Sigma convertors use analogue filters and the chipset manufacturers tell you how to implement them. They give you a choice of implementations......

You're not joking, are you? Not only did you stop looking at DACs some decades ago, but you left it when your understanding was, shall we say similar to your understanding of the standards necessary for valid blind tests.

 

[amirm]

I think you're making all this much more complicated than it is. DACs are simple and self contained so there is no longer the scope for incompetents to get them wrong. Critical listening is a human skill, we all do it every day, all of us, not just audiophiles, so why suddenly do some need training. Hang on! I don't need that one answered.......

I will answer anyway .

Ideally we would use real instruments and not humans in these evaluations. Instruments are unbiased and accurate. Humans are not. Since our instruments don't tell us what is audible, what is not in many cases, we are stuck using humans.

But just because someone has a pair of ears, it doesn't mean they make good instruments. Our goal as system designers is to find and fix flaws. If we can train someone to find these flaws faster, and more accurately, we shortcut the design processes immensely. Further, we are no longer exposed to people in the field finding the same problems.

In my case, prior to becoming trained, I was totally deaf to artifacts much higher than what we are talking about here. A few months of training and all of a sudden, I was able to hear an ant crawling on my speaker cable. Not really but it sounded good saying that. Seriously, it did become that simple. It was like someone took the blindfold off my eyes. I was seeing a world like I had never, ever seen before. I then became a strong asset for my team developing audio technology. They would test things themselves, then our test group but the final outcome would wait for me to do a listen. If I found a problem, they would go and fix it and not release the software.

As I have said countless times, there were people who beat me. And some of them had no training whatsoever. Because those people exist, then it is our job to seek and find differences that someone like me could hear.

 

[amirm]

I'm not an engineer and my business partner is away right now, but surely digital filters are chips used with multi-bit DACs. Modern mostly Delta Sigma convertors use analogue filters and the chipset manufacturers tell you how to implement them. They give you a choice of implementations......

The DAC silicon itself has indeed become superb. Alas, once someone puts them in a box, they often degrade the performance substantially. These are very sensitive circuits. While as you say anyone can take the reference design and make a box -- which is behind the avalanche of DACs being released -- there are countless ways to screw things up.

Here is a real example of a $400 DAC that my son bought:

Who designs a DAC chip with that huge distortion spike? No one, right? We can prove this further. That was using the S/PDIF. This is what happens when you use USB:

The jitter spike is going but then we have a bunch of low frequency jitter components. But wait, this is the exact same measurement but with a different media player driving the DAC over USB:

Where in the datasheet for the DAC would I find dependency on a media player??? Nowhere. These are system design problems where the digital side of the DAC is not isolated properly from the DAC. No reference design tells you how to solve these problems. They assume competent designers don't make mistakes like this.

 

[Ashley James]

Jk

The digital filter is in the chip, but there is an external analogue one.

Amirm

There maybe ways to screw DACs up and if it's going to happen it'll be small audiophile companies without properly qualified engineers that manage it. They always have. But not proper engineers at companies like Yamaha who are world leaders. Or AVI for that matter because Martin, my business partner is an engineer and has been working with DACs since the original 8 Bit Monolithics he used in military avionics way back.

Years ago, one highly regarded hi fi company got rave reviews for a bitstream convertor that stopped FM tuners working anything up to thirty yards away so badly was it radiating! At the same time a surprising number of amplifier manufacturers had terrible problems coping with the crap from the Philips digital filter that accompanied the TDA 1541. However, times have changed, the technology has improved so if you buy an Airport Express from Apple, you'll be fine.

I've nearly run my race here, but I'd repeat that properly constructed AB, ABX or even swift comparisons are the only really way to be sure and they really are infallible. When differences are as subtle as some are suggesting here, then you need people in their early twenties to hear them. You could argue that if thirty plus's can't hear theses tiny differences, they shouldn't worry about them, especially not if they can hear enormous differences between speakers and also read amount the huge amounts of distortion they produce.

I'm afraid I just don't get all this microscopic dissection of minutae, to me it's like scraping the paint off a Ferrari to save weight and go faster. If people are genuinely interested in improving the sound quality they hear, they need to stop wasting time on forums arguing, work out the differences that are audible and start with the huge ones first. But that's just me.

 

[Ashley James]

Amir

Just so you know, we produced our first bitstream DAC as part of an Integrated amp in the eighties, in 1990 we introduced an audiophile one use the Philips Crown S1 TDA1541 chip with a discrete current to voltage convertor and tuned amplifier to receive the clock and avoid jitter. It used Philips digital filter.

A year later we produced our first CD player, we wrote all our own software, used a CDM9 mechanism and twin Burr Brown 20 Bit multi Bit DACs with their digital filter and op amps. The chip designer even popped over from Japan and visited us to compare it with his specially made reference. There was no audible difference and that puzzled him.

Subsequently we may six or seven more CD players before stopping production of them and all separates in 2006, when we shifted to active speakers with built in, remote controlled, multi input preamplifier and DAC. In this instance a Wolfson WM8741 with SRC and Crystal receiver chip. It works well and better suits modern homes.

Sorry for the sales pitch, but to show that we do know what we're talking about and it'd be more if my business partner wasn't having a holiday. I'm more into transducers.