Conclusive "Proof" that higher resolution audio sounds different

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Hello everyone.

Arguments about value of sampling rate > 44.1 Khz and bit depth > 16 bits has filled thousands upon thousands of pages on various audio forums. There has been very little formal testing but what there is, such as Meyer and Moran point to no audible difference (yes, the test was flawed but this thread obsoletes what they did anyway).

Personally I must have wasted hundreds of hours talking about the need for transparency in our audio equipment and I don't know how to get there if it is limited to 16 bits/44.1K. I have provided the math, research, and tons of explanation to no avail. Folks demand "listening tests" when they run out of technical arguments to make.

As it turns out there was major advancement in this discussion in the last week or so. I was engaged on AVS Forum in yet another dog fight on this front with folks arguing that even 16-bit/44.1 Khz is "too much" let alone having something at higher spec. On this front, Arny Krueger claimed that he had "brickwall" filtered audio down to 16 Khz (32 K sampling) and no one could tell it apart from the original. And that he had run this test blind and double blind. Here is the original claim:

Yes. Take the best audio system you can find. Take the best recordings you can find - recordings that sound great and also have significant content > 20 KHz, even > 35 KHz. Switch a 16 KHz brick wall filter in and out of the signal path. Nobody notices nuttin'

People say: "But I can hear pure tones at 21 KHz". Probably true. But that is without content at other frequencies masking it. Music is composed of many tones at many different frequencies. Masking in the upward direction frequency-wise is very strong.

Turns out this is an experiment that I had also run years ago but to my surprise, I found clear audible difference. Yet, I could not hear the frequencies that I had taken out!

The discussion went on for a bit and then Arny post a recording of keys jingling that he had recorded and then sampled down to 44.1 Khz and 32 Khz. He then resampled them back out to the same 24/96 Khz of the original track. So all files had the same sample rate but the other two versions had their frequency response chopped down to 22 and 16 Khz respectively. Here is the link to the post and the files in dropbox:

Thanks for the encouragement.

Here is the link for my idea of killer 24/96 .wav files files for using ABX to hear potential audible differences due to differences in bandpass and resolution:

https://www.dropbox.com/sh/l86f7oc7cdsb0xr/AAB4feU8z3Xww14Mrta1zs3Ca

Be careful with these files - the full bandpass version has so much ultrasonic energy that it can possibly take out your tweeters and you won't even think that it is playing very loud. They are also highly diagnostic for high frequency IM.

The source is my keychain ca. Y2K being jangled about a foot away from a DPA 4006 measurement mic, as precisely on axis as I could get it in a low reverberent room. This particular test has been well known among studio engineers for about half a century, and it has the advantage that for testing microphones and speakers, the only special test equipment has the right price!

Despite my success in hearing differences I expected this comparison to be a tough one. After all, I would think that Arny would first listen to these files and once he was assured there was no audible difference, he would post them. Such was not the case.

The challenges on AVS always demanded use of Foobar2000 ABX test and while I don't like that tool, I used it anyway. Here are my results. The format is N correct votes/Trial number M and (probability of chance). A score of say 10/11 says that 10 correct votes were made out of 11 tries (i.e. one mistake).

32 Khz versus 96 Khz
=================================
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/09 06:10:07

File A: C:\Users\Amir\Music\Arnys Filter Test\keys jangling band resolution limited 3216 2496.wav
File B: C:\Users\Amir\Music\Arnys Filter Test\keys jangling full band 2496.wav

06:10:07 : Test started.
06:10:38 : 01/01 50.0%
06:10:50 : 02/02 25.0%
06:11:07 : 03/03 12.5%
06:11:23 : 04/04 6.3%
06:11:36 : 05/05 3.1%
06:12:00 : 06/06 1.6%
06:12:14 : 07/07 0.8%
06:12:26 : 08/08 0.4%
06:12:38 : 09/09 0.2%
06:12:49 : 10/10 0.1%
06:13:00 : 11/11 0.0%
06:13:23 : 12/12 0.0%
06:13:42 : 13/13 0.0%
06:13:48 : Test finished.

----------
Total: 13/13 (0.0%)


44.1 versus 96 Khz
---------------------------------

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/09 06:32:02

File A: C:\Users\Amir\Music\Arnys Filter Test\keys jangling band resolution limited 4416 2496.wav
File B: C:\Users\Amir\Music\Arnys Filter Test\keys jangling full band 2496.wav

06:32:02 : Test started.
06:33:07 : 01/01 50.0%
06:33:17 : 02/02 25.0%
06:33:24 : 03/03 12.5%
06:33:36 : 04/04 6.3%
06:33:47 : 05/05 3.1%
06:33:58 : 06/06 1.6%
06:34:12 : 07/07 0.8%
06:34:15 : Test finished.

----------
Total: 7/7 (0.8%)

As you see, this is a perfect score when it comes to finding the difference between the original 96 Khz and down sampling to 44.1 and 32 Khz.


I found the 32 Khz/16 K max frequency comparison to be trivial. The difference was easily heard. Indeed another poster on AVS repeated the same perfect score I had. The second one with limiting the sampling to 44.1 Khz was much harder but I was still able to identify the right segment and as you see, achieve perfect score of 7/7. I could have kept going but hit the wrong button in foobar and it stopped there. The other poster however was not able to differentiate the files.

Please download the files and see how you do. My testing was all on my HP Zbook 14 laptop and etymotic ER4p headphone. The DAC was the internal sound of the laptop so nothing special there either. Foobar was in default mode using Windows Audio stack so additional dither was being applied. The sampling rate in Windows Sound control panel was set to 24/96 Khz to match the files.

In Part 2 I will provide the results of a second listening test that cements these findings.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
The above discussion was triggered by a formal effort on AVS Forum to compare CD sampling rate and bit depth to "High res music" (24-bit/96 Khz). You can read more about it here including a link to the files: http://www.avsforum.com/forum/91-au...-aix-high-resolution-audio-test-take-2-a.html

Yet again, I did not think I could hear the differences. But the results proved otherwise :). Here are my double blind ABX tests of the files:

Thank you Scott! Much appreciated the effort you have put on this project Scott. For the first time I feel that the forum is moving forward toward better understanding of this topic.

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 18:50:44

File A: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_B2.wav

18:50:44 : Test started.
18:51:25 : 00/01 100.0%
18:51:38 : 01/02 75.0%
18:51:47 : 02/03 50.0%
18:51:55 : 03/04 31.3%
18:52:05 : 04/05 18.8%
18:52:21 : 05/06 10.9%
18:52:32 : 06/07 6.3%
18:52:43 : 07/08 3.5%
18:52:59 : 08/09 2.0%
18:53:10 : 09/10 1.1%
18:53:19 : 10/11 0.6%
18:53:23 : Test finished.

----------
Total: 10/11 (0.6%)

The third track was pretty easy. First segment picked was quite revealing:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 21:01:16

File A: C:\Users\Amir\Music\AIX AVS Test files\Just_My_Imagination_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\Just_My_Imagination_B2.wav

21:01:16 : Test started.
21:02:11 : 01/01 50.0%
21:02:20 : 02/02 25.0%
21:02:28 : 03/03 12.5%
21:02:38 : 04/04 6.3%
21:02:47 : 05/05 3.1%
21:02:56 : 06/06 1.6%
21:03:06 : 07/07 0.8%
21:03:16 : 08/08 0.4%
21:03:26 : 09/09 0.2%
21:03:45 : 10/10 0.1%
21:03:54 : 11/11 0.0%
21:04:11 : 12/12 0.0%
21:04:24 : Test finished.

----------
Total: 12/12 (0.0%)


:)

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/11 06:18:47

File A: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_B2.wav

06:18:47 : Test started.
06:19:38 : 00/01 100.0%
06:20:15 : 00/02 100.0%
06:20:47 : 01/03 87.5%
06:21:01 : 01/04 93.8%
06:21:20 : 02/05 81.3%
06:21:32 : 03/06 65.6%
06:21:48 : 04/07 50.0%
06:22:01 : 04/08 63.7%
06:22:15 : 05/09 50.0%
06:22:24 : 05/10 62.3%
06:23:15 : 06/11 50.0% <---- difference found reliably. Note the 100% correct votes from here on.
06:23:27 : 07/12 38.7%
06:23:36 : 08/13 29.1%
06:23:49 : 09/14 21.2%
06:24:02 : 10/15 15.1%
06:24:10 : 11/16 10.5%
06:24:20 : 12/17 7.2%
06:24:27 : 13/18 4.8%
06:24:35 : 14/19 3.2%
06:24:40 : 15/20 2.1%
06:24:46 : 16/21 1.3%
06:24:56 : 17/22 0.8%
06:25:04 : 18/23 0.5%
06:25:13 : 19/24 0.3%
06:25:25 : 20/25 0.2%
06:25:32 : 21/26 0.1%
06:25:38 : 22/27 0.1%
06:25:45 : 23/28 0.0%
06:25:51 : 24/29 0.0%
06:25:58 : 25/30 0.0%

06:26:24 : Test finished.

----------
Total: 25/30 (0.0%)


So we now have 3 out of 3 positive detection of differences in Scott's clips.

Summarizing, I managed to consistently tell all three files apart from their downsampled 44.1 Khz/16 bit versions.

Track 1 and 3 were relatively easy to tell apart but both took fair amount of effort to find the critical segments were the difference could be heard. The middle track #2 was essentially not distinguishable but I managed to finally find the difference.

Again, remember all of these tests are double blind ABX, exactly the kind demanded to be provided for "proof." Levels were matched. So there is really no wiggle room here. I was able to find all the differences.

So far no one has repeated my success here. Anecdotal reports say that people could not distinguish the files from each other.

Please give this test a try. Again, I used my laptop for this so all that is required is keen sense of hearing to find the differences. While you can perform spectral analysis to find out which file is which, please don't do that. I did not. And importantly, if you do perform that, please do NOT announce which file is which so that we don't pollute the vote of others taking the test.

My results for the first time ever caused all the critiques of high res audio to go quiet. No other vocal critique would report their results. My conclusion is that they could not hear the difference.

I am not yet ready to speculate why I am able to hear the differences. Only that I have the skills to find and focus on difficult segments and know what to look for. My hearing is shot above 12 Khz so I don't think any of this due to me actually "hearing" ultrasonics. My headphone response also doesn't go that high.

In summary, this is in my opinion a watershed event. For a long time I have been saying that the typical hobbyist double blind tests are faulty in the way they don't include trained listeners, nor do people know what distortions they are looking for. The results of my testing completely invalidates the claim that "there are no golden ears." And that we all hear the same. When it comes to non-linear distortions, we absolutely do not hear the same as these results conclusively show.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Also note the speed with which I was able to identify the right tracks. On the average it was just 10 seconds which included listening to both "X" and "Y" tracks, voting, and the telling foobar to go to the next comparison. To do that requires remembering exactly what the difference is and having no need to listen to A or B reference samples again.
 

RogerD

VIP/Donor
May 23, 2010
3,734
318
565
BiggestLittleCity
I can hear the difference with one ear closed:D
 

c1ferrari

Member Sponsor & WBF Founding Member
May 15, 2010
2,162
51
1,770
So, what did Arny have to say? ;)
Seriously, very cool, Amir :D
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
So, what did Arny have to say? ;)
He said he is going to be busy with life and family and may not be posting as much. And just like that, he stopped posting on AVS Forum! Prior to that he was in practically every audio thread and forum. Maybe he will come back but this has been a dramatic shift. I think he just realized much of what he has believed may not be true.

Seriously, very cool, Amir :D
Thanks :). I must say, I completely surprised myself. I thought with deteriorating hearing that I would not be able to do this but I did.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
I agree 16/44 is not good enough and why isn't the AES getting about establishing a realistic consumer standard that is high enough to cause for a reach in implementing current design and able to stop the detractors from dissing the standard.
The problem is that the unwritten rule at AES is that you cannot say anything without a double blind test. And current published double blind tests point to no audible difference in just about everything.

I am actually thinking of proposing a paper and get some of this data on record. It is remarkable how our leading engineering/research organization has done so little to bring light to this issue.

Amir, are you blessed or cursed with your ability to distinguish digital artifacts ;) and do you think you could do as well listening to say you favorite rock and roll song or jazz song in circumstances as you did above?
With audio compression I always could. In this situation I don't know yet. As usual, masking is the enemy here so if there is a lot going on for the whole track, it is hard to hear the small differences.
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Many, congrats, Amir
Now the $64,000 question - what are you zoning in on in your listening & what's your theory about what you are hearing in 44.1K that you are not hearing in 96K - it's certainly not >20Khz frequencies? IMD, perchance?
I'm surprised nobody has asked this question already
 

Orb

New Member
Sep 8, 2010
3,010
2
0
Amir,
Kudos to still get involved in such threads on AVSF.
I still am incredulous that many there still ignore acknowledged experts in the field that show technically why one needs roughly 20 bits and 96khz (although technically this is still a little bit more than is required in context of ideal filters-etc).
It is as if anyone who is involved in audio is not counted as an expert; hence Bob Stuart, Bruno Putzeys, John Westlake, Robert Watts (one if not the very first to develop commercial FPGA DAC),etc etc etc and the list is biiig.
And then there are those with or working in studios on top of this.
My sigh can be heard across the Atlantic :)
Wonder if I can bring myself to read the thread at AVSF....

Anyway gratz on your hearing-perception-cognitive skills and importantly taking time to do that test :)
Im sure usually they do not use good native hirez files (or at least verify it is not upsampled from a lower rez file and without digital artifacts-issues) any of their tests; HifiNews (both Paul Miller and Keith Howard) have shown there are a lot of iffy hirez downloads out there.

Cheers
Orb
 

MylesBAstor

Well-Known Member
Apr 20, 2010
11,236
81
1,725
New York City
Amir,
Kudos to still get involved in such threads on AVSF.
I still am incredulous that many there still ignore acknowledged experts in the field that show technically why one needs roughly 20 bits and 96khz (although technically this is still a little bit more than is required in context of ideal filters-etc).
It is as if anyone who is involved in audio is not counted as an expert; hence Bob Stuart, Bruno Putzeys, John Westlake, Robert Watts (one if not the very first to develop commercial FPGA DAC),etc etc etc and the list is biiig.
And then there are those with or working in studios on top of this.
My sigh can be heard across the Atlantic :)
Wonder if I can bring myself to read the thread at AVSF....

Anyway gratz on your hearing-perception-cognitive skills and importantly taking time to do that test :)
Im sure usually they do not use good native hirez files (or at least verify it is not upsampled from a lower rez file and without digital artifacts-issues) any of their tests; HifiNews (both Paul Miller and Keith Howard) have shown there are a lot of iffy hirez downloads out there.

Cheers
Orb

You're the lucky coin Amir. Now where's James Randi when you need him? Tell the difference between cables and make some serious coin. :)
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Anybody else here try the test?
I did on my laptop - up to 16/32 is easy but 16/44 not so
I'll try on my DAC when I get over my hatred of jangling keys.
BTW, I would love to hear what Amir has picked up on before I give a link to where this same test has been done before & the conclusions :)
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
You're the lucky coin Amir. Now where's James Randi when you need him? Tell the difference between cables and make some serious coin. :)
Ah, I had forgotten about making a million dollars that way :). Had he ever put forward a challenge against high-res audio?
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Anybody else here try the test?
I did on my laptop - up to 16/32 is easy but 16/44 not so
I'll try on my DAC when I get over my hatred of jangling keys.
BTW, I would love to hear what Amir has picked up on before I give a link to where this same test has been done before & the conclusions :)
Let's not dig into the details for now and let folks run the test.

I will of course at some point elaborate more on what I heard.
 

MylesBAstor

Well-Known Member
Apr 20, 2010
11,236
81
1,725
New York City
Ah, I had forgotten about making a million dollars that way :). Had he ever put forward a challenge against high-res audio?

No but I'm sure we can get Randi stirred up enough he would. We'll keep you in our back pocket and just like happens in those TV "pool" games, have you as our ringer! :) As long as you're willing to split the pot!
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
Let's not dig into the details for now and let folks run the test.

I will of course at some point elaborate more on what I heard.

OK, you deserve to bask in some glory for a while :)
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
No but I'm sure we can get Randi stirred up enough he would. We'll keep you in our back pocket and just like happens in those TV "pool" games, have you as our ringer! :) As long as you're willing to split the pot!
No worries. We will set up a fund for anyone who has suffered in the hands of Randi and crew and hire a lawyer to decide how much they deserve! :D
 

TBone

New Member
Nov 15, 2012
1,237
1
0
Arguments about value of sampling rate > 44.1 Khz and bit depth > 16 bits has filled thousands upon thousands of pages on various audio forums.

Amirm ... in the grand scheme of things, these test only deflect the real issues concerning the problems of digital reproduction. In my books, although it's obvious higher# may offer potentially superior SQ, the entire sampling rate/bits debate is chalk-full of audiophile hypocrisy.

Digital done right or wrong, isn't simply a sampling-rate/bit depth issue, so many examples exist ...

tb1
 

c1ferrari

Member Sponsor & WBF Founding Member
May 15, 2010
2,162
51
1,770
I am actually thinking of proposing a paper and get some of this data on record.
Splendid proposition, Amir :D

It is remarkable how our leading engineering/research organization has done so little to bring light to this issue.
I would sincerely expect, if the science is grounded in basic research, then scientists would be receptive to conflicting data/analysis/conclusion. :cool:
 

Orb

New Member
Sep 8, 2010
3,010
2
0
Amirm ... in the grand scheme of things, these test only deflect the real issues concerning the problems of digital reproduction. In my books, although it's obvious higher# may offer potentially superior SQ, the entire sampling rate/bits debate is chalk-full of audiophile hypocrisy.

Digital done right or wrong, isn't simply a sampling-rate/bit depth issue, so many examples exist ...

tb1
You are right, but sample rate and bit depth IS an issue in the context of ideal filter and studio processes involved, including the application of how often/when TPDF dither is used as an example (remember music is from a big chain-process when considering mic-recording-mixing-mastering-etc).

Cheers
Orb
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Amirm ... in the grand scheme of things, these test only deflect the real issues concerning the problems of digital reproduction.
Those issues are dismissed out of hand though. My tests pull that back in the realm of possibility and now, expecting a different explanation than "it can't happen."
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing