Double blind testing and stress

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
I have read every paper I have found from AES and elsewhere and yet to see anything that comes close to telling me what jitter is audible.

Not the AES, but these guys did a bunch of tests and you can download the PDF for free:

Detection Threshold For Jitter

From their conclusion:

Kaoru Ashihara et al said:
The results indicate that the threshold for random jitter on program materials is several hundreds ns for well-trained listeners under their preferable listening conditions. The threshold values seem to be sufficiently larger than the jitter actually observed in various consumer products.

The key is the last part, that even inexpensive consumer gear has jitter low enough to not matter. This makes sense, since jitter can also be expressed as artifacts some number of dB below the music. The graph below from Ken Pohlmann's book Principles of Digital Audio shows a typical level. If artifacts are 100 dB or more below the music, it's a stretch IMO to think that could ever be audible while the music plays.

What I don't understand is why people seem to conclude that whatever shortcomings they may hear in their gear, it's due to jitter. Why jitter? Doesn't it make sense that room acoustic problems - which are at least four or five orders of magnitude larger! - is a far more likely culprit for lack of detail, smearing, and imaging etc?

--Ethan

____________________________________________​

 

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
What I am not a fan of is subjecting everyday audiophiles to trick tests (sorry Ethan :))

LOL, most of the trick tests I've done were not meant to be that! The last one I did, I screwed up an export of a mix, and two of the three files were bit-identical. Yet, people reported obvious night and day differences between those two files. So I turned lemons into lemonade and made the best of my mistake by pointing that out.

--Ethan
 

Gregadd

WBF Founding Member
Apr 20, 2010
10,517
1,774
1,850
Metro DC
What I don't understand is why people seem to conclude that whatever shortcomings they may hear in their gear, it's due to jitter. Why jitter? Doesn't it make sense that room acoustic problems - which are at least four or five orders of magnitude larger! - is a far more likely culprit for lack of detail and imaging etc?

Becasue we hate digtal.:D

I don't want to debate the issue but one reason might be because the problem is only present when playback is in the digital medium.
 

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
Becasue we hate digtal.:D

At least you admit your prejudice. :rolleyes:

the problem is only present when playback is in the digital medium.

What problem? When done properly, digital audio is transparent. A few years ago a company that sells high-end A/D/A converters did some listening tests with professional engineers, and even after ten generations nobody was able to pick out the 10th copy of a copy from the source. I did something similar with 20 generations through a $25 SoundBlaster sound card. I won't say the 20th copy is indistinguishable from the original source, but it's a thousand times closer than a 20th generation of analog tape or LP recording. Files here if you or anyone else cares:

http://www.ethanwiner.com/aes/

--Ethan
 

Gregadd

WBF Founding Member
Apr 20, 2010
10,517
1,774
1,850
Metro DC
This problem:

What I don't understand is why people seem to conclude that whatever shortcomings they may hear in their gear, it's due to jitter. Why jitter? Doesn't it make sense that room acoustic problems - which are at least four or five orders of magnitude larger! - is a far more likely culprit for lack of detail, smearing, and imaging etc?

--Ethan
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Not the AES, but these guys did a bunch of tests and you can download the PDF for free:
I have read that paper as it is routinely posted on forums as proof that jitter is inaudible. As much as I admire anyone trying to characterize distortions like this, their approach is exceedingly faulty and the reason I made the comment that I did.

From their conclusion: The results indicate that the threshold for random jitter on program materials is several hundreds ns for well-trained listeners under their preferable listening conditions. The threshold values seem to be sufficiently larger than the jitter actually observed in various consumer products.
I am not going rehash all that they have done wrong as I have written about it many times. But just a few quick points for now:

1. "Random" jitter or for that matter, any other other random distortion turns out to be far more benign than program correlated jitter. If your projector has a fan, I bet you don't hear it most of the time. But imagine if the fan came on every time the screen became bright. I bet you will hear it much more often then. The brain tends to filter non-variant things as a way for you to focus on what is important.

Jitter induced in Audio equipment usually is not random. It could be induced by the power supply ripple voltage, it could be the oscillator for the FL display, it could be the video circuit bleeding into it, etc. Or it could track the source signal in certain ways. All of this generates much more objectionable distortion.

2. They let people play the content through their PCs. Did they profile those PCs first to see what level of jitter they had in them to start? No.

3. You have to pick material that is revealing of the distortion you are trying to find. I can take classical music, encode it at 64kbps and have 99% of the people say it is the same as music. I can then encode some guitar music and have everyone be able to tell the difference no mater how deaf. It almost seems like people assume there is no difference so they don't pay attention to things like this. As I have said before, lack of difference found is more of a fault of the technique than people's perception if the science says there is a difference (and the science definitely does as confirmed in the very article).

On this front, jitter destroys low order bits and can be especially audible where the ear is most sensitive. Knowing this and the fact that the higher the frequency, the more the problem, would lead you to special set of content you would want to use, rather than whatever happens to be on a CD on your desk.

4. As I have noted before, there will never be any proof that jitter is inaudible for the simple reason that jitter has infinite characteristics. It can be caused by one or more modulating signals each of which can have arbitrary frequency, magnitude and waveforms. So no study, no matter how comprehensive, can ever provide sufficient data that jitter cannot be heard in other circumstances.

The key is the last part, that even inexpensive consumer gear has jitter low enough to not matter.
A conclusion which is not supported in any way by their research since they did not show that consumer gear has random jitter. Or even what jitter they have regardless of type.

This makes sense, since jitter can also be expressed as artifacts some number of dB below the music. The graph below from Ken Pohlmann's book Principles of Digital Audio shows a typical level. If artifacts are 100 dB or more below the music, it's a stretch IMO to think that could ever be audible while the music plays.
Is the "music" always at 0db? It has to be for your statement to be true.

What I don't understand is why people seem to conclude that whatever shortcomings they may hear in their gear, it's due to jitter. Why jitter? Doesn't it make sense that room acoustic problems - which are at least four or five orders of magnitude larger! - is a far more likely culprit for lack of detail, smearing, and imaging etc?

--Ethan
In my case it is easy as i use electorstatic headphones to evaluate DACs so I am not impacted by the room.

That aside, it is common fallacy to assume that room distortion masks other types of distortion. Room distortion unless the walls are rattling, is linear in nature. Distortion such as jitter is not. Here is an example that may be easier to understand. If i take a 10 watt amp and drive it at its max volume with 20% THD, would I not hear the distortion due to room acoustics? Of course I would. The non-linearities would be quite obvious despite anything the room might do to the sound.

To be fair, it is true that people talk about jitter all the time as if they really understand it when in reality, it is a complex topic few of us have a good grasp of it, even though we are in the industry :).

Keep in mind that we can show mathematically what must be true for a digital system to reproduce its source. If we wan to regard digital perfect in this ability, we can't in the next breath dismiss the math and say that 8 bits of effective resolution at 20Khz is good enough all of a sudden because subjectively and anecdotally, we think that is OK. One has to either believe in the science or not :).
 

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
I am not going rehash all that they have done wrong as I have written about it many times.

That's fine. I haven't read everything you've written criticizing their method, and you may not have read all of my writings on this either. So I'll hit just the high points:

1. "Random" jitter or for that matter, any other other random distortion turns out to be far more benign than program correlated jitter.

Sure, but if a noise or buzz or program-related IM-like artifact is 120 dB below the music, nobody can possibly hear it regardless of whether the sound is correlated or not. This is very different from your example of a fan that's clearly loud enough to be audible, but eventually becomes ignored due to being on constantly. One section in my AES Audio Myths video plays music while a very nasty noise is turned on and off at varying levels. Even though the noise goes on and off, and you can see on the play meter where it starts and stops, the noise becomes inaudible at levels far louder than what is typical for jitter.

The only way to know at what level jitter becomes audible is to use a device that lets you vary the amount of jitter while you listen. As far as I know, no such device exists. At least the paper I cited tried to use that approach, even if they had to simulate the jitter using software. For me the bottom line is that jitter artifacts are exceedingly soft. As far as I know, it has never been shown that any human can hear any sound when it's 100+ dB below another sound and thus masked by the louder sound.

2. They let people play the content through their PCs. Did they profile those PCs first to see what level of jitter they had in them to start? No.

On the surface that seems reasonable, but the jitter of any computer sound card is well below the thresholds they measured so that shouldn't matter. Let's use another example. Suppose you wanted to test at what level wow or flutter become audible. This is easy to do on a PC using an FM modulator type audio plug-in, and I've done this a few times. So let's say that when the modulation rate is 10 Hz, the pitch has to vary by 2 cents before anyone can hear it. The PC's own time-based errors are vastly smaller, so I'd be confident accepting the 2 cents result. Does this make sense?

jitter destroys low order bits and can be especially audible where the ear is most sensitive.

Again, how do you know this for sure? Have you done a test where you increased the jitter amount starting from zero and going higher?

there will never be any proof that jitter is inaudible for the simple reason that jitter has infinite characteristics. It can be caused by one or more modulating signals each of which can have arbitrary frequency, magnitude and waveforms. So no study, no matter how comprehensive, can ever provide sufficient data that jitter cannot be heard in other circumstances.

It's not possible to give negative proof. Common sense (as I see it) says that nothing 100+ dB below the music could possibly be audible no matter what the makeup of that sound. So this is why I feel the burden of proof is on those who claim that typical amounts of jitter are ever audible.

Is the "music" always at 0db? It has to be for your statement to be true.

No, but one example (starting at 33:28) in the video linked above has gentle classical music with an average level of -30, peaking around -15 when it gets to the loudest spot. By the time the nasty noise is reduced to -70, it's then only 40 dB softer than the music at -30. Even cranking the volume way up it's difficult or impossible to hear the noise. And that's only a 40 dB difference! If jitter is at -100 it's still 30 dB softer, and only 70 dB below the music, and it's still inaudible. Further, and this is key, the amount of jitter is relative to the music volume. So as the music gets softer the jitter is lower too.

--Ethan
 
Last edited:

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing