Sampling 101

Uh, Ethan said he uploaded, yes? Ethan normally links to his site, but ran a test case for me -- thanks Ethan! That matches what I see -- my higher-res pix, even though <20k, get transformed into something with lower quality...

Ethan, the program I am using for this can export as bitmaps or Windows Metafiles. I have been copying and pasting from the program into e.g. Paint (or one of several other programs, but they all seem to do about the same thing). A screen shot, while great for REW, RPlusD, and some other programs, does not work well for this one (for complicated reasons I don't fully recall).

Amir, I think another site I help with allows ~80k (it's some rather strange limit) uploads, and allows links to pix of any size (natch). Last time I had big pictures, a friend hosted them for me, but that's not a long-term solution. For now, I think I will try to make my small files work by tweaking the text size and such rather than setting up a picture account somewhere and hoping it doesn't "poof" sometime I least expect it. The sysadmin is a college prof and probably not around today but I can ping him next week to see if he's around and see what he suggests. I have a vague memory of there being size and compression defaults and limits for uploads he could tweak, but web/BBS programming ain't my thing...

I thank you all very much for all the help! - Don

p.s. I tried uploading a PNG and it converted it to a JPG; I also tried a bitmap (256-color BMP) but the image was 600 x 480 and I got an error saying max size for that type was 620 x 280 and it could not convert it. Since I suspect it would also end up as a JPEG I did not go further. My students in college many years ago gave me a plaque with the definition of a computer I (over)used: "computer (n). A mindless, inarticulate piece of machinery that hates your guts." Another saying they immortalized for me was "I hate it when the computer insists on doing what I tell it and not what I want!"

Ah, playing with the definition of insanity, Les?

I posted the aliasing info in another thread -- seemed easier that way. I pasted the figure as an inline JPEG and it seemed to do OK this time around. I hope! - Don

Thank you!

The simple answer to your last question is "we can't". The most you can sample is something along the edge, you cannot capture a signal with higher frequency content than Nyquist and hope to get it back (see the aliasing thread)...

For example, it takes about five harmonics to make a square wave approach what we think of when we hear (or read about) one, so above 4 kHz or so it's almost impossible to capture a square wave that looks really square out of a 44 kS/s system. It is important to remember that, while the ADC cannot acquire anything above 22.05 kHz, a DAC can produce higher frequencies since it's output is after the sampling process. An ideal DAC clocked at >40 kS/s can produce a perfect 20 kHz square wave, though the rest of the system may have trouble with such high harmonic frequencies, and most likely the DAC's output is filtered to prevent harsh sound and tweeter failures.*

And yes, the signals are band limited (filtered) before the ADC. Otherwise, we'd get strange (aliased -- see the other thread) signals out that could really wreck the sound. Thus, the DAC in a music system never sees a 20 kHz square wave (unless a test signal or digitally-generated music).

The good news is that a "fast" attack in audio is not nearly as fast as people might think, and for instance that "wallop"from a bass drum comes mostly from relatively low-frequency energy rather than a super-fast edge. That is, it's fast for a drum, but is likely well under our 4 kHz limit (there are other, higher-frequency components). The other thing to recall is that the power series for a square wave decreases fairly rapidly. A rectangular pulse train goes as (1/2 + 2/pi * cosW - 2/3pi * cos3W...) where W = 2*pi*t/T, t = the time and T the period, so that initial edge's higher harmonics are smaller and fall off fairly quickly. Other waveforms roll off even more quickly; for example, a full-wave rectified sine wave (think of a series of rounded "humps" one after the other) has a power series with amplitude terms of (1/2, 1/3, 1/15, 1/35, etc.) Since in the real world we cannot produce infinite-bandwidth signals, it is not quite so bad when we can't reproduce them in our sound systems, either.

Here's fodder for the anti-digital crowd: there's actually higher frequency content on many old LPs than on any CD. Of course, most of us can't hear it, so it doesn't really matter...

A little deep, but HTH - Don

* It is those very fast DAC output edges that give a lot of components fits, however, particularly when the DAC drives an op-amp that badly slew-limits, spewing distortion across the audio band. Some of the early op-amp's outputs would actually invert when overdriven badly; not a pleasant thing!

No, that's the error as a result of sampling and quantizing the signal. Note the error picture looks the same if we are talking about an ADC or a DAC. You raise an interesting point, because some systems actually do that, manipulating the input signal, sampled signal, and error signal to refine the results. You must have access to all those signals for that to work, naturally. In general, at the user level (and especially for audio), the orginal recording and the reconstruction process are quite far apart in time and space so there's no original reference signal to compare with and compensate the error. The error just "is"...

No, but I think I may have confused you (me, everybody). There are two considerations here, recording and playback. I was talking about playback, but let me try again and see if I can be more clear.

To use the error signal to compensate the ADC (or DAC), we need to know what the original signal is for reference. Otherwise, we have nothing to compare the error to, and no way to correct it. The input signal may be e.g. a recording of a band. The ADC captures it, it is recorded to a CD, then sometime later (and someplace else) we play it back on our stereo. At that point, we do not have the original signal (the live band), so we do not know how the DAC's output compares to that original live signal. We can't compensate what we don't know. That's where I was going in my turbid post above, but now we can take it a little further...

During the recording process, the signal is sampled and quantized to whatever the ADC can do, to the best of its ability. We can (and do) run calibration signals through an ADC first and compare them to the (known) input test signals to characterize its error. Then, digital techniques can be used to improve the ADC's performance. This is not uncommon, and is often part of final test for an ADC. So, we end up with the best ADC possible for a given resolution (and price point, since this takes time and money). Of course, over time (temperature, power supply rails, etc.) the ADC may drift a bit, so the calibration doesn't work as well (some ADCs include self-calibration routines to compensate for drift over various parameters). Once cal and comp is done, most systems no longer look at the input, and just capture the data. So, when the actual recording is made, it's all up to the ADC to get it right.

Similar things can happen with DACs, calibrating and compensating initial errors in production, then perhaps self-calibrating later. However, though we may be able to create and maintain a near-perfect DAC, as discussed above the DAC doesn't know (and has no way of finding out) the original signal's shape and so cannot compensate to it. We make the best DAC we can, and try to keep it that way, then it's up to it to faithfully recreate whatever is on the CD (or whatever).

Hope that helps - Don

You ask good questions, sure you aren't an engineer? Anyway, "true fidelity" is an ambiguous term, and answering your question is a bit tricky. If you capture just two samples of a 20 kHz (k = 1000, Hz = Hertz, capital H because it's a person's name) sine wave then you have all the information you need to perfectly reconstruct the signal (to within the resolution of the system, e.g. the number of bits in the ADC). BUT, with only two samples the noise floor of the FFT will be about as high as the signal, and of course in the real world it's hard to hear just one cycle of anything. However, on the output (DAC) side when you pass just two samples through an ideal low-pass filter, you get a perfect 20 kHz output pulse (assuming no other errors in the system). So, if you have acquired those two samples with high fidelity, then you can reconstruct an equally high-fidelity output.

Now, your question really leads into two different areas: repetitive sampling, and FFT noise floor.

1. If you are thinking high fidelity is not achieved because we have only two data points at 20 kHz so the wave form looks very squared-off, that's not quite right, as discussed above. If we filter those two samples, we will indeed get our 20 kHz sine wave back. If you are thinking we need more samples to get a continuous-looking signal, then that's true but does not affect the fidelity of the output (as measured on a ideal spectrum analyzer, for instance, with that caveat about FFT noise floor I'll get to in a bit).

I think you are thinking about sampling the sine wave over and over to get more points along the waveform. For this to work, the signal must not be perfectly synchronized to the sampling frequency, or we could end up just sampling those same two points over and over. If the signal is just a little out of phase with the sampling clock, then over time we will add more points and eventually grab every value the ADC can produce. In between are frequencies synchronized to the ADC but chosen to fit a particular sample size, e.g. if we are measuring an ADC and our system can capture 64K (K = 1024 so 16K = 65,536, 2^16) samples, we will choose a waveform that starts and ends exactly at 0 and 65,535 to get "clean" FFTs. In that case, we may not acquire all the different values the ADC can provide, and if we take more samples it does not matter -- we'll just start repeating earlier samples.

However, 2 samples or 65,536, the fidelity at the output after filtering is the same, and as good as the resolution of the ADC. The wave just doesn't look as pretty before filtering.

2. When we measure converters we use fast Fourier transforms (FFTs) that are a discrete version of the Fourier transform. The Fourier transform allows us to go between time and frequency domain pictures of the signal, and is useful when we want to see the frequency response of a signal from the time samples the ADC provides (or vice-versa). Fast versions are called that because they use sample sets of power-of-two lengths and that allows the math to be much simpler (and amenable to parallel processing). In audio systems we are also taking real data (vs. complex) and that allows further mathematical tricks to speed up the transforms. When you are looking at my (and most other's) spectral plots (frequency response), you are looking at the output of an FFT.

The FFT potentially provides processing gain that lowers the FFT's noise floor. We speak of FFTs having a bin width, the span of frequencies covered by a single "bin" in the FFT. If you take 128 points, you end up with 128 frequency bins (I won't get into the math). So, if you are sampling at 44 kHz and take 128 points, each bin is 44 kHz/128 = 343.75 Hz. This means all the quantization noise within that range falls into that one bin. If we take 16K (16,384) samples the FFT bins are only 2.7 Hz wide, so much less noise falls into each bin. This means the FFT noise floor, distinct from the ADC's noise floor, depends on how many samples we take. The net result is that for an N-bit ADC the noise floor of the FFT plot is the ADC's noise floor plus the effect of having a finite number of samples in the FFT. This modifies the SNR equation a little:

SNR (max) = 6.021N + 1.761 + 10log(M/2) for M samples in the FFT record and a perfect ADC

In practice, we will (almost) always collect enough samples so that the last term is so much larger than the intrinsic ADC's SNR that we can neglect the added noise from the FFT. Thus, when we show an FFT plot, you are looking at the ADC's quantization noise floor, not that of the FFT.

Hopefully that helps! This is getting a bit complicated, but you asked for it... - Don

No worries Les, but yeah this is getting to be a lot of work! I'm going to ask Amir for a raise.

Real signals do not have infinite bandwidth and thus cannot have zero rise time. So, no matter how square it looks, a square wave still has limited bandwidth. Take a look at the "Building a Square Wave" thread for more insight. If you sample a 20 kHz square wave, the ADC's anti-alias filter will roll off all the higher harmonics, so you'll get a sine-wave pulse out with no frequency content higher than 20 kHz (22 kHz, but let's forget that for now). We have to roll it of to prevent aliasing (see the Aliasing 101 thread) and introducing a bunch of unwanted spurs into the signal band. So, we now have a digital signal, and we are talking about this (very unrealistic) case of just a couple of samples of a maximum-frequency input signal.

A DAC typically switches very fast, so those two samples can be output with much higher frequency content (and thus faster edges, or attacks) than what the ADC saw. The output of a DAC is after the sampling process, and can indeed put out very wide-band spurs. Most of that is not needed, and is in fact undesirable. You can't hear it, and it contains switching artifacts (glitches) with very high frequency content, so why fry your tweeters with it? Thus we have the infamous DAC output filter so maligned by the audio press. With good reason in some cases, but at least now it outputs what the ADC saw, for better or worse.

Regarding those fast attacks, well, a drum's leading edge does not have much if any energy at 20 kHz, or anywhere near it. The closest you get is with a big symbol crash, and while there is content up there, most of what you hear is in the band up to 8 - 10 kHz or so. Yes, there can be content above 20 kHz, but very few of us could hear it even if it matched the fundamental's amplitude, and in fact it is much lower than that. Consider that what most of us perceive as the attack of a kick drum is that percussive leading edge that hits us in the chest so we go "wow!" So, what does everybody say we need to really get that solid bass "wump"? Good bass. Maybe a subwoofer. I have not read a review recently touting how a tweeter made the drums sound so much better (OK, maybe the cymbals, but the bass drum? Hmmm...) The fact is, the bandwidth of that attack is not all that great.

Now, let's look at a piano. Elsewhere (I forgot where, senility) I noted that you need about 5x the fundamental frequency to have a square wave that looks square'ish with a leading edge nearly as fast as an ideal square wave. (Look again at the square wave thread.) If we have 20 kHz bandwidth, the highest square wave we can get with five harmonics is 4 kHz. And the highest key on a piano (C8) is -- taa-daa! -- 4.186 kHz. Which is, I must say, a couple of notes above the highest note normally written for piccolo, an instrument with nearly as pure a single-tone sine wave as you can get. Yes, some instruments, including piano, put out higher harmonics, but realistically if you have 20 kHz of bandwidth you pretty much have everything covered and then some. (There are other reasons why wider bandwidth may be desirable for some components, but I'm getting tired.)

So, those fast attacks we all like to talk about, aren't as fast as we'd like to think. Your average system has more than enough bandwidth to handle them. How well, well, that's another story, probably a little too subjective for here!

G'night - Don

p.s. For the record, the lowest string on a bass guitar is about 40 Hz. A contrabass tuba goes to about 35 Hz, harp to about 33 Hz, and piano to 27.5 Hz. A really big pipe organ goes to 16 Hz, but most only to 32 Hz. Kettle drums (tympani) only to 80 -90 Hz, about the same as a really deep bass singer (and sopranos can hit about 1100 Hz). Guitar, piano, harp, and drum (among others) are all percussive instruments, with frequency content below their fundamental when struck, however.

You're welcome! I should note that, while the edge speed is pretty well set, the energy out to the 5th is still significant albeit less than half as loud as the fundamental. Fortunately, square waves are rarely found in music... Here are the relative harmonic levels for a square wave (remember the even harmonic terms are zero for an ideal square wave):

H1 = 1 = 0 dB (fundamental)
H3 = 0.333 = -9.542 dB
H5 = 0.2 = -13.979 dB
H7 = 0.143 = -16.902 dB
H9 = 0.111 = -19.085 dB
H11 = 0.091 = -20.828 dB
H13 = 0.077 = -22.279 dB
H15 = 0.067 = -23.522 dB

Notice they get smaller but closer in amplitude as the harmonics get higher. So, they are contributing less, and their amplitudes are more similar to each other as the harmonic number increases. What does that mean? Reviewing the square wave picture in the "Building a Square Wave" thread, you can see how the edges do not get much faster, but the "flat" region gets flatter as the higher harmonics are added since they contribute more and more equally (in amplitude) to the response. Now you know why. It's sort of like a bunch of ripples in a pool or pan of water; poke your finger in once in a pan of water and you see nice big waves radiating out; dip it in a bunch of times rapidly at different places (different "phases") and you get a more complex pattern that's actually a little "flatter" overall than a single big ripple due to all the wave interference going on.

HTH - Don

http://www.whatsbestforum.com/showthread.php?990-Units-Symbols-and-Terms-Oh-My!