DSP: Nyquist Frequency

In my last post, I mentioned that you have to sample at least twice the rate of the highest frequency component you expect in your system. For example, if you expect a signal that you’re sampling to only have frequencies below 100 kHz, then you have to sample at least 200 thousand times per second. Where does this limitation come from, and what happens if you violate it?

Theory

Before we talk about minimum or recommended sampling rate, let’s first see what happens when you sample a signal, first in the time domain, then in the frequency domain.

An analog signal is continuous in two ways. First, it is continuous in time: the signal continues to exist in between any two instances in time, no matter how close those instances are. Second, the signal is continuous in value: it can take an infinite number of values even within a tiny range. By sampling, you make the signal discrete in time; the signal can still take any value it wants, but the signal only exists at discrete steps. (Sampling also makes the signal discrete in value in the real world; for example, a 10 bit ADC will reduce the number of values the signal can be to 1024 values, but that’s not the focus of this post.)

The graph above shows an analog waveform on the left, and a discrete waveform created by sampling that waveform on the right. When you’re sampling a signal, you’re effectively multiplying that signal by a train of impulses, which is shown in the middle. The train of impulses has impulses that are a set time apart; if you’re sampling at 1 kHz, then the impulses are 1 ms apart, for example. But this multiplication is key to understanding the Nyquist frequency, which is the minimum frequency you need to sample your signal.

Multiplication in the time domain, as shown above, is convolution in the frequency domain. As it turns out, a train of impulses in the time domain is also a train of impulses in the frequency domain. If you’re sampling at 1 kHz, then the train of impulses in the frequency domain are 1 kHz part. The convolution of the original signal and the train of impulses is where minimum frequency requirement comes from.

Here are two rules to know when it comes to convolving impulses:

1. Any signal convolved with an impulse will result in the original signal.
2. Any signal convolved with an impulse shifted by F will result in the original signal, but shifted by F

The train of impulses is just a bunch of impulses shifted by different amounts; if you’re sampling at 1 kHz, then you have impulses shifted by 0 kHz (so an impulse at the origin), 1 kHz, 2 kHz, etc. That means the original signal, in the frequency domain, will be shifted by 0 kHz, 1 kHz, 2 kHz, etc.

Another important fact to remember is that frequencies have positive and negative components; for example, if the input signal has frequency components up to 300 Hz, then it also has frequency components down to -300 Hz, as shown above.

When the signal is convolved with a train of impulses which are 1 kHz apart, then a new signal is generated, which is shown above. The new signal has the original signal shifted by various amounts; in this case, 0 kHz, 1 kHz, 2 kHz, etc. One of the copies, which is shifted by 1 kHz, exists in the range of 700 Hz to 1300 Hz. There is another copy at 2 kHz, which similarly exists in the range of 1700 Hz to 2300 Hz. The copies also exist for negative frequencies; for example, the copy at -1 kHz exists from -1300 Hz to -700 Hz.

Of critical importance here is that the signals do not overlap; since the original signal has a full range of 600 Hz (-300 Hz to 300 Hz), shifting it by more than 600 Hz will ensure the copies do not interact with each other. If the copies do overlap, then the original signal is distorted and ultimately destroyed; we’ll see an example of this below. This distortion, or misrepresentation of the original signal, is called aliasing. But key here is the fact that copies must be placed far enough apart that they do not interact. The bare minimum is twice the highest frequency component in the signal.

Let’s do a quick review:

1. The original signal exists, and is composed of frequencies [-Fc, Fc]
2. You wish to sample this signal at a rate of Fs
3. In the frequency domain, you have the original signal [-Fc, Fc], but also have a copy at [-Fc+Fs, Fs+Fc]
4. In order to ensure the original and the copy do not interact, they must not touch. That is, Fc < -Fc+Fs.
5. Simplifying the inequality reveals that 2Fc < Fs; in other words, the sampling frequency must be greater than twice the highest frequency component in the original signal.

Examples

Let’s see what a 20 kHz sinusoid looks like when it is sampled at various rates. In the pictures shown below, the original signal is blue, while the signal constructed through sampling is shown in black.

Let’s ignore the fact that the sampled signal signal is very, very jagged. The original signal is 20 kHz, but the sampled signal has a frequency of 5 kHz! What happened?

The original, because it is a sin wave, only exists at one frequency, 20 kHz. It also has the negative component at -20 kHz. Let’s see what happens when we sample this signal at 25 kHz.

The original signal existed at -20 kHz and 20 kHz. When you sample at 25 kHz, you create copies that are shifted by intervals of 25 kHz. This means when the signal is shifted right by 25 kHz, the copy exists at 5 kHz and 45 kHz. When the signal is shifted left by 25 kHz, the copy exists at -45 kHz and -5 kHz. The copies that exist at -5 kHz and 5 kHz is what we’re seeing in the sample; that is, the sampled signal, due to aliasing, is misrepresenting 20 kHz as 5 kHz.

So now let’s try sampling at the Nyquist frequency, 40 kHz.

That’s not much better. Kind of worse, in fact. The 20 kHz looks like a DC signal. But we’re sampling at the correct frequency; what happened? Let’s try sampling at 40 kHz again, but shift the sampling point a bit.

So the sinusoid now looks like a triangle wave. Makes sense, actually; the sinusoid is 20 kHz, and you’re sampling at 40 kHz. That means you’ll only have two points per period; if you connect the dots together, then you either get a flat line or a triangle wave. But can you really say that the triangle wave accurately represents the original sinusoid? I mean the triangle wave has the right frequency of 20 kHz, but it has a smaller amplitude than the original signal. Even more distressing is the fact that the amplitude of the triangle wave varies depending on how much shift the sampling point! You can get a triangle wave with an amplitude of zero (as we’ve seen), an amplitude that’s the same as the original sinuisoid, or anything in-between.

This is where theory clashes with practice. In theory, you only have to sample at the Nyquist frequency to accurately capture the signal. And by accurately capture the signal, I mean prevent aliasing. What this means is that sampling at or above the Nyquist frequency only guarantees you’ll have accurate frequency data; it does not guarantee you’ll have accurate amplitude data. In fact, it is impossible to deduce the amplitude of the original sinusoid when sampling at the Nyquist frequency!

Here, we’re sampling at 90 kHz, which is 4.5 times the sinusoid frequency. You can see that it looks much better than when we sampled at the Nyquist frequency, but there’s still a lot of room for improvement. We get a much better idea of the amplitude of the sinusoid based off of the samples, but the samples within a period do not look consistent from one period to the next, so it almost makes it look like the sinusoid is modulated.

Here, we’re sampling at 235 kHz, which is 11.75 times the sinusoid frequency. I’d say this is a reasonable frequency to sample at. The samples always come close enough to the minimum and maximum values each period to capture the amplitude of the sinusoid pretty well.

Finally, we have sampling at 517 kHz, which is over 25 times the sinusoid frequency. Thanks to the very high sampling rate, we get a very accurate representation of the sinusoid. We have captured the frequency of the sinusoid, as well as accurately represented the amplitude. Remember that fully capturing the amplitude is nearly impossible; in order to truly capture the amplitude, you would have to sample the sinusoid right at its maximum or minimum value, which only exists for an instant.

I hope this post was helpful in understanding aliasing, the Nyquist frequency, minimum sampling rate, and why you usually want to sample much higher than the Nyquist frequency.