From Time to Frequency: An AI Odyssey in Signal Processing#

1.1 Continuous vs. Discrete Signals#

In practice, signals can be:

1. Continuous-time: The signal is defined at every instant in time. An analog microphone output before digitization is an example.
2. Discrete-time: The signal is defined at specific time intervals. For instance, a digitized audio sample at a sampling rate of 44.1 kHz.

Although continuous signals are ideal mathematical models, discrete signals dominate in digital hardware and software systems. Much of modern signal processing focuses on discrete-time signals because computers process data in discrete steps.

1.2 Deterministic vs. Random Signals#

A signal can also be described by whether its future values can be exactly predicted:

• Deterministic: The signal can be described by a specific function or formula.
• Random (Stochastic): Future values have some inherent randomness. For instance, noise captured by sensors is often treated as a random process.

Whether you’re dealing with deterministic signals (like pure sinusoids) or random signals (like thermal noise), you often need similar mathematical tools, but you apply them differently depending on the behavior you expect.

1.3 Representation in the Time Domain#

The most direct way to represent a signal is as a sequence or function of time, like x(t) in the continuous case or x[n] in the discrete case (where n is an integer index corresponding to the sample number). To analyze temporal features such as the amplitude changes, peaks, or zero crossings, you can simply plot x(t) or x[n] against time.

2.1 The Need for Sampling#

Real-world signals are analog and vary continuously. However, a computer needs discrete samples to store and process signals digitally. At regular intervals ( T_s ) seconds, you record the amplitude, creating a discrete-time signal:

[ x[n] = x(nT_s), \quad n = 0, 1, 2, \dots ]

The quantity ( T_s ) is the sampling period, and its reciprocal ( f_s = \frac{1}{T_s} ) is the sampling rate.

2.2 The Nyquist-Shannon Sampling Theorem#

A key question is how fast you must sample a signal to capture its information without losing details. The Nyquist-Shannon sampling theorem states:

To perfectly reconstruct a band-limited signal from its samples, the sampling rate must be at least twice the signal’s highest frequency component.

If the highest frequency in a signal is ( f_{max} ), the sampling rate ( f_s ) should satisfy:

[ f_s \ge 2 f_{max} ]

For example, if the highest audio frequency you wish to capture is 20 kHz (the nominal upper limit of human hearing), then you must sample at least at 40 kHz. That’s why standard audio is sampled at 44.1 kHz or 48 kHz—just above the threshold.

Sampling below this threshold causes aliasing, where high frequencies “fold�?and become indistinguishable from lower frequencies. To mitigate aliasing, real-world systems also use anti-aliasing filters, low-pass filters that remove frequencies above the half of the sampling rate.

3.1 Why Transform to Frequency Domain?#

When analyzing a signal in the time domain, it’s straightforward to see when a certain amplitude spike occurs. However, if you want to know which frequencies are present and in what proportions, the time-domain representation can be obscure. Transforming a signal into the frequency domain is often more insightful for filtering, compression, feature extraction, and system analysis.

3.2 The Concept of Orthogonal Bases#

Think of a signal as a combination of simpler waveforms. The mathematical foundation for transforming signals is that you can represent them in an orthonormal basis—like how any point in 2D space can be expressed as a combination of two perpendicular vectors. In frequency analysis, these “basis functions�?are typically complex exponentials (or sines and cosines), which are naturally orthogonal across frequency.

3.3 The Fourier Series (for Periodic Signals)#

If a continuous signal ( x(t) ) is periodic, you can represent it by a sum of sines and cosines (or complex exponentials). This sum is the Fourier series:

[ x(t) = a_0 + \sum_{k=1}^{\infty} \left[ a_k \cos(k \omega_0 t) + b_k \sin(k \omega_0 t)\right] ]

where ( \omega_0 ) is the fundamental frequency in radians per second. Although the Fourier series is typically taught early in signal processing, it applies best to pure periodic signals (like a perfect repeating wave).

3.4 The Fourier Transform (for Aperiodic Signals)#

Real signals often aren’t perfectly periodic. Enter the Fourier transform, denoted ( X(\omega) ) for continuous-time signals, which provides a continuous frequency spectrum:

[ X(\omega) = \int_{-\infty}^{\infty} x(t) e^{-j \omega t} , dt ]

For discrete-time signals, the Discrete-Time Fourier Transform (DTFT) provides a continuous spectrum over the frequency range (-\pi) to (\pi). But we often use an even more practical version, the Discrete Fourier Transform (DFT), which we’ll talk about next.

4.1 DFT Definition#

While the DTFT is conceptually perfect, it produces an uncountably infinite set of frequency points. A computer can’t handle infinite points at once. Instead, we compute a finite number of frequency bins using the DFT. For an ( N )-point discrete signal ( x[n] ), the DFT is defined as:

[ X[k] = \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi}{N}kn}, \quad k = 0, 1, \dots, N-1. ]

4.2 Fast Fourier Transform (FFT)#

Computing this sum directly is (O(N^2)) in complexity. The Fast Fourier Transform (FFT) reduces it to (O(N \log N)), making it feasible to analyze signals with thousands or millions of samples.

4.3 Practical Code Example#

Below is a small Python snippet demonstrating how to compute the DFT via NumPy, which uses FFT algorithms under the hood:

1
import numpy as np
2
import matplotlib.pyplot as plt
3

4
# Create a time-domain signal: sum of two sinusoids
5
fs = 1000  # Sampling rate (Hz)
6
t = np.arange(0, 1, 1/fs)  # 1 second of data
7
f1, f2 = 50, 120  # Frequencies of the sinusoids
8
x = np.sin(2*np.pi*f1*t) + 0.5*np.sin(2*np.pi*f2*t)
9

10
# Compute the FFT
11
X = np.fft.fft(x)
12
N = len(X)
13

14
# Frequency axis
15
freqs = np.fft.fftfreq(N, 1/fs)
16

17
# Plot the magnitude spectrum
18
plt.figure()
19
plt.stem(freqs[:N//2], np.abs(X)[:N//2], use_line_collection=True)
20
plt.xlabel('Frequency (Hz)')
21
plt.ylabel('Magnitude')
22
plt.title('Magnitude Spectrum')
23
plt.show()

In this code:

1. We create a one-second signal containing two sinusoids at 50 Hz and 120 Hz.
2. We apply the FFT and get frequency bins up to ( fs/2 ) (the Nyquist frequency).
3. We plot the result and see two peaks, correlating to the two tones in the signal.

5.1 Windowing#

A raw, finite-time signal can produce spectral leakage due to sharp boundaries in the discrete data sequence. Windowing helps mitigate these artifacts. Common windows include:

• Rectangular (no additional weighting),
• Hanning (smooth edges),
• Hamming (similar to Hanning but with different coefficients),
• Blackman, and more.

They have different frequency-domain characteristics such as main-lobe width (frequency resolution) and side-lobe levels (leakage). A table summarizing them might look like this:

(Above table is generalized; actual numeric details can vary based on definitions.)

5.2 Short-Time Fourier Transform (STFT)#

While the Fourier transform is powerful, it assumes stationarity—frequencies do not change over time. Many real signals (e.g., speech, music) are non-stationary. The Short-Time Fourier Transform (STFT) addresses this by segmenting the signal into short frames and applying an FFT to each frame.

1. Divide the signal ( x[n] ) into overlapping segments ( x_m[n] ), each windowed by a function ( w[n] ).
2. Compute the DFT of each windowed segment to obtain ( X_m[k] ).
3. Concatenate these spectra into a time-frequency representation.

This produces a spectrogram—an image showing how frequency content evolves over time.

5.3 Example: Speech Spectrogram#

1
import numpy as np
2
import matplotlib.pyplot as plt
3
from scipy.io import wavfile
4
from scipy.signal import stft
5

6
# Load an example WAV file
7
fs, audio = wavfile.read('example_speech.wav')  # Suppose it exists in your environment
8

9
# Compute STFT
10
f, t, Zxx = stft(audio, fs=fs, nperseg=256, noverlap=128)
11

12
# Plot spectrogram
13
plt.pcolormesh(t, f, np.abs(Zxx), shading='gouraud')
14
plt.ylabel('Frequency (Hz)')
15
plt.xlabel('Time (s)')
16
plt.title('Speech Spectrogram')
17
plt.colorbar(label='Magnitude')
18
plt.show()

This script:

1. Reads a WAV file (a speech sample).
2. Performs STFT with 256-sample windows and 128-sample overlap (50% overlap).
3. Plots the spectrogram (with frequency on the y-axis and time on the x-axis).

6.1 Limitations of STFT#

The STFT has a trade-off between time resolution and frequency resolution: small windows capture fine time details but poor frequency resolution; large windows capture frequency details well but blur short events. Sometimes, the STFT is not flexible enough.

6.2 Introduction to Wavelets#

Wavelet transforms address these limitations by using time-scalable windows. Instead of fixed-length windows, wavelets adapt, giving higher time resolution at high frequencies and higher frequency resolution at low frequencies. Wavelet functions can be real or complex, and they allow you to zoom in on specific structures in your signal.

6.3 Continuous Wavelet Transform (CWT)#

The Continuous Wavelet Transform is defined by convolving the signal ( x(t) ) with scaled and translated versions of a wavelet ( \psi ):

[ W(a, b) = \frac{1}{\sqrt{a}} \int_{-\infty}^{\infty} x(t) , \psi!\Big(\frac{t - b}{a}\Big), dt ]

where ( a ) is the scale (inversely related to frequency) and ( b ) is the translation in time.

6.4 Discrete Wavelet Transform (DWT)#

In practice, the Discrete Wavelet Transform (DWT) is used more frequently for computational clarity. DWT decomposes the signal into scales using filter banks (low-pass and high-pass filters), halving the time resolution at each level. This forms the basis of wavelet-based compression (e.g., JPEG2000 for images).

6.5 Python Example with PyWavelets#

1
import pywt
2
import numpy as np
3
import matplotlib.pyplot as plt
4

5
# Sample: create a synthetic signal
6
fs = 500
7
t = np.linspace(0, 2, 2*fs, endpoint=False)
8
sig = np.sin(2*np.pi*50*t) + 0.5*np.sin(2*np.pi*80*t)
9

10
# Add a short burst of high-frequency noise
11
sig[250:270] += 2.0 * np.random.randn(20)
12

13
# Perform continuous wavelet transform with 'morl' wavelet
14
scales = np.arange(1, 128)
15
coefficients, frequencies = pywt.cwt(sig, scales, 'morl', sampling_period=1/fs)
16

17
# Plot the result
18
plt.figure(figsize=(8,6))
19
plt.imshow(np.abs(coefficients), extent=[0, t[-1], scales[-1], scales[0]], cmap='viridis', aspect='auto')
20
plt.title('CWT of Synthetic Signal')
21
plt.ylabel('Scale')
22
plt.xlabel('Time (s)')
23
plt.colorbar(label='Magnitude')
24
plt.show()

In this code:

1. We build a two-second signal with two sinusoids and a burst of high-frequency noise.
2. We compute the CWT using the Morlet wavelet.
3. We visualize it over the time-scale plane (which can be recast in time-frequency, given scale (\leftrightarrow) frequency relationships).

7.1 Low-Pass, High-Pass, and Band-Pass Filtering#

Filters can remove unwanted components from a signal (e.g., hum noise in audio). Rather than building a complicated time-domain filter, you can:

1. Transform the signal into the frequency domain (via FFT).
2. Zero out or attenuate unwanted frequencies.
3. Return to the time domain (via inverse FFT).

For example, in audio denoising, you might zero out frequencies above 15 kHz if you know the noise is out of the voice range.

7.2 Example Filtering Code#

1
import numpy as np
2
import matplotlib.pyplot as plt
3

4
fs = 1000
5
t = np.arange(0,1,1/fs)
6
signal = np.sin(2*np.pi*50*t) + 0.25*np.random.randn(len(t))
7

8
# FFT
9
S = np.fft.fft(signal)
10
freqs = np.fft.fftfreq(len(S), 1/fs)
11

12
# Design a simple low-pass cutoff
13
cutoff = 100  # Hz
14
S_filtered = np.where(np.abs(freqs) > cutoff, 0, S)
15

16
# IFFT
17
signal_filtered = np.fft.ifft(S_filtered).real
18

19
# Compare signals
20
plt.figure()
21
plt.subplot(2,1,1)
22
plt.plot(t, signal)
23
plt.title('Original Signal')
24
plt.subplot(2,1,2)
25
plt.plot(t, signal_filtered, 'g')
26
plt.title('Filtered Signal')
27
plt.tight_layout()
28
plt.show()

This simple approach works for offline processing or for signals you can handle in batch. For real-time applications, you often need digital filter design (e.g., FIR/IIR filters) that operate incrementally on incoming data. But conceptually, the process is the same—removing undesired frequencies and keeping what you need.

8.1 From Features to Machine Learning#

Machine learning requires features that help differentiate signals of interest. The transformations (FFT, STFT, wavelet) can reveal meaningful patterns embedded in the frequency or time-frequency domain. By extracting:

• Spectral peaks or bandwidths,
• Time-frequency energy distributions,
• Wavelet coefficients,

…you feed more discriminative information to a machine learning model (e.g., logistic regression, random forests, or neural networks).

8.2 Traditional Approaches: Feature Engineering#

In many classic approaches:

1. Compute STFT or wavelet transform.
2. Extract summary statistics (e.g., energy in certain frequency bands, wavelet subband energies).
3. Feed these features into a traditional classifier (e.g., SVM) or regressor.

For an ECG classification example, you might:

• Extract R-peak intervals for heart rate,
• Compute wavelet decomposition to analyze specific frequency bands,
• Use wavelet coefficients as features in an SVM that classifies normal vs. arrhythmic heartbeats.

8.3 Deep Learning Approach: End-to-End#

Deep learning often bypasses the need for extensive hand-crafted features:

• A Convolutional Neural Network (CNN) can learn to detect relevant time-frequency patterns directly from spectrograms.
• Recurrent Neural Networks (RNNs) can capture temporal dependencies in raw signals.
• Transformers, originally for language, have found use in advanced time-series classification.

Below is a simplified PyTorch example of a CNN architecture for a spectrogram classification task:

1
import torch
2
import torch.nn as nn
3
import torch.nn.functional as F
4

5
class SpectrogramCNN(nn.Module):
6
    def __init__(self, num_classes=10):
7
        super(SpectrogramCNN, self).__init__()
8
        self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1)
9
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1)
10
        self.fc1 = nn.Linear(32*32*32, 128)
11
        self.fc2 = nn.Linear(128, num_classes)
12

13
    def forward(self, x):
14
        # x shape: (batch_size, 1, freq_bins, time_steps)
15
        x = F.relu(self.conv1(x))
16
        x = F.max_pool2d(x, 2)
17
        x = F.relu(self.conv2(x))
18
        x = F.max_pool2d(x, 2)
19
        x = x.view(x.size(0), -1)
20
        x = F.relu(self.fc1(x))
21
        x = self.fc2(x)
22
        return x

1. Input: a batch of spectrogram images with dimensions (frequency × time).
2. Layers: two convolutional layers, each followed by ReLU and max-pooling, then two fully connected layers.
3. Output: a probability distribution over the given classes.

9.1 Adaptive Time-Frequency Techniques#

You may need techniques that adapt more dynamically than STFT or wavelets. Empirical Mode Decomposition (EMD) or Variational Mode Decomposition (VMD) adaptively extract intrinsic modes from signals, which can then be fed into machine learning models. These are especially helpful for non-linear or extremely non-stationary signals.

9.2 Transfer Learning for Signal Processing#

Deep networks typically require large labeled datasets, which may not always be available. Transfer learning can be a game-changer:

• Pre-train a neural network on a large corpus of similar signals (or even images)
• Fine-tune it on your smaller domain-specific dataset.

9.3 Data Augmentation#

To bolster model robustness, you can artificially inflate your dataset:

• Mix signals,
• Add noise,
• Shift the time axis,
• Apply random filtering.

In the frequency domain, you might randomize phase or slightly shift frequencies. This controlled “augmentation�?approach helps the model generalize better.

9.4 Graph Signal Processing and GNNs#

An emerging area is Graph Signal Processing (GSP), where signals are defined on graphs (e.g., sensor networks). The frequency domain in such settings is defined by the graph Laplacian’s eigenvalues. Graph neural networks (GNNs) are then used to process these graph signals, opening new frontiers for complex sensor-based data analysis.

9.5 Real-Time Processing on Embedded Devices#

Edge computing can bring real-time signal analysis to embedded devices. Techniques involve optimizing neural network architectures (e.g., quantization, pruning) and using custom hardware accelerators (FPGAs, GPUs, or specialized ASICs). The challenge is to maintain accuracy while drastically reducing memory and computational load.