Weekly blog 4: Derivations of the Fourier Transform

I swear I'll get to the discussion of first and second order systems in a bit, but before leaving the topic of transforms altogether, I wanted to write another blog on the Fourier transform. You see, the Fourier transform is special to me because, despite the fact it is really easy to understand intuitively, no one ever gave me this intuition and I had to build it on my own. That's really a shame, since I believe intuition can be a much more agile tool than rigor, especially for people who are practitioners rather than just theorists, and so I wanted to take some time to document my own intuition for the Fourier transform hoping that someone else might find it useful.

1. Sinusoidal Representations of Functions

Most people know that functions can be represented and approximated by other functions in a methodical way. For instance, the Taylor series can be used to approximate a function by a series of polynomials with the right coefficients, in the case of Taylor given by their derivatives evaluated at the expansion point. Most people also know that series of sines and cosines can be used to do the job for Periodic functions. In particular the series

$$S = \left\{\sin\frac{2\pi n t }{T}\right\}_{n\in\mathbb{N}}$$

is orthonormal with respect to the inner product 

$$(f,g) = \frac{1}{T}\int_{-T/2}^{T/2}f(t) g(t)\ dt,$$

and thus, for an arbitrary function \(f\) on \([0,T]\), we can write an approximation consisting of all sines, e.g. find \(a_0,a_1,a_2,\dots,a_N\) such that 

$$f(t) \approx \sum_{n=0}^Na_n\sin\frac{2\pi n t }{T},\ \ t\in [0,T].$$

The same is true of cosines. Notice that these are all sines and cosines whose frequencies are discrete harmonics of the fundamental frequency, defined by \(\omega_n = 2\pi n/T\).

2. The Spring Analogy

The trick I keep in mind when thinking about the Fourier transform is simple: I know that these sines and cosines arise as solutions for simple harmonic oscillators (say ideal, massless, infinitely elastic, undamped springs) which each satisfy the equation

$$\ddot{x} = -\omega^2_nx.$$

Actually, we can even write these solutions together, in an exponential representation given by \(e^{\pm j\omega_nt}\). What the Fourier series of a function essentially says is that the function may be approximated by a weighted sum of the solutions to simple harmonic oscillator equations.  This is equivalent to weighting the solution by either the harmonics of a single spring of the fundamental of a (possibly infinite) series of springs whose natural frequencies are all harmonics of each other.

Now we imagine including not just positive harmonics but (mathematically possible) negative harmonics of an infinite series of springs. We write

$$f(t) \approx \sum_{n=-\infty}^{\infty}a_nS_n, $$

where \(S_n\) is any choice for representing the solution to the spring equation. Finally, we imagine an uncountably infinite (stay with me) number of springs which have natural frequencies at every number from \(-\infty\) to \(\infty\) and go ahead and use \(S_n = e^{j\omega_n t}\) where now \(n\in\mathbb{R}\). We can drop the subscripts on the frequencies (since now there's no point) and turn the sum into an integral. Since each frequency needs a unique coefficient (to determine its weighting in the overall sum) the sequence \(a_n\) becomes a function \(a(\omega)\) and we may write

$$f(t) = \int_{-\infty}^{\infty}a(\omega)e^{j\omega t}\ d\omega.$$

\(a(\omega)\) still represents the weighting of the springs, but using a continuous index now instead of a discrete index, and just as \(a_n\) as a sequence represented the function by the weighting given to a series of springs whose solutions were understood, so too does \(a(\omega)\). The difference however is that while \(a_n\) approximately represented \(f\) using a set number of harmonics, \(a(\omega)\) gives an exact representation in terms of all possible frequencies. Thus we say that \(a(\omega)\) is the frequency representation of \(f(t)\), and can be thought of as the function which assigns a weighting to the set of all possible springs so that their combined behavior imitates \(f(t)\) exactly. This representation is one-to-one with the set of all continuous, integrable, bounded functions \(f(t)\). In fact there is an inverse:
$$a(\omega) = \int_{-\infty}^{\infty}f(t)e^{-j\omega t}\ dt.$$
Now where have I seen that before?