Weekly blog 4: Derivations of the Fourier Transform

I swear I'll get to the discussion of first and second order systems in a bit, but before leaving the topic of transforms altogether, I wanted to write another blog on the Fourier transform. You see, the Fourier transform is special to me because, despite the fact it is really easy to understand intuitively, no one ever gave me this intuition and I had to build it on my own. That's really a shame, since I believe intuition can be a much more agile tool than rigor, especially for people who are practitioners rather than just theorists, and so I wanted to take some time to document my own intuition for the Fourier transform hoping that someone else might find it useful.

1. Sinusoidal Representations of Functions

Most people know that functions can be represented and approximated by other functions in a methodical way. For instance, the Taylor series can be used to approximate a function by a series of polynomials with the right coefficients, in the case of Taylor given by their derivatives evaluated at the expansion point. Most people also know that series of sines and cosines can be used to do the job for Periodic functions. In particular the series

$$S = \left\{\sin\frac{2\pi n t }{T}\right\}_{n\in\mathbb{N}}$$

is orthonormal with respect to the inner product 

$$(f,g) = \frac{1}{T}\int_{-T/2}^{T/2}f(t) g(t)\ dt,$$

and thus, for an arbitrary function \(f\) on \([0,T]\), we can write an approximation consisting of all sines, e.g. find \(a_0,a_1,a_2,\dots,a_N\) such that 

$$f(t) \approx \sum_{n=0}^Na_n\sin\frac{2\pi n t }{T},\ \ t\in [0,T].$$

The same is true of cosines. Notice that these are all sines and cosines whose frequencies are discrete harmonics of the fundamental frequency, defined by \(\omega_n = 2\pi n/T\).

2. The Spring Analogy

The trick I keep in mind when thinking about the Fourier transform is simple: I know that these sines and cosines arise as solutions for simple harmonic oscillators (say ideal, massless, infinitely elastic, undamped springs) which each satisfy the equation

$$\ddot{x} = -\omega^2_nx.$$

Actually, we can even write these solutions together, in an exponential representation given by \(e^{\pm j\omega_nt}\). What the Fourier series of a function essentially says is that the function may be approximated by a weighted sum of the solutions to simple harmonic oscillator equations.  This is equivalent to weighting the solution by either the harmonics of a single spring of the fundamental of a (possibly infinite) series of springs whose natural frequencies are all harmonics of each other.

Now we imagine including not just positive harmonics but (mathematically possible) negative harmonics of an infinite series of springs. We write

$$f(t) \approx \sum_{n=-\infty}^{\infty}a_nS_n, $$

where \(S_n\) is any choice for representing the solution to the spring equation. Finally, we imagine an uncountably infinite (stay with me) number of springs which have natural frequencies at every number from \(-\infty\) to \(\infty\) and go ahead and use \(S_n = e^{j\omega_n t}\) where now \(n\in\mathbb{R}\). We can drop the subscripts on the frequencies (since now there's no point) and turn the sum into an integral. Since each frequency needs a unique coefficient (to determine its weighting in the overall sum) the sequence \(a_n\) becomes a function \(a(\omega)\) and we may write

$$f(t) = \int_{-\infty}^{\infty}a(\omega)e^{j\omega t}\ d\omega.$$

\(a(\omega)\) still represents the weighting of the springs, but using a continuous index now instead of a discrete index, and just as \(a_n\) as a sequence represented the function by the weighting given to a series of springs whose solutions were understood, so too does \(a(\omega)\). The difference however is that while \(a_n\) approximately represented \(f\) using a set number of harmonics, \(a(\omega)\) gives an exact representation in terms of all possible frequencies. Thus we say that \(a(\omega)\) is the frequency representation of \(f(t)\), and can be thought of as the function which assigns a weighting to the set of all possible springs so that their combined behavior imitates \(f(t)\) exactly. This representation is one-to-one with the set of all continuous, integrable, bounded functions \(f(t)\). In fact there is an inverse:
$$a(\omega) = \int_{-\infty}^{\infty}f(t)e^{-j\omega t}\ dt.$$
Now where have I seen that before? 

Weekly Blog 3: More About Integral Transforms

Welcome to the 3rd installment of the weekly controls blog! I was going to do a blog on first and second-order systems, but wow, those Laplace transforms last time just got my blood pumped to extend the discussion on integral transforms a bit! I've also recently become interested in applications of integral equations, Fredholm theory, and fractional calculus to control and I figure recounting a bit of common knowledge on integral transforms is a good way to dive into this.

Of course I won't do hardly any more on integral transforms in the large scope, there are 500 page books for that. Instead I'll work a little bit more with the Laplace Transform
$$(\mathcal{L}f)(s) = \int_{0}^{\infty}e^{-st}f(t)\ dt,$$
and the Fourier transform
$$(\mathcal{F}f)(\omega) =  \int_{-\infty}^{\infty}e^{-j\omega t}f(t)\ dt,$$
where, since I'm an engineer, I prefer to annoy mathematicians by defining \(j^2 = -1\).

1. Laplace Transform of a Delay

We've seen the widely used result \(\mathcal{L}\dot{y}(t) = sY(s)\) in the previous blog. This can also be seen as the Laplace transform of the operator \(d/dt\) which gives us the operator equation

$$ \mathcal{L}\frac{d}{dt} = s,$$

or, recursively

$$ \mathcal{L}\frac{d^n}{dt^n} = s^n.$$

We proved this by taking an arbitrary sample function, in this case \(y\), and using derivatives of this function (with the known Laplace transform \(Y(s)\)) we were able to compute the Laplace transform for the derivative itself. 

Now let \(D_\tau\) be the delay operator which is defined as the operator for which

$$D_\tau f(t) = f(t-\tau).$$

That is, \(D_\tau\) simply delays an arbitrary function \(f\) by some amount of time \(\tau\). The question of this section will be to compute \(\mathcal{L}D_\tau\). We do this again by using a sample function, say \(f\) so that we have

$$ \mathcal{L}D_\tau f = \int_{0}^{\infty}e^{-st}D_\tau f(t)\ dt = \int_{0}^{\infty}e^{-st}f(t-\tau)\ dt.$$

Letting \(\sigma(t) = t-\tau\) we find \(d\sigma = dt\) and \(\sigma(0) = -\tau,\sigma(\infty)=\infty\). Then

$$\int_{0}^{\infty}e^{-st}f(t-\tau)\ dt = \int_{-\tau}^{\infty}e^{-s(\sigma+\tau)}f(\sigma)\ d\sigma = e^{-s\tau}\left(\int_{-\tau}^{0}e^{-s\sigma}f(\sigma)\ d\sigma + \int_{0}^{\infty}e^{-s\sigma}f(\sigma)\ d\sigma \right).$$

As before we define \(F(s) = \mathcal{L}f\). Now we need only take care of that bizarre term with the delay in the integral bounds. We have a couple of options: most people define \(f\) as zero on \(0\leq t\leq\tau\) which you will find makes the integral above work out to have only \(F(s)\) and not the second term, but this is ad hoc for me. We can equivalently simply define \(f = 0\) for all \(t < 0\), but again why should we? We obviously need to do something. For now let's just say

$$\int_{-\tau}^{0}e^{-s\sigma}f(\sigma)\ d\sigma + F(s) \approx F(s)$$

but keep that issue in the back of our minds. We thus have

$$\mathcal{L}(D_\tau) = e^{-s\tau}.$$

2. Fourier Transform of the Derivative and Neper Frequency

I'm finished with the Laplace transform for now, though I make no promises of never returning to it again...Let's go ahead and figure out what the Fourier transform of the derivative is. Say \(y\) is our sample function whose Fourier transform is \(Y(\omega)\). Then

$$(\mathcal{F}\dot{y})(s) = \int_{-\infty}^{\infty}e^{-j\omega t}\dot{y}(t)\ dt = -j\omega e^{-j\omega t}y(t)|_{-\infty}^{\infty} - \left(-j\omega\int_{-\infty}^{\infty}e^{-j\omega t}y(t)\ dt\right)$$

so, assuming again the surface term vanishes, 

$$(\mathcal{F}\dot{y})(s)  = j\omega Y(\omega),$$

or

$$\mathcal{F}\frac{d}{dt} =  j\omega.$$

This in particular allows us to easily map linear systems from the \(s\)-domain to the frequency domain by sending \(s\mapsto j\omega\). This is the so-called Joukowski substitution, though it is unclear whether or not Joukowski was the first one to notice this fact. 

Although the Joukowski substitution is the most typically used way of mapping into the frequency domain, it should be noted for completeness that it assumes the forcing frequency is perfect and non-attenuating. Consider for instance the signal 

$$y(t) = \sin(\omega t)$$

The Joukowski substitution works (in the complexified case) and we have no problem, but what about 

$$y(t) = A(t)\sin(\omega t),$$

where \(A\rightarrow 0\) as \(t\rightarrow \infty\)? In this case the Joukowski substitution is not the correct map to the frequency domain. Instead we must introduce the Neper Frequency, \(\sigma\), which keeps track of the signal attenuation, and send \(s\mapsto \sigma+j\omega\) to get into the frequency domain. In practice however, the frequency domain representation of a function is regarded as being consistent with a series of ideal simple harmonic oscillators being used to represent a function and the Neper frequency rarely comes into play when analyzing systems at a purely mathematical level.