# Math2111: Chapter 1: Fourier series. Additional Material: L_2 convergence of Fourier series.

In this blog entry you can find lecture notes for Math2111, several variable calculus. See also the table of contents for this course. This blog entry printed to pdf is available here.

In this post I present some ideas which shed light on the question why one can expect the Fourier series to converge to the function (under certain assumptions).

Complete orthonormal systems

To do so let us first study a simpler case, one with which you are familiar with. (Throughout this section I will ignore convergence questions.)

Consider the vector space $\mathbb{R}^n$. For two vectors $\boldsymbol{u} = (u_1,\ldots, u_n)^\top$ and $\boldsymbol{v} = (v_1,\ldots, v_n)^\top$ we define the inner product

$\displaystyle \langle \boldsymbol{u}, \boldsymbol{v} \rangle = u_1 v_1 + \cdots + u_n v_n.$

Assume we have given a set of orthonormal vectors $\boldsymbol{u}_1, \ldots, \boldsymbol{u}_k,$ that is, we have

$\displaystyle \langle \boldsymbol{u}_k, \boldsymbol{u}_l \rangle = \left\{ \begin{array}{rl} 0 & \mbox{if } k \neq l, \\ & \\ 1 & \mbox{if } k = l. \end{array} \right.$

Now the question arises whether for every vector $\boldsymbol{v} \in \mathbb{R}^n$ there are $\lambda_1, \ldots, \lambda_k \in \mathbb{R}$ such that

$\displaystyle \boldsymbol{v} = \lambda_1 \boldsymbol{u}_1 + \cdots + \lambda_k \boldsymbol{u}_k?$

The answer is of course: it depends. If $k = n$, then yes, if $k \textless n$ then no. (Notice that $k \textgreater n$ is not possible.)

Consider now $\ell^2(\mathbb{Z})$ equipped with the standard inner product (defined here). Then let us ask the same question. Assume we have given a set of orthonormal vectors $\boldsymbol{u}_1, \ldots, \boldsymbol{u}_k,$ that is, we have

$\displaystyle \langle \boldsymbol{u}_k, \boldsymbol{u}_l \rangle = \left\{ \begin{array}{rl} 0 & \mbox{if } k \neq l, \\ & \\ 1 & \mbox{if } k = l. \end{array} \right.$

Is it true that for every vector $\boldsymbol{v} \in \ell^2(\mathbb{Z})$ there are $\lambda_1, \ldots, \lambda_k \in \mathbb{R}$ such that

$\displaystyle \boldsymbol{v} = \lambda_1 \boldsymbol{u}_1 + \cdots + \lambda_k \boldsymbol{u}_k?$

The answer is again it depends. If $k \textless \infty$, then certainly not because $\ell^2(\mathbb{Z})$ is infinite dimensional. If $k=\infty$, then we still do not know. Why? To illustrate this, let $A_k$ be the vector whose entries are all $0$ except the entry at position ${}k$ is ${}1$. More precisely, for $k \in \mathbb{Z}$ let

$\displaystyle A_k = (\ldots, b_{2,k}, b_{1,k}, a_{0,k}, a_{1,k}, a_{2,k}, \ldots)$

where we set

$\displaystyle \begin{array}{rcl} b_{n,k} & = & 0 \quad \mbox{if } n \neq -k, \\ && \\ b_{n,k} & = & 1 \quad \mbox{if } n = -k, \\ && \\ a_{n,k} & = & 0 \quad \mbox{if } n \neq k, \\ && \\ a_{n,k} & = & 1 \quad \mbox{if } n = k. \end{array}$

Choose now an arbitrary element

$\displaystyle C = (\ldots, d_2, d_1, c_0, c_1, c_2, \ldots) \in \ell^2(\mathbb{Z}).$

In general, which vectors $A_k$ do we need to represent $C?$ The answer is: all of them. Since we have

$\displaystyle C = \sum_{k=1}^\infty d_k A_{-k} + \sum_{k=0}^\infty c_k A_k.$

If we consider for instance the set of vectors

$\displaystyle \mathscr{A} = \{A_k \in \ell^2(\mathbb{Z}): k \in \mathbb{Z} \setminus \{23\}\}$

then the element $A_{23} \in \ell^2(\mathbb{Z})$ cannot be represented using the elements in $\mathscr{A}.$ In other words, $A_{23}$ is missing from the set $\mathscr{A}$.

Hence, for an infinite dimensional vector space it is not enough to have infinitely many orthonormal vectors in order to be able to represent any element. Notice that, if we have an arbitrary infinite set of orthonormal vectors

$\displaystyle \mathscr{B} =\{ \ldots, B_{-2}, B_{-1}, B_0, B_1, B_2, \ldots \}$

and a vector $A$ which cannot be represented by the vectors in $\mathscr{B}$, then there exists a vector $B$ such that

$\langle B, B_k\rangle = 0 \quad \mbox{for all } k \in \mathbb{Z}.$

Indeed, we can define the vector $B$ by

$\displaystyle B = A - \sum_{k \in \mathbb{Z}} \langle A, B_k \rangle B_k.$

Since $A$ cannot be represented by the elements in $\mathscr{B}$, it follows that $B$ is not the zero vector. Further

$\displaystyle \begin{array}{rcl} \langle B, B_n \rangle & = & \langle A, B_n \rangle - \sum_{k=-\infty}^\infty \langle A, B_k \rangle \langle B_k, B_n \rangle \\ && \\ & = & \langle A, B_n \rangle - \langle A, B_n \rangle \langle B_n, B_n \rangle = 0, \end{array}$

since we have chosen the elements $\ldots, B_{-2}, B_{-1}, B_0, B_1, B_2, \ldots$ such that

$\displaystyle \langle B_k, B_n \rangle = \left\{\begin{array}{rl} 1 & \mbox{if } k = n, \\ & \\ 0 & \mbox{if } k \neq n. \end{array} \right.$

Now let us consider Fourier series. Let us ignore questions concerning convergence in the following. For instance, say, let us only consider continuously differentiable functions. The same argument as for $\ell^2(\mathbb{Z})$ also applies in this case. If there would be a function $f$ for which we do not have

$\displaystyle f(x) = \frac{a_0}{2} + \sum_{k=1}^\infty \left[a_k \cos kx + b_k \sin kx \right]$

then there would have to be a (continuously differentiable), $2\pi$-periodic function $\phi:\mathbb{R} \to \mathbb{R}$ such that

$\displaystyle \begin{array}{rll} \langle \phi, \cos kx \rangle & = \int_{-\pi}^\pi \phi(x) \cos kx \, \mathrm{d} x & = 0 \quad \mbox{for all } k = 0, 1, 2, \ldots \mbox{and } \\ && \\ \langle \phi, \cos kx \rangle & = \int_{-\pi}^\pi \phi(x) \sin kx \, \mathrm{d} x & = 0 \quad \mbox{for all } k = 0, 1, 2, \ldots . \end{array}$

Therefore every (continuously differentiable), $2\pi$ periodic function can be represented by its Fourier series if and only if the functions

$\displaystyle \cos k x, \quad k = 0, 1, 2, \ldots \quad \mbox{and } \sin kx, \quad k = 1,2, \ldots \quad (1)$

are a complete orthonormal system, that is, there is no (continuously differentiable), $2\pi$ periodic function $\phi:\mathbb{R} \to \mathbb{R}$ which is orthogonal to all trigonometric functions given in line (1).

This does indeed hold, which follows from the convergence of the Fourier series to the function in the mean square sense. We prove this result in the next section.

$\rhd$ Mean square convergence

We prove now that the Fourier series of $f$ converges to $f$ in the mean square sense. That is,

$\displaystyle \lim_{N \to \infty} \|S_Nf- f\|_2 = 0,$

where $S_N f$ is the $N$ th partial sum of the Fourier series and the $L_2$-norm is defined by

$\displaystyle \|f\|_2 = \int_{-\pi}^\pi |f(x)|^2 \, \mathrm{d} x.$

To prove this we need some preparation. Instead of using $\cos kx$ for $k = 0,1,2, \ldots$and $\sin kx$ for $k = 1,2, \ldots$ we use in the following the functions $\mathrm{e}^{2 \pi \mathrm{i} k x}$ where $k \in \mathbb{Z}$. For complex valued functions $f, g: [-\pi, \pi] \to \mathbb{C}$, we define the inner product by

$\displaystyle \langle f, g \rangle = \int_{-\pi}^\pi f(x) \overline{g(x)} \, \mathrm{d} x,$

where $\overline{c}$ stands for the complex conjugate of a complex number $c \in \mathbb{C}$. Hence, we consider the Fourier series

$\displaystyle Sf(x) = \sum_{k=-\infty}^\infty a_k \mathrm{e}^{2 \pi {\rm i} k x},$

where the Fourier coefficients are given by

$\displaystyle a_k = \frac{1}{\pi} \int_{-\pi}^\pi f(x) \mathrm{e}^{-2\pi {\rm i} k x} \, \mathrm{d} x. \qquad\qquad\qquad (2)$

Further, the $N$th partial sum of the Fourier series is given by

$\displaystyle S_Nf(x) = \sum_{k=-N}^N a_k \mathrm{e}^{2 \pi {\rm i} k x},$

Exercise Show that (2) holds if $f$ is a trigonometric polynomial $f(x) = \sum_{n=-N}^N a_k \, \mathrm{e}^{2 \pi {\rm i} k x}$.

Definition (Dirichlet kernel) The trigonometric polynomial defined for $x \in [-\pi, \pi]$ by

$\displaystyle D_N(x) = \sum_{n=-N}^N {\rm e}^{2 \pi {\rm i} n x}$

is called the $N$th Dirichlet kernel.

Exercise Show that

$\displaystyle D_N(x) = \frac{\sin ((N+1/2) x)}{\sin (x/2)} .$

Definition (Fejér kernel) The trigonometric polynomial defined for $x \in [-\pi, \pi]$ by

$\displaystyle F_N(x) = \frac{1}{N} \sum_{n= 0}^{N-1} D_n(x)$

is called the $N$th Fejér kernel.

Exercise Show that

$\displaystyle F_N(x) = \frac{1}{N} \frac{\sin^2 (Nx/2)}{\sin^2 (x/2)} .$

Lemma
The Fejér kernel satisfies the following properties:

1. For all $N \ge 1$ we have

$\displaystyle \frac{1}{2\pi} \int_{-\pi}^\pi F_N(x) \, \mathrm{d} x = 1.$

2. There exists an $M > 0$ such that for all $N \ge 1,$

$\displaystyle \int_{-\pi}^\pi |F_N(x)|\, \mathrm{d} x \le M.$

3. For every $\delta > 0,$

$\displaystyle \int_{\delta \le |x| \le \pi} |F_N(x)| \, \mathrm{d} x \to 0 \mbox{ as } N \to \infty.$

Proof
For the first part we just note that $\int_{-\pi}^\pi {\rm e}^{2 \pi {\rm i} k x} \, \mathrm{d} x = 0$ for $k \neq 0$ and $1$ for $k = 0$. The second part follows from the first, since $F_N(x) = \frac{1}{N} \left(\frac{\sin(Nx/2)}{\sin(x/2)}\right)^2 \ge 0$.

To prove the third part, note that for $\delta > 0$ there exists a $c_\delta > 0$ such that $\sin(x/2) > c_\delta$ for all $\delta \le |x| \le \pi$. Hence $|F_N(x)| \le \frac{1}{N c^2_\delta}$ for all $\delta \le |x| \le \pi$ and therefore

$\displaystyle \int_{\delta \le |x| \le \pi} |F_N(x)| \, \mathrm{d} x \to 0 \mbox{ as } N \to \infty.$

Hence the result follows. $\Box$

Theorem Let $f:\mathbb{R} \to \mathbb{R}$ be continuous and periodic with period $2\pi$. Let the convolution of $f$ and $F_N$ be given by

$\displaystyle f \star F_N(x) = \frac{1}{2\pi} \int_{-\pi}^\pi f(x-y) F_N(y) \, \mathrm{d} x.$

Then

$\displaystyle \lim_{N\to \infty} \left\| f \star F_N - f \right\|_\infty = 0.$

Proof
Since $f$ is continuous it follows that $f$ is uniformly continuous on any bounded interval. For $\varepsilon > 0$ choose $\delta > 0$ such that for $|y|< \delta$ we have $|f(x-y)-f(x)| < \varepsilon$. Then, by the first property of the above lemma we have

$\displaystyle (f\star F_N)(x) - f(x) = \frac{1}{2\pi} \int_{-\pi}^\pi F_N(y) \left[f(x-y) - f(x) \right] \, \mathrm{d} x.$

Therefore,

$\displaystyle \begin{array}{rcl} 2\pi |(f\star F_N)(x) - f(x)| & \le & \int_{-\pi}^\pi F_N(y) |f(x-y)-f(x)| \, \mathrm{d} x \\ && \\ & \le & \int_{|y| < \delta} F_N(y) |f(x-y)-f(x)| \, \mathrm{d} x \\ && \\ && + \int_{\delta \le |y| \le \pi} F_N(y) |f(x-y) - f(x)| \, \mathrm{d} x \\ && \\ & \le & \varepsilon \int_{-\pi}^\pi F_N(y) \, \mathrm{d} x + 2 B \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y \\ && \\ & = & \varepsilon + 2 B \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y, \end{array}$

where $B = \max_{-\pi \le x \le \pi} |f(x)|$. Therefore

$\displaystyle |(f\star F_N)(x) - f(x)| \le \frac{\varepsilon}{2\pi} + \frac{B}{\pi} \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y.$

The result follows now by the third property of $F_N$. $\Box$

The function $f\star F_N$ is a trigonometric polynomial. Indeed, using the variable transformation $z = x-y$ we obtain

$\displaystyle \begin{array}{rcl} f\star F_N(x) & = & \frac{1}{2\pi} \int_{x-\pi}^{x+\pi} f(z) F_N(x-z) \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi} \int_{-\pi}^{\pi} f(z) F_N(x-z) \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi N} \sum_{m=0}^{N-1} \sum_{n=-m}^m \mathrm{e}^{2\pi {\rm i} n x} \int_{-\pi}^\pi f(z) \mathrm{e}^{-2\pi {\rm i} n z} \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi N} \sum_{m=0}^{N-1} \sum_{n=-m}^m a_n \mathrm{e}^{2 \pi {\rm i} n x}, \end{array}$

where the $a_n$ are the Fourier coefficients of $f$.

Hence the theorem above shows that a continuous, $2\pi$ periodic function $f$ can be uniformly approximated by a trigonometric polynomial. The result now follows from the best approximation lemma.

Lemma (Best approximation lemma)
If $f$ is integrable with Fourier coefficients $a_n$, then

$\displaystyle \|f- \sum_{n=-N}^N a_n \mathrm{e}^{2 \pi {\rm i} n x}\|_2 \le \|f - \sum_{n = -N}^N c_n {\rm e}^{2\pi {\rm i} n x}\|_2$

for any complex numbers $c_{-N}, \ldots, c_N$.

Proof
Let $a_{-N}, \ldots, a_N$ be Fourier coefficients of $f$ and set $b_n = a_n - c_n$ for $-N \le n \le N$. Then

$\displaystyle f - \sum_{n=-N}^N c_n \mathrm{e}^{2\pi {\rm i} n x} = f - S_N f + \sum_{n = - N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x}.$

Now we have

$\displaystyle \langle f-S_Nf, \sum_{n=-N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x} \rangle = \sum_{n=-N}^N b_n \left[\langle f, \mathrm{e}^{2\pi {\rm i} n x} \rangle - \langle S_N f, \mathrm{e}^{2\pi {\rm i} n x} \rangle\right].$

Since $\langle f, \mathrm{e}^{2 \pi {\rm i} n x} \rangle = a_{-n}$ and $\langle S_N f, \mathrm{e}^{2 \pi {\rm i} n x} \rangle = a_{-n}$, it follows that the inner product is $0$. Hence we can use the Pythagorean theorem to obtain

$\displaystyle \| f - \sum_{n=-N}^N c_n \mathrm{e}^{2\pi {\rm i} n x} \|_2 = \|f - S_N f \|_2 + \|\sum_{n = - N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x}\|_2 \ge \|f - S_N f\|_2.$

$\Box$

Corollary
Let $f:\mathbb{R} \to \mathbb{R}$ be $2 \pi$ periodic and continuous. Then

$\displaystyle \lim_{N\to \infty} \|f-S_Nf\|_2 = 0.$

Proof
The proof follows from the observation

$\displaystyle \|f-S_Nf\|_2 \le \|f- (f\star F_N)\|_2 \le \sqrt{2\pi} \|f-(f\star F_N)\|_\infty.$

$\Box$

The corollary states that trigonometric polynomials are dense in the space of continuous functions.

So far we have shown that the mean square convergence for Fourier series holds for continuous functions. To show that it also holds for merely integrable and bounded functions we need the following lemma.

Lemma
Suppose that $f:[a,b]\to \mathbb{R}$ is integrable and bounded by ${}B$. There there exists a sequence of continuous and periodic functions $\{f_k\}_{m\ge 1}$ with $f_m:[a,b]\to\mathbb{R}$ so that

$\displaystyle \sup_{x \in [a,b]} |f_k(x)| \le B \quad \mbox{for all } m = 1,2,\ldots$

and

$\displaystyle \int_a^b |f(x)-f_m(x)| \, \mathrm{d}x \to 0 \quad \mbox{as } m \to \infty.$

Proof
Given an integer $L \textgreater 0$, there is a partition of $[a,b]$ given by $a=x_{L,0}\textless x_{L,1}\textless \cdots \textless x_{L,N}=b$ such that the upper and lower Riemann sums of ${}f$ differ by at most $1/L$. Let

$\displaystyle f^\ast(x)=\sup_{x_{L,n-1}\le y \le x_{L,n}} f(y) \quad \mbox{if } x \in [x_{L,n-1},x_{L,n}] \mbox{ for } 1 \le n \le N.$

Thus we have $\sup_{x\in [a,b]} |f^\ast(x)|\le B$ and

$\displaystyle \int_a^b |f^\ast(x)-f(x)|\,\mathrm{d}x = \int_a^b (f^\ast(x)-f(x))\,\mathrm{d}x < \frac{1}{L}.$

We construct now functions $g_{L,k}$ in the following way. Let $K\in\mathbb{N}$ be large enough such that $1/K \textless \min_{1\le n\le N} |x_{n-1}-x_n|/2$. Then, for $k\ge K$ construct $g_{L,k}$ by setting

$\displaystyle g_{L,k}(x)=f^\ast(x) \quad \mbox{for } x\in [a,b]\setminus \bigcup_{0\le n \le N} (x_{L,n}-1/k,x_{L,n}+1/k).$

For $1 \le n\textless N$ and $x \in (x_{L,n}-1/k,x_{L,n}+1/k)$ let

$\displaystyle \begin{array}{rcl} g_{L,k}(x) &=& f^\ast(x_{L,n}-1/k) \\ && \\ && + (f^\ast(x_{L,n}+1/k)-f^\ast(x_{L,n}-1/k)) (x-x_{L,n}+1/k) k/2. \end{array}$

Further, for $x\in[a,a+1/k)$ let

$\displaystyle g_{L,k}(x)= f^\ast(a+1/k) (x-a)k$

and for $x \in (b-1/k,b]$ let

$\displaystyle g_{L,k}(x) = f^\ast(b-1/k) (b-x)k.$

For $1 \le k < K$ we set $g_{L,k}=g_{L,K}$.

Hence $g_{L,1},g_{L,2},\ldots$ is a sequence of continuous and periodic functions defined on $[a,b].$ Further, $g_{L,k}$ differs from $f^\ast$ only in the intervals $(x_{L,n}-1/k,x_{L,n}+1/k)$, and in those intervals by at most $2B$. Therefore

$\displaystyle \int_a^b |f^\ast(x)-g_{L,k}(x)|\,\mathrm{d} x \le 2BN \frac{2}{k}.$

By choosing $k\in\mathbb{N}$ large enough, say $k = k^\ast(L)$ we obtain

$\displaystyle \int_a^b |f^\ast(x)-g_{L,k^\ast(L)}(x)|\,\mathrm{d} x < \frac{1}{L}.$

Using the triangle inequality we obtain

$\displaystyle \int_a^b |f(x)-g_{L,k^\ast(L)}(x)|\, \mathrm{d} x < \frac{2}{L}.$

Now we define

$\displaystyle f_m(x)=g_{2m,k^\ast(2m)}(x) \quad\mbox{for } x\in[a,b] \mbox{ and } m = 1,2,\ldots,$

and extend $f_m$ periodically to $\mathbb{R}$. Then, $f_1,f_2,\ldots$ are periodic and continuous, and, by the above construction,

$\displaystyle \int_a^b |f(x)- f_m(x)|\,\mathrm{d}x < \frac{1}{m}$

which proves the result.
$\Box$

Theorem
Let $f:\mathbb{R} \to \mathbb{R}$ be $2 \pi$ periodic, integrable and bounded. Then

$\displaystyle \lim_{N\to \infty} \|f-S_Nf\|_2 = 0.$

Proof
Let ${}f$ be $2\pi$ periodic, bounded and integrable. Let $\varepsilon\textgreater 0$. Then, by the above lemma, there exists a $2\pi$ periodic, continuous function ${}g$ such that

$\displaystyle \sup_{x\in[-\pi,\pi]} |g(x)| \le \sup_{x\in[-\pi,\pi]} |f(x)| = B,$

and

$\displaystyle \int_{-\pi}^\pi |f(x)-g(x)|\,\mathrm{d}x < \frac{\varepsilon^2}{8B}.$

Hence we get

$\displaystyle \begin{array}{rcl} \|f-g\|^2_{2} &= &\int_{-\pi}^\pi |f(x)-g(x)| |f(x)-g(x)|\,\mathrm{d}x \\ &&\\ &\le & 2B\int_{-\pi}^\pi |f(x)-g(x)|\,\mathrm{d}x <\frac{\varepsilon^2}{4}. \end{array}$

By the corollary above, there exists a trigonometric polynomial $P$ such that $\|g-P\|_2< \varepsilon/2$ and hence, using the triangle inequality,

$\displaystyle \|f-P\|_2=\|f-g+g-P\| \le \|f-g\|_2 + \|g-P\|_2 < \varepsilon.$

By the best approximation lemma it follows that

$\displaystyle \|f-S_Nf\|_2 \le \|f-P\|_2 < \varepsilon$

where ${}N$ is the degree of ${}P$. Hence the result follows.
$\Box$

For more information see for example E.M. Stein and R. Shakarchi, Princeton lectures in analysis I, Fourier analysis. Princeton University Press, Princeton, 2003. See also the Riesz-Fischer theorem and the Dini test.