Math2111: Chapter 1: Fourier series. Additional Material: L_2 convergence of Fourier series.

In this blog entry you can find lecture notes for Math2111, several variable calculus. See also the table of contents for this course. This blog entry printed to pdf is available here.

In this post I present some ideas which shed light on the question why one can expect the Fourier series to converge to the function (under certain assumptions).

Complete orthonormal systems

To do so let us first study a simpler case, one with which you are familiar with. (Throughout this section I will ignore convergence questions.)

Consider the vector space \mathbb{R}^n. For two vectors \boldsymbol{u} = (u_1,\ldots, u_n)^\top and \boldsymbol{v} = (v_1,\ldots, v_n)^\top we define the inner product

\displaystyle \langle \boldsymbol{u}, \boldsymbol{v} \rangle = u_1 v_1 + \cdots + u_n v_n.

Assume we have given a set of orthonormal vectors \boldsymbol{u}_1, \ldots, \boldsymbol{u}_k, that is, we have

\displaystyle \langle \boldsymbol{u}_k, \boldsymbol{u}_l \rangle = \left\{ \begin{array}{rl} 0 & \mbox{if } k \neq l, \\ & \\ 1 & \mbox{if } k = l. \end{array} \right.

Now the question arises whether for every vector \boldsymbol{v} \in \mathbb{R}^n there are \lambda_1, \ldots, \lambda_k \in \mathbb{R} such that

\displaystyle \boldsymbol{v} = \lambda_1 \boldsymbol{u}_1 + \cdots + \lambda_k \boldsymbol{u}_k?

The answer is of course: it depends. If k = n, then yes, if k \textless n then no. (Notice that k \textgreater n is not possible.)

Consider now \ell^2(\mathbb{Z}) equipped with the standard inner product (defined here). Then let us ask the same question. Assume we have given a set of orthonormal vectors \boldsymbol{u}_1, \ldots, \boldsymbol{u}_k, that is, we have

\displaystyle \langle \boldsymbol{u}_k, \boldsymbol{u}_l \rangle = \left\{ \begin{array}{rl} 0 & \mbox{if } k \neq l, \\ & \\ 1 & \mbox{if } k = l. \end{array} \right.

Is it true that for every vector \boldsymbol{v} \in \ell^2(\mathbb{Z}) there are \lambda_1, \ldots, \lambda_k \in \mathbb{R} such that

\displaystyle \boldsymbol{v} = \lambda_1 \boldsymbol{u}_1 + \cdots + \lambda_k \boldsymbol{u}_k?

The answer is again it depends. If k \textless \infty, then certainly not because \ell^2(\mathbb{Z}) is infinite dimensional. If k=\infty, then we still do not know. Why? To illustrate this, let A_k be the vector whose entries are all 0 except the entry at position {}k is {}1. More precisely, for k \in \mathbb{Z} let

\displaystyle A_k = (\ldots, b_{2,k}, b_{1,k}, a_{0,k}, a_{1,k}, a_{2,k}, \ldots)

where we set

\displaystyle \begin{array}{rcl} b_{n,k} & = & 0 \quad \mbox{if } n \neq -k, \\ && \\ b_{n,k} & = & 1 \quad \mbox{if } n = -k, \\ && \\ a_{n,k} & = & 0 \quad \mbox{if } n \neq k, \\ && \\ a_{n,k} & = & 1 \quad \mbox{if } n = k. \end{array}

Choose now an arbitrary element

\displaystyle C = (\ldots, d_2, d_1, c_0, c_1, c_2, \ldots) \in \ell^2(\mathbb{Z}).

In general, which vectors A_k do we need to represent C? The answer is: all of them. Since we have

\displaystyle C = \sum_{k=1}^\infty d_k A_{-k} + \sum_{k=0}^\infty c_k A_k.

If we consider for instance the set of vectors

\displaystyle \mathscr{A} = \{A_k \in \ell^2(\mathbb{Z}): k \in \mathbb{Z} \setminus \{23\}\}

then the element A_{23} \in \ell^2(\mathbb{Z}) cannot be represented using the elements in \mathscr{A}. In other words, A_{23} is missing from the set \mathscr{A}.

Hence, for an infinite dimensional vector space it is not enough to have infinitely many orthonormal vectors in order to be able to represent any element. Notice that, if we have an arbitrary infinite set of orthonormal vectors

\displaystyle \mathscr{B} =\{ \ldots, B_{-2}, B_{-1}, B_0, B_1, B_2, \ldots \}

and a vector A which cannot be represented by the vectors in \mathscr{B}, then there exists a vector B such that

\langle B, B_k\rangle = 0 \quad \mbox{for all } k \in \mathbb{Z}.

Indeed, we can define the vector B by

\displaystyle B = A - \sum_{k \in \mathbb{Z}} \langle A, B_k \rangle B_k.

Since A cannot be represented by the elements in \mathscr{B}, it follows that B is not the zero vector. Further

\displaystyle \begin{array}{rcl} \langle B, B_n \rangle & = & \langle A, B_n \rangle - \sum_{k=-\infty}^\infty \langle A, B_k \rangle \langle B_k, B_n \rangle \\ && \\ & = & \langle A, B_n \rangle - \langle A, B_n \rangle \langle B_n, B_n \rangle = 0, \end{array}

since we have chosen the elements \ldots, B_{-2}, B_{-1}, B_0, B_1, B_2, \ldots such that

\displaystyle \langle B_k, B_n \rangle = \left\{\begin{array}{rl} 1 & \mbox{if } k = n, \\ & \\ 0 & \mbox{if } k \neq n. \end{array} \right.

Now let us consider Fourier series. Let us ignore questions concerning convergence in the following. For instance, say, let us only consider continuously differentiable functions. The same argument as for \ell^2(\mathbb{Z}) also applies in this case. If there would be a function f for which we do not have

\displaystyle f(x) = \frac{a_0}{2} + \sum_{k=1}^\infty \left[a_k \cos kx + b_k \sin kx \right]

then there would have to be a (continuously differentiable), 2\pi-periodic function \phi:\mathbb{R} \to \mathbb{R} such that

\displaystyle \begin{array}{rll} \langle \phi, \cos kx \rangle & = \int_{-\pi}^\pi \phi(x) \cos kx \, \mathrm{d} x & = 0 \quad \mbox{for all } k = 0, 1, 2, \ldots \mbox{and } \\ && \\ \langle \phi, \cos kx \rangle & = \int_{-\pi}^\pi \phi(x) \sin kx \, \mathrm{d} x & = 0 \quad \mbox{for all } k = 0, 1, 2, \ldots . \end{array}

Therefore every (continuously differentiable), 2\pi periodic function can be represented by its Fourier series if and only if the functions

\displaystyle \cos k x, \quad k = 0, 1, 2, \ldots \quad \mbox{and } \sin kx, \quad k = 1,2, \ldots \quad (1)

are a complete orthonormal system, that is, there is no (continuously differentiable), 2\pi periodic function \phi:\mathbb{R} \to \mathbb{R} which is orthogonal to all trigonometric functions given in line (1).

This does indeed hold, which follows from the convergence of the Fourier series to the function in the mean square sense. We prove this result in the next section.

\rhd Mean square convergence

We prove now that the Fourier series of f converges to f in the mean square sense. That is,

\displaystyle \lim_{N \to \infty} \|S_Nf- f\|_2 = 0,

where S_N f is the N th partial sum of the Fourier series and the L_2-norm is defined by

\displaystyle \|f\|_2 = \int_{-\pi}^\pi |f(x)|^2 \, \mathrm{d} x.

To prove this we need some preparation. Instead of using \cos kx for k = 0,1,2, \ldots and \sin kx for k = 1,2, \ldots we use in the following the functions \mathrm{e}^{2 \pi \mathrm{i} k x} where k \in \mathbb{Z}. For complex valued functions f, g: [-\pi, \pi] \to \mathbb{C}, we define the inner product by

\displaystyle \langle f, g \rangle = \int_{-\pi}^\pi f(x) \overline{g(x)} \, \mathrm{d} x,

where \overline{c} stands for the complex conjugate of a complex number c \in \mathbb{C}. Hence, we consider the Fourier series

\displaystyle Sf(x) = \sum_{k=-\infty}^\infty a_k \mathrm{e}^{2 \pi {\rm i} k x},

where the Fourier coefficients are given by

\displaystyle a_k = \frac{1}{\pi} \int_{-\pi}^\pi f(x) \mathrm{e}^{-2\pi {\rm i} k x} \, \mathrm{d} x. \qquad\qquad\qquad (2)

Further, the Nth partial sum of the Fourier series is given by

\displaystyle S_Nf(x) = \sum_{k=-N}^N a_k \mathrm{e}^{2 \pi {\rm i} k x},

Exercise Show that (2) holds if f is a trigonometric polynomial f(x) = \sum_{n=-N}^N a_k \, \mathrm{e}^{2 \pi {\rm i} k x}.

Definition (Dirichlet kernel) The trigonometric polynomial defined for x \in [-\pi, \pi] by

\displaystyle D_N(x) = \sum_{n=-N}^N {\rm e}^{2 \pi {\rm i} n x}

is called the Nth Dirichlet kernel.

Exercise Show that

\displaystyle D_N(x) = \frac{\sin ((N+1/2) x)}{\sin (x/2)} .

Definition (Fejér kernel) The trigonometric polynomial defined for x \in [-\pi, \pi] by

\displaystyle F_N(x) = \frac{1}{N} \sum_{n= 0}^{N-1} D_n(x)

is called the Nth Fejér kernel.

Exercise Show that

\displaystyle F_N(x) = \frac{1}{N} \frac{\sin^2 (Nx/2)}{\sin^2 (x/2)} .

Lemma
The Fejér kernel satisfies the following properties:

  1. For all N \ge 1 we have

    \displaystyle \frac{1}{2\pi} \int_{-\pi}^\pi F_N(x) \, \mathrm{d} x = 1.

  2. There exists an M > 0 such that for all N \ge 1,

    \displaystyle \int_{-\pi}^\pi |F_N(x)|\, \mathrm{d} x \le M.

  3. For every \delta > 0,

    \displaystyle \int_{\delta \le |x| \le \pi} |F_N(x)| \, \mathrm{d} x \to 0 \mbox{ as } N \to \infty.

Proof
For the first part we just note that \int_{-\pi}^\pi {\rm e}^{2 \pi {\rm i} k x} \, \mathrm{d} x = 0 for k \neq 0 and 1 for k = 0. The second part follows from the first, since F_N(x) = \frac{1}{N} \left(\frac{\sin(Nx/2)}{\sin(x/2)}\right)^2 \ge 0.

To prove the third part, note that for \delta > 0 there exists a c_\delta > 0 such that \sin(x/2) > c_\delta for all \delta \le |x| \le \pi. Hence |F_N(x)| \le \frac{1}{N c^2_\delta} for all \delta \le |x| \le \pi and therefore

\displaystyle \int_{\delta \le |x| \le \pi} |F_N(x)| \, \mathrm{d} x \to 0 \mbox{ as } N \to \infty.

Hence the result follows. \Box

Theorem Let f:\mathbb{R} \to \mathbb{R} be continuous and periodic with period 2\pi. Let the convolution of f and F_N be given by

\displaystyle f \star F_N(x) = \frac{1}{2\pi} \int_{-\pi}^\pi f(x-y) F_N(y) \, \mathrm{d} x.

Then

\displaystyle \lim_{N\to \infty} \left\| f \star F_N - f \right\|_\infty = 0.

Proof
Since f is continuous it follows that f is uniformly continuous on any bounded interval. For \varepsilon > 0 choose \delta > 0 such that for |y|< \delta we have |f(x-y)-f(x)| < \varepsilon. Then, by the first property of the above lemma we have

\displaystyle (f\star F_N)(x) - f(x) = \frac{1}{2\pi} \int_{-\pi}^\pi F_N(y) \left[f(x-y) - f(x) \right] \, \mathrm{d} x.

Therefore,

\displaystyle \begin{array}{rcl} 2\pi |(f\star F_N)(x) - f(x)| & \le & \int_{-\pi}^\pi F_N(y) |f(x-y)-f(x)| \, \mathrm{d} x \\ && \\ & \le & \int_{|y| < \delta} F_N(y) |f(x-y)-f(x)| \, \mathrm{d} x \\ && \\ && + \int_{\delta \le |y| \le \pi} F_N(y) |f(x-y) - f(x)| \, \mathrm{d} x \\ && \\ & \le & \varepsilon \int_{-\pi}^\pi F_N(y) \, \mathrm{d} x + 2 B \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y \\ && \\ & = & \varepsilon + 2 B \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y, \end{array}

where B = \max_{-\pi \le x \le \pi} |f(x)|. Therefore

\displaystyle |(f\star F_N)(x) - f(x)| \le \frac{\varepsilon}{2\pi} + \frac{B}{\pi} \int_{\delta \le |y| \le \pi} F_N(y) \, \mathrm{d} y.

The result follows now by the third property of F_N. \Box

The function f\star F_N is a trigonometric polynomial. Indeed, using the variable transformation z = x-y we obtain

\displaystyle \begin{array}{rcl} f\star F_N(x) & = & \frac{1}{2\pi} \int_{x-\pi}^{x+\pi} f(z) F_N(x-z) \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi} \int_{-\pi}^{\pi} f(z) F_N(x-z) \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi N} \sum_{m=0}^{N-1} \sum_{n=-m}^m \mathrm{e}^{2\pi {\rm i} n x} \int_{-\pi}^\pi f(z) \mathrm{e}^{-2\pi {\rm i} n z} \, \mathrm{d} z \\ && \\ & = & \frac{1}{2\pi N} \sum_{m=0}^{N-1} \sum_{n=-m}^m a_n \mathrm{e}^{2 \pi {\rm i} n x}, \end{array}

where the a_n are the Fourier coefficients of f.

Hence the theorem above shows that a continuous, 2\pi periodic function f can be uniformly approximated by a trigonometric polynomial. The result now follows from the best approximation lemma.

Lemma (Best approximation lemma)
If f is integrable with Fourier coefficients a_n, then

\displaystyle \|f- \sum_{n=-N}^N a_n \mathrm{e}^{2 \pi {\rm i} n x}\|_2 \le \|f - \sum_{n = -N}^N c_n {\rm e}^{2\pi {\rm i} n x}\|_2

for any complex numbers c_{-N}, \ldots, c_N.

Proof
Let a_{-N}, \ldots, a_N be Fourier coefficients of f and set b_n = a_n - c_n for -N \le n \le N. Then

\displaystyle f - \sum_{n=-N}^N c_n \mathrm{e}^{2\pi {\rm i} n x} = f - S_N f + \sum_{n = - N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x}.

Now we have

\displaystyle \langle f-S_Nf, \sum_{n=-N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x} \rangle = \sum_{n=-N}^N b_n \left[\langle f, \mathrm{e}^{2\pi {\rm i} n x} \rangle - \langle S_N f, \mathrm{e}^{2\pi {\rm i} n x} \rangle\right].

Since \langle f, \mathrm{e}^{2 \pi {\rm i} n x} \rangle = a_{-n} and \langle S_N f, \mathrm{e}^{2 \pi {\rm i} n x} \rangle = a_{-n}, it follows that the inner product is 0. Hence we can use the Pythagorean theorem to obtain

\displaystyle \| f - \sum_{n=-N}^N c_n \mathrm{e}^{2\pi {\rm i} n x} \|_2 = \|f - S_N f \|_2 + \|\sum_{n = - N}^N b_n \mathrm{e}^{2 \pi {\rm i} n x}\|_2 \ge \|f - S_N f\|_2.

\Box

Corollary
Let f:\mathbb{R} \to \mathbb{R} be 2 \pi periodic and continuous. Then

\displaystyle \lim_{N\to \infty} \|f-S_Nf\|_2 = 0.

Proof
The proof follows from the observation

\displaystyle \|f-S_Nf\|_2 \le \|f- (f\star F_N)\|_2 \le \sqrt{2\pi} \|f-(f\star F_N)\|_\infty.

\Box

The corollary states that trigonometric polynomials are dense in the space of continuous functions.

So far we have shown that the mean square convergence for Fourier series holds for continuous functions. To show that it also holds for merely integrable and bounded functions we need the following lemma.

Lemma
Suppose that f:[a,b]\to \mathbb{R} is integrable and bounded by {}B. There there exists a sequence of continuous and periodic functions \{f_k\}_{m\ge 1} with f_m:[a,b]\to\mathbb{R} so that

\displaystyle \sup_{x \in [a,b]} |f_k(x)| \le B \quad \mbox{for all } m = 1,2,\ldots

and

\displaystyle \int_a^b |f(x)-f_m(x)| \, \mathrm{d}x \to 0 \quad \mbox{as } m \to \infty.

Proof
Given an integer L \textgreater 0, there is a partition of [a,b] given by a=x_{L,0}\textless x_{L,1}\textless \cdots \textless x_{L,N}=b such that the upper and lower Riemann sums of {}f differ by at most 1/L. Let

\displaystyle f^\ast(x)=\sup_{x_{L,n-1}\le y \le x_{L,n}} f(y) \quad \mbox{if } x \in [x_{L,n-1},x_{L,n}] \mbox{ for } 1 \le n \le N.

Thus we have \sup_{x\in [a,b]} |f^\ast(x)|\le B and

\displaystyle \int_a^b |f^\ast(x)-f(x)|\,\mathrm{d}x = \int_a^b (f^\ast(x)-f(x))\,\mathrm{d}x < \frac{1}{L}.

We construct now functions g_{L,k} in the following way. Let K\in\mathbb{N} be large enough such that 1/K \textless \min_{1\le n\le N} |x_{n-1}-x_n|/2. Then, for k\ge K construct g_{L,k} by setting

\displaystyle g_{L,k}(x)=f^\ast(x) \quad \mbox{for } x\in [a,b]\setminus \bigcup_{0\le n \le N} (x_{L,n}-1/k,x_{L,n}+1/k).

For 1 \le n\textless N and x \in (x_{L,n}-1/k,x_{L,n}+1/k) let

\displaystyle \begin{array}{rcl} g_{L,k}(x) &=& f^\ast(x_{L,n}-1/k) \\ && \\ && + (f^\ast(x_{L,n}+1/k)-f^\ast(x_{L,n}-1/k)) (x-x_{L,n}+1/k) k/2. \end{array}

Further, for x\in[a,a+1/k) let

\displaystyle g_{L,k}(x)= f^\ast(a+1/k) (x-a)k

and for x \in (b-1/k,b] let

\displaystyle g_{L,k}(x) = f^\ast(b-1/k) (b-x)k.

For 1 \le k < K we set g_{L,k}=g_{L,K}.

Hence g_{L,1},g_{L,2},\ldots is a sequence of continuous and periodic functions defined on [a,b]. Further, g_{L,k} differs from f^\ast only in the intervals (x_{L,n}-1/k,x_{L,n}+1/k), and in those intervals by at most 2B. Therefore

\displaystyle \int_a^b |f^\ast(x)-g_{L,k}(x)|\,\mathrm{d} x \le 2BN \frac{2}{k}.

By choosing k\in\mathbb{N} large enough, say k = k^\ast(L) we obtain

\displaystyle \int_a^b |f^\ast(x)-g_{L,k^\ast(L)}(x)|\,\mathrm{d} x < \frac{1}{L}.

Using the triangle inequality we obtain

\displaystyle \int_a^b |f(x)-g_{L,k^\ast(L)}(x)|\, \mathrm{d} x < \frac{2}{L}.

Now we define

\displaystyle f_m(x)=g_{2m,k^\ast(2m)}(x) \quad\mbox{for } x\in[a,b] \mbox{ and } m = 1,2,\ldots,

and extend f_m periodically to \mathbb{R}. Then, f_1,f_2,\ldots are periodic and continuous, and, by the above construction,

\displaystyle \int_a^b |f(x)- f_m(x)|\,\mathrm{d}x < \frac{1}{m}

which proves the result.
\Box

Theorem
Let f:\mathbb{R} \to \mathbb{R} be 2 \pi periodic, integrable and bounded. Then

\displaystyle \lim_{N\to \infty} \|f-S_Nf\|_2 = 0.

Proof
Let {}f be 2\pi periodic, bounded and integrable. Let \varepsilon\textgreater 0. Then, by the above lemma, there exists a 2\pi periodic, continuous function {}g such that

\displaystyle \sup_{x\in[-\pi,\pi]} |g(x)| \le \sup_{x\in[-\pi,\pi]} |f(x)| = B,

and

\displaystyle \int_{-\pi}^\pi |f(x)-g(x)|\,\mathrm{d}x < \frac{\varepsilon^2}{8B}.

Hence we get

\displaystyle \begin{array}{rcl} \|f-g\|^2_{2} &= &\int_{-\pi}^\pi |f(x)-g(x)| |f(x)-g(x)|\,\mathrm{d}x \\ &&\\ &\le & 2B\int_{-\pi}^\pi |f(x)-g(x)|\,\mathrm{d}x <\frac{\varepsilon^2}{4}. \end{array}

By the corollary above, there exists a trigonometric polynomial P such that \|g-P\|_2< \varepsilon/2 and hence, using the triangle inequality,

\displaystyle \|f-P\|_2=\|f-g+g-P\| \le \|f-g\|_2 + \|g-P\|_2 < \varepsilon.

By the best approximation lemma it follows that

\displaystyle \|f-S_Nf\|_2 \le \|f-P\|_2 < \varepsilon

where {}N is the degree of {}P. Hence the result follows.
\Box

For more information see for example E.M. Stein and R. Shakarchi, Princeton lectures in analysis I, Fourier analysis. Princeton University Press, Princeton, 2003. See also the Riesz-Fischer theorem and the Dini test.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s