Math2111: Chapter 1: Fourier series. Section 2: Inner product and norm

Posted on April 20, 2010 | 2 Comments

In this blog entry you can find lecture notes for Math2111, several variable calculus. See also the table of contents for this course. This blog entry printed to pdf is available here.

We repeat two fundamental concepts which you should have seen in linear algebra already.
Inner product and norm in \small \mathbb{R}^n
Let \boldsymbol{u}, \boldsymbol{v} \in \mathbb{R}^n be vectors with

\displaystyle \boldsymbol{u} = (u_1, \ldots, u_n)^\top, \quad \boldsymbol{v} = (v_1,\ldots, v_n)^\top

where (u_1,\ldots, u_n)^\top stands for the transpose of the vector (u_1,\ldots, u_n).

Then the dot product of these vectors is defined by

\displaystyle \boldsymbol{u} \cdot \boldsymbol{v} = u_1 v_1 + \cdots + u_n v_n.

The dot product of two vectors has a nice geometrical interpretation and is useful in a range of problems.

In fact, the dot product is an example of an inner product and has the following properties:

  1. \boldsymbol{u} \cdot \boldsymbol{u} \ge 0
  2. \boldsymbol{u} \cdot \boldsymbol{u} = 0 if and only if \boldsymbol{u} =\boldsymbol{0}
  3. (\lambda \boldsymbol{u} + \mu \boldsymbol{w}) \cdot \boldsymbol{v} = \lambda (\boldsymbol{u} \cdot \boldsymbol{v}) + \mu (\boldsymbol{w} \cdot \boldsymbol{v})
  4. \boldsymbol{u} \cdot \boldsymbol{v} = \boldsymbol{v} \cdot \boldsymbol{u}

Notice that for a given vector \boldsymbol{u} \in \mathbb{R}^n we can interpret the quantity \sqrt{\boldsymbol{u} \cdot \boldsymbol{u}} as the length of the vector \boldsymbol{u}. By property 1. \sqrt{\boldsymbol{u} \cdot\boldsymbol{u}} is well defined and, by 2. the length of \boldsymbol{u} is zero if and only if \boldsymbol{u} = \boldsymbol{0}

Another useful property is the following. Let \boldsymbol{u}_1, \ldots, \boldsymbol{u}_n \in \mathbb{R}^n be an orthonormal bases, that is

  1. \boldsymbol{u}_k \cdot \boldsymbol{u}_k = 1 for 1 \le k \le n,
  2. \boldsymbol{u}_k \cdot \boldsymbol{u}_l = 0 for 1 \le k < l \le n

Let \boldsymbol{v} \in \mathbb{R}^n and suppose we want to find \mu_1,\ldots, \mu_n such that

\displaystyle \boldsymbol{v} = \mu_1 \boldsymbol{u}_1 + \cdots + \mu_n \boldsymbol{u}_n

Then the following calculation shows how this can be done:

\displaystyle \begin{array}{lcl} \boldsymbol{v} \cdot \boldsymbol{u}_k & = & (\mu_1 \boldsymbol{u}_1 + \cdots + \mu_n \boldsymbol{u}_n) \cdot \boldsymbol{u}_k \\ && \\ & = & \mu_1 \boldsymbol{u}_1 \cdot \boldsymbol{u}_k + \cdots + \mu_n \boldsymbol{u}_n \cdot \boldsymbol{u}_k \\ && \\ & = & \mu_k \boldsymbol{u}_k \cdot \boldsymbol{u}_k = \mu_k. \end{array}

Hence we have the convenient formula:

\displaystyle \mu_k = \boldsymbol{v} \cdot \boldsymbol{u}_k.

Note that this formula holds whenever the four properties of the dot product mentioned above hold and when the vectors \boldsymbol{u}_1,\ldots, \boldsymbol{u}_n form an orthonormal bases.

Instead of \mathbb{R}^n we consider now more general vector spaces (in particular infinite dimensional vector spaces).

Inner product and norm in vector spaces

We consider now general vector spaces V over \mathbb{R}.

Definition
Let V be a vector space over \mathbb{R}. Then a function \langle \cdot, \cdot \rangle: V \times V \to \mathbb{R} is called an inner product if the following properties hold for all u, v, w \in V and all \lambda, \mu \in \mathbb{R}:

  1. \langle u, u \rangle \ge 0
  2. \langle u, u \rangle = 0 if and only if u = 0 (the zero vector)
  3. \langle \lambda u + \mu v, w \rangle = \lambda \langle u, w \rangle + \mu \langle v, w \rangle
  4. \langle u, v \rangle = \langle v, u\rangle

We call two vectors \small u, v \in V orthogonal if and only if \small \langle u, v \rangle = 0. This notion originates from our understanding of vectors in \small \mathbb{R}^2.

Exercise (T) Show that the inner product is also linear in the second component, that is,

\displaystyle \langle u, \lambda v + \mu w \rangle = \lambda \langle u, v\rangle + \mu \langle u, w \rangle.\Box

In the following we define a norm in a vector space.

Definition
Let V be a vector space over \mathbb{R}. Then a function \| \cdot \|: V \to \mathbb{R} is called a norm if the following properties hold for all u, v \in V and all \lambda \in \mathbb{R}:

  1. \| u \| \ge 0
  2. \| u \| = 0 if and only if u = 0 (the zero vector)
  3. \| \lambda u \| = |\lambda| \|u\|
  4. \| u + v \| \le \|u\| + \|v\|

The following results are of importance.

Theorem
Let V be a vector space over \mathbb{R} with inner product \langle \cdot, \cdot \rangle. Let \|u\| = \sqrt{\langle u, u \rangle}. Then we have

  1. Pythagorean theorem: If u, v \in V are orthogonal, then

    \displaystyle \|u+v\|^2 = \|u\|^2 + \|v\|^2;

  2. Cauchy-Schwarz inequality: for any u, v \in V we have

    \displaystyle |\langle u, v \rangle| \le \|u\| \|v\|;

  3. Triangle inequality: for any u, v \in V we have

    \displaystyle \|u+v\| \le \|u\| + \|v\|;

In particular, \|u\| = \sqrt{\langle u, u\rangle} defines a norm.

Exercise (T)
For the vector space \mathbb{R}^2 we can define a norm by

\|\boldsymbol{u}\| = length of the vector \boldsymbol{u}.

Show that all the properties of a norm are satisfied. \Box

Exercise (T)
For the vector space C([a,b]) of all continuous functions we can define a norm by

\|f\| = \max_{a \le x \le b} |f(x)|.

Show that all the properties of a norm are satisfied. (Note that the maximum is well defined by the extreme value theorem.) \Box

Exercise (T)
Consider now the space \mathcal{B}([a,b]) of bounded functions f:[a,b] \to \mathbb{R}, that is, for each f \in \mathcal{B}([a,b]) there is a constant such that

\displaystyle |f(x)| \le K < \infty \quad \mbox{for all } a \le x \le b.

Since the functions are not required to be continuous, it follows that they might not have a maximum. In this case we can use the least upper bound or supremum

\displaystyle \|f\| = \sup_{a \le x \le b} |f(x)|.

Show that this defines a norm on \mathcal{B}([a,b]). \Box

We give an example of an infinite dimensional space.

Exercise (T)
Let \ell^2(\mathbb{Z}) be the set of all sequences of real numbers

\displaystyle A = (\ldots, b_2, b_1, a_0, a_1, a_2, \ldots)

such that

\displaystyle \sum_{n=0}^\infty |a_n|^2 + \sum_{n=1}^\infty |b_n|^2 < \infty

For vectors A = (\ldots, b_2, b_1, a_0, a_1,a_2, \ldots), \quad A^\prime = (\ldots, b_2^\prime, b^\prime_1, a_0^\prime, a_1^\prime, a_2^\prime, \ldots) and \lambda \in \mathbb{R} let

\displaystyle A + A^\prime = (\ldots, b_2 + b_2^\prime, b_1 + b_1^\prime, a_0 + a_0^\prime, a_1 + a_1^\prime, a_2 + a_2^\prime, \ldots),

\displaystyle \lambda A = (\ldots, \lambda b_2, \lambda b_1, \lambda a_0, \lambda a_1, \lambda a_2, \ldots).

We define an inner product by

\displaystyle \langle A, A' \rangle = \sum_{n=0}^\infty a_n a_n^\prime + \sum_{n=1}^\infty b_n b_n^\prime.

Show that \ell^2(\mathbb{Z}) is a vector space and that \langle \cdot, \cdot \rangle defines an inner product.

We have seen that the dot product of vectors in the vector space \mathbb{R}^n is an inner product. We provide another example which is closer to the our ultimate goal of calculating the Fourier coefficients.

Example
Let now C([a,b]) be the vector space of continuous functions f:[a, b] \to \mathbb{R} over \mathbb{R}. Then we can define an inner product for functions f, g \in C([a,b]) by

\displaystyle \langle f, g \rangle = \int_a^b f(x) g(x) \, \mathrm{d} x.

Exercise Show that the inner product in the example above is indeed an inner product. \Box

An essential step towards finding the a_k, b_l, where k \ge 0 and l \ge 1 is given by the following example.

Exercise
Show that the functions

\displaystyle \cos (k x), \quad k = 0, 1, 2, \ldots,

\displaystyle \sin (k x), \quad k = 0, 1, 2, \ldots,

are orthogonal with respect to the inner product

\displaystyle \langle f, g \rangle = \int_{-\pi}^\pi f(x) g(x) \,\mathrm{d} x.

Further show that

\displaystyle \langle 1, 1 \rangle = 2\pi, \quad \langle \cos (k x), \cos(k x) \rangle = \pi, \quad \langle \sin (l x), \sin (l x) \rangle = \pi

for all integers k, l \ge 1. Hence find an infinite set of orthonormal functions.

The above example shows, in particular, that the space C([-\pi, \pi]) is infinite dimensional, since we have found in infinite set of functions which are orthonormal.

Definition (Trigonometric polynomial of degree n)
A trigonometric polynomial of degree n with period 2 \pi is a function of the form

\displaystyle f(x) = \frac{a_0}{2} + \sum_{k=1}^n \left[a_k \cos (kx) + b_k \sin (kx) \right]

for some a_0, a_1, b_1, a_2, b_2, \ldots \in \mathbb{R} with a_n or b_n nonzero.

The following result now follows.

Theorem
If f is a trigonometric polynomial of finite degree n with period 2 \pi then its Fourier coefficients are given by

\displaystyle a_k = \frac{1}{\pi} \int_{-\pi}^\pi f(x) \cos (kx) \, \mathrm{d} x \quad \mbox{for } 0 \le k \le n,

\displaystyle b_k = \frac{1}{\pi} \int_{-\pi}^\pi f(x) \sin (kx) \, \mathrm{d} x \quad \mbox{for } 1 \le k \le n.

In the infinite case one needs to take convergence considerations into account which we consider subsequently. For example the following question arises.

Question (T) State a condition on the function f:[-\pi,\pi] \to \mathbb{R} do we have

\displaystyle \left|\int_{-\pi}^\pi f(x) \cos (kx) \, \mathrm{d} x \right| < \infty

and

\displaystyle \left|\int_{-\pi}^\pi f(x) \sin (kx) \, \mathrm{d} x \right| < \infty

for all k = 0, 1, 2, \ldots?

This entry was posted in Math2111 Higher Several Variable Calculus, Teaching and tagged , , , , , . Bookmark the permalink.

2 responses to “Math2111: Chapter 1: Fourier series. Section 2: Inner product and norm”

  1. math2111 student | April 21, 2010 at 11:36 am | Reply

    Can you post the notes by themselves somewhere else? It’s too hard to print with the same formatting otherwise.

  2. math2111 student2 | April 21, 2010 at 11:38 am | Reply

    there is a lot of latex format in the notes, which cannot be displayed properly…
    Can you compile the notes and put up a pdf file? Thanks

Leave a reply to math2111 student2 Cancel reply