We repeat two fundamental concepts which you should have seen in linear algebra already.
Inner product and norm in
Let be vectors with
where stands for the transpose of the vector .
Then the dot product of these vectors is defined by
The dot product of two vectors has a nice geometrical interpretation and is useful in a range of problems.
In fact, the dot product is an example of an inner product and has the following properties:
- if and only if
Notice that for a given vector we can interpret the quantity as the length of the vector . By property 1. is well defined and, by 2. the length of is zero if and only if
Another useful property is the following. Let be an orthonormal bases, that is
Let and suppose we want to find such that
Then the following calculation shows how this can be done:
Hence we have the convenient formula:
Note that this formula holds whenever the four properties of the dot product mentioned above hold and when the vectors form an orthonormal bases.
Instead of we consider now more general vector spaces (in particular infinite dimensional vector spaces).
Inner product and norm in vector spaces
We consider now general vector spaces over
Let be a vector space over . Then a function is called an inner product if the following properties hold for all and all :
- if and only if (the zero vector)
We call two vectors orthogonal if and only if . This notion originates from our understanding of vectors in .
Exercise (T) Show that the inner product is also linear in the second component, that is,
In the following we define a norm in a vector space.
Let be a vector space over . Then a function is called a norm if the following properties hold for all and all :
- if and only if (the zero vector)
The following results are of importance.
Let be a vector space over with inner product Let . Then we have
- Pythagorean theorem: If are orthogonal, then
- Cauchy-Schwarz inequality: for any we have
- Triangle inequality: for any we have
In particular, defines a norm.
For the vector space we can define a norm by
length of the vector
Show that all the properties of a norm are satisfied.
For the vector space of all continuous functions we can define a norm by
Show that all the properties of a norm are satisfied. (Note that the maximum is well defined by the extreme value theorem.)
Consider now the space of bounded functions , that is, for each there is a constant such that
Since the functions are not required to be continuous, it follows that they might not have a maximum. In this case we can use the least upper bound or supremum
Show that this defines a norm on
We give an example of an infinite dimensional space.
Let be the set of all sequences of real numbers
For vectors and let
We define an inner product by
Show that is a vector space and that defines an inner product.
We have seen that the dot product of vectors in the vector space is an inner product. We provide another example which is closer to the our ultimate goal of calculating the Fourier coefficients.
Let now be the vector space of continuous functions over . Then we can define an inner product for functions by
Exercise Show that the inner product in the example above is indeed an inner product.
An essential step towards finding the , where and is given by the following example.
Show that the functions
are orthogonal with respect to the inner product
Further show that
for all integers Hence find an infinite set of orthonormal functions.
The above example shows, in particular, that the space is infinite dimensional, since we have found in infinite set of functions which are orthonormal.
Definition (Trigonometric polynomial of degree )
A trigonometric polynomial of degree with period is a function of the form
for some with or nonzero.
The following result now follows.
If is a trigonometric polynomial of finite degree with period then its Fourier coefficients are given by
In the infinite case one needs to take convergence considerations into account which we consider subsequently. For example the following question arises.
Question (T) State a condition on the function do we have
for all ?