74

CHAPTER 5 Mean-square convergence of Fourier series

This chapter has two main themes: convergence in the L^{2}-sense of the partial Fourier sums S_{N}f to f, and the completeness of the trigonometric functions in L^{2}(𝕋). They are intimately related. We begin by setting the scene.

In the previous chapter, we discussed pointwise and uniform convergence of the partial Fourier sums S_{N}f to the function f. We have learned that continuity of f is not sufficient to ensure pointwise convergence; we need something more. In particular we showed that if f has two continuous derivatives, then we get uniform convergence. If instead of taking the partial Fourier sums we take their averages (Cesàro sums), then we get uniform convergence for all continuous functions.

In this chapter we discuss mean-square convergence, also known as convergence in L^{2}(𝕋). Among other things, we will learn that the N^{th}-partial Fourier sum S_{N}f is the trigonometric polynomial of degree less than or equal to N which is closest to f in the L^{2}-norm. In fact it is the orthogonal projection onto the subspace of L^{2}(𝕋) consisting of N^{th}-degree trigonometric polynomials. In particular, this holds for continuous functions on 𝕋 and more generally for functions in L^{2}(𝕋). The energy or L^{2}-norm of the function can be recovered from just the knowledge of the Fourier coefficients via the celebrated Parseval’s Identity, which is an infinite-dimensional analogue of the Pythagorean Theorem.

What is behind all these results can be summarized in one very important sentence.

The trigonometric functions {e^{inθ}}_{n∈ℤ} form an orthonormal basis for L^{2}(𝕋).

We can see immediately that the trigonometric functions are orthonormal on 𝕋, because

\frac{1}{2π} ∫_{−π}^{π} e^{inθ}e^{−imθ} dθ = δ_{n,m},

see (1.6). What is really noteworthy here is that the family of trigonometric functions (with discrete integer values of n) is complete, in the sense that they span L^{2}(𝕋): every square-integrable function on 𝕋 can be written as a possibly infinite linear combination of the functions e^{inθ}.

5.1. Basic Fourier theorems in L^{2}(𝕋)

In Section 2.1 we introduced the space L^{2}(𝕋) of square-integrable functions on 𝕋 as the space of function f : 𝕋 → ℂ such that

∫_{−π}^{π} |f(θ)|^{2} dθ < ∞.

The integration here is in the Lebesgue sense, but for most purposes we can think of it as Riemann integration.

Recall that the L^{2}-norm or energy of a function f ∈ L^{2}(𝕋), denoted by ∥f∥_{L^{2}(𝕋)}, is the non-negative, finite quantity defined by

∥f∥_{L^{2}(𝕋)} := (\frac{1}{2π} ∫_{−π}^{π} |f(θ)|^{2} dθ)^{1/2} .

Before we describe in more detail the geometric structure of this space, let us take a step ahead and describe the main results concerning Fourier series and L^{2}(𝕋).

The Fourier coefficients for f ∈ L^{2}(𝕋) are well defined (since square-integrable functions on 𝕋 are integrable), as are its partial Fourier sums

S_{N}f(θ) = ∑_{|n|≤N} f̂(n)e^{2πinθ}.

Theorem 5.1. If f ∈ L^{2}(𝕋), then its partial Fourier sums converge to f in the L^{2}-sense, meaning that

∫_{−π}^{π} |S_{N}f(θ) − f(θ)|^{2} dθ → 0 as N → ∞.

We can restate the mean-square convergence theorem in terms of norms. Namely, if f ∈ L^{2}(𝕋) then

∥S_{N}f − f∥_{L^{2}(𝕋)} → 0 as N → ∞.

We can recover (the square of) the energy or norm of an L^{2}(𝕋) function f by adding the squares of the absolute values of its Fourier coefficients, as Parseval’s Identity^{1} shows.

Theorem 5.2 (Parseval’s Identity). If f ∈ L^{2}(𝕋), then

∥f∥_{L^{2}(𝕋)}^{2} = ∑_{n=−∞}^{∞} |f̂(n)|^{2}.

We prove Theorems 5.1 and 5.2, and more, in the course of this chapter. Parseval’s Identity immediately gives us an insight into the behavior of Fourier coefficients of (at least) square-integrable functions. Since the series on the right hand side of Parseval’s Identity converges, the coefficients must vanish as |n| → ∞, as described in the following lemma.

Corollary 5.3 (Riemann–Lebesgue Lemma in L^{2}(𝕋)). If f ∈ L^{2}(𝕋), then

f̂(n) → 0 as |n| → ∞.

Exercise 5.4. Deduce Corollary 5.3 from Parseval’s Identity. Note that we proved this result for the larger class L^{1}(𝕋) in Lemma 4.41.

The space of square-summable sequences, denoted by ℓ^{2}(ℤ) (“little ell-two of the integers”), is defined by

(5.1)  ℓ^{2}(ℤ) = {{a_{n}}_{n∈ℤ} : a_{n} ∈ ℂ and ∑_{n=−∞}^{∞} |a_{n}|^{2} < ∞} .

The space ℓ^{2}(ℤ) is a normed space, with norm given by

∥{a_{n}}∥_{ℓ^{2}(ℤ)} = ( ∑_{n=−∞}^{∞} |a_{n}|^{2})^{1/2} .

1Named after the French mathematician Marc-Antoine Parseval de Chênes (1755–1836).

5.2. GEOMETRY OF THE HILBERT SPACE L^{2}(𝕋) 77

In the language of norms, we can restate Parseval’s Identity as

(5.2)  ∥f∥_{L^{2}(𝕋)} = ∥{f̂(n)}∥_{ℓ^{2}(ℤ)}.

In other words, the Fourier mapping F : L^{2}(𝕋) → ℓ^{2}(ℤ), which associates to each function f ∈ L^{2}(𝕋) the sequence of its Fourier coefficients F(f) = {f̂(n)}_{n∈ℤ}, preserves norms.

In Appendix B, you will find the definition of normed spaces and complete inner-product vector spaces, also known as Hilbert spaces. It turns out that L^{2}(𝕋) and ℓ^{2}(ℤ) are Hilbert spaces^{2}. The Fourier mapping also preserves the inner products. In fact, the Fourier mapping is an isometry between these two Hilbert spaces. In Section 5.2 we describe the space L^{2}(𝕋) in more detail than we did in Chapter 2.

5.2. Geometry of the Hilbert space L^{2}(𝕋)

We denote by L^{2}(𝕋) the space of square-integrable functions on 𝕋. The space L^{2}(𝕋) is a vector space with the usual addition and scalar multiplication for functions. The L^{2}-norm is induced by an inner product defined for f, g ∈ L^{2}(𝕋) by

⟨f, g⟩ := \frac{1}{2π} ∫_{−π}^{π} f(θ)\overline{g(θ)} dθ,

in the sense that

∥f∥_{L^{2}(𝕋)} = ⟨f, f⟩^{1/2}.

Some, but not all, inner-product vector spaces satisfy another property, to do with convergence of Cauchy sequences of vectors. Namely, they are complete innerproduct vector spaces, also known as Hilbert spaces. In particular, L^{2}(𝕋) is a Hilbert space, while R(𝕋) with the L^{2}-inner product is not a Hilbert space.

Definition 5.5. A sequence of functions f_{n} ∈ L^{2}(𝕋) is a Cauchy sequence if for every ε > 0 there exists N > 0 such that ∥f_{n} − f_{m}∥_{L^{2}(𝕋)} < ε for all n, m > N.

We say that a sequence of functions {f_{n}} converges to f in the mean-square sense or the L^{2}-sense if

∥f_{n} − f∥_{L^{2}(𝕋)} → 0 as n → ∞.

It is well known that if a sequence is converging to a limit, then as n → ∞ the functions in the sequence are getting closer to each other, in the sense that they form a Cauchy sequence. The converse is true in the case of L^{2}(𝕋), a remarkable fact that we state without proof now (see [SS2] for example).

Theorem 5.6. The space L^{2}(𝕋) is a Hilbert space, that is, a complete innerproduct vector space. In other words, every Cauchy sequence in L^{2}(𝕋) (with respect to the norm induced by the inner product) converges to a function in L^{2}(𝕋).

Notice that here we are using the word “complete” with a different meaning than in Definition 5.21 where we will consider “complete systems of orthonormal functions”. It will be clear from the context whether we are talking about a “complete space” or a “complete system {f_{n}} of functions”.

2For this statement to be true in the case of L^{2}(𝕋), we really need the Lebesgue integral.

If we insist on using only the Riemann integral, then we still get an inner-product vector space.

However, that space is not complete: there are Cauchy sequences of Riemann square-integrable functions that converge to functions that are not Riemann square integrable, but are necessarily Lebesgue square integrable. See Section 5.2.

More can be said.

Theorem 5.7. The space L^{2}(𝕋) of Lebesgue square-integrable functions on 𝕋 consists of the collection R(𝕋) of all Riemann-integrable functions on 𝕋, together with all the functions that arise as limits of Cauchy sequences in R(𝕋) with respect to the the L^{2}-norm. In other words, L^{2}(𝕋) is the completion of R(𝕋) with respect to the L^{2}-metric.

Example 5.8. The space of square-summable sequences ℓ^{2}(ℤ) is also an innerproduct vector space with inner-product

⟨\{a_{n}\}, \{b_{n}\}⟩_{ℓ^{2}(ℤ)} = ∑_{n=−∞}^{∞} a_{n}\overline{b_{n}}.

Moreover, ℓ^{2}(ℤ) is complete. Therefore it is a Hilbert space. ♢

Exercise 5.9. Prove that ℓ^{2}(ℤ) is complete.

Recall that Lebesgue square-integrable functions on 𝕋 are always Lebesgue integrable on 𝕋. In other words, L^{2}(𝕋) ⊂ L^{1}(𝕋)^{3}, see Section 2.1. On the other hand, there are functions in L^{1}(𝕋) that are not in L^{2}(𝕋). Such functions must be unbounded. In fact if f is a bounded function in L^{1}(𝕋), then f is in L^{2}(𝕋); see Exercise 2.20. In particular, Riemann-integrable functions are necessarily bounded, and are Lebesgue integrable, and so they are in L^{2}(𝕋). That is, R(𝕋) ⊂ L^{2}(𝕋). Thus in each theorem in this section we could replace f ∈ L^{2}(𝕋) by f ∈ R(𝕋), obtaining a less general theorem that does not involve the notion of Lebesgue integration.

5.2.1. Three key results involving orthogonality. Orthogonality is one of the most important ideas in the context of inner-product vector spaces. It generalizes, to infinite-dimensional spaces, the familiar idea of two perpendicular vectors in the plane.

We recall here the definitions for the specific cases of L^{2}(𝕋) and ℓ^{2}(ℤ) that we are studying. In Appendix B we recall the definitions in the setting of a general inner-product vector space. The reader will find that the statements and proofs we give here do generalize to that context without much extra effort. It may take a little while to get used to thinking of functions as vectors, however.

Definition 5.10. Two functions f, g ∈ L^{2}(𝕋) are orthogonal if ⟨f, g⟩ = 0. Sometimes we use the notation f ⊥ g to indicate that f and g are orthogonal. A collection of functions A ⊂ L^{2}(𝕋) is orthogonal if f ⊥ g for all f, g ∈ A with f ≠ g. Two subsets of A, B of L^{2}(𝕋) are orthogonal, written A ⊥ B, if f ⊥ g for all f ∈ A and all g ∈ B.

The trigonometric functions {e_{n}(θ) := e^{inθ}}_{n∈ℤ} are orthonormal in L^{2}(𝕋). In other words, not only are they an orthogonal set, but each element in the set has L^{2}- norm equal to one (the vectors have been “normalized”). Checking orthonormality of the trigonometric system is equivalent to verifying that

(5.3)  ⟨e_{k}, e_{m}⟩ = δ_{k,m}.

The Pythagorean Theorem, the Cauchy–Schwarz Inequality, and the Triangle Inequality are well-known in every inner-product vector space over ℂ. We state them explicitly for the case of L^{2}(𝕋).

3The analogous statement for ℝ is false. The function f(x) = (1 + |x|)^{−1} is not integrable on ℝ, but it is square integrable on ℝ. So L^{2}(ℝ) ⊄ L^{1}(ℝ).

Theorem 5.11 (Pythagorean Theorem). If f, g ∈ L^{2}(𝕋) are orthogonal, then

∥f + g∥_{L^{2}(𝕋)}^{2} = ∥f∥_{L^{2}(𝕋)}^{2} + ∥g∥_{L^{2}(𝕋)}^{2}.

Theorem 5.12 (Cauchy–Schwarz Inequality). For all f, g ∈ L^{2}(𝕋),

|⟨f, g⟩| ≤ ∥f∥_{L^{2}(𝕋)}∥g∥_{L^{2}(𝕋)}.

Theorem 5.13 (Triangle Inequality). For all f, g ∈ L^{2}(𝕋),

∥f + g∥_{L^{2}(𝕋)} ≤ ∥f∥_{L^{2}(𝕋)} + ∥g∥_{L^{2}(𝕋)}.

Exercise 5.14. Prove Theorems 5.11, 5.12, and 5.13. In particular, show that ∥f∥_{L^{2}(𝕋)} = \sqrt{⟨f, f⟩} is a norm on L^{2}(𝕋).

Exercise 5.15. Deduce from the Cauchy–Schwarz Inequality that if f and g are in L^{2}(𝕋), then |⟨f, g⟩| < ∞.

An important consequence of the Cauchy–Schwarz inequality is that we can interchange limits in L^{2} and the inner product (or in language that we will meet later in the book, that the inner product is continuous).

Proposition 5.16. Fix g ∈ L^{2}(𝕋). If a sequence of L^{2}-functions {f_{n}}_{n∈ℕ} converges to f in L^{2}-sense, then

lim_{n→∞}⟨g, f_{n}⟩ = ⟨g, f⟩.

In other words, we can interchange the limit and the inner product with g.

Proof. Assume g, f_{n} ∈ L^{2}(𝕋) and ∥f_{n} − f∥_{L^{2}(𝕋)} → 0 as n → ∞. Since L^{2}(𝕋) is complete, it follows that f ∈ L^{2}(𝕋). By the Cauchy–Schwarz Inequality,

|⟨g, f_{n} − f⟩| ≤ ∥g∥_{L^{2}(𝕋)}∥f_{n} − f∥_{L^{2}(𝕋)}.

We conclude that

lim_{n→∞}⟨g, f_{n} − f⟩ = 0,

and so

lim_{n→∞}⟨g, f_{n}⟩ = ⟨g, f⟩

by the linear properties of the inner product. This proves the proposition. □

As a consequence, we can also interchange the inner product with a convergent infinite sum in L^{2}(𝕋).

Corollary 5.17. Fix g ∈ L^{2}(𝕋). Suppose f = ∑_{k=1}^{∞} h_{k} in the L^{2}-sense. Then

⟨g, ∑_{k=1}^{∞} h_{k}⟩ = ∑_{k=1}^{∞}⟨g, h_{k}⟩.

Exercise 5.18. Prove Corollary 5.17 by setting f_{n} = ∑_{k=1}^{n} h_{k} in Proposition 5.16.

Exercise 5.19. Verify that if f = ∑_{n∈ℤ} a_{n}e_{n} and g = ∑_{n∈ℤ} b_{n}e_{n}, where the equalities are in L^{2}-sense, and {e_{n}} are the trigonometric functions, then

⟨f, g⟩ = ∑_{n∈ℕ} a_{n}\overline{b_{n}}.

Exercise 5.20. Suppose {f_{λ}}_{λ∈Λ}, with Λ an arbitrary index set, is an orthogonal family in L^{2}(𝕋). Assume all the functions f_{λ} are non-zero in L^{2}(𝕋). Show that they are linearly independent; in other words, for any finite subset of indices {λ_{1}, …, λ_{n}} ⊂ Λ,

a_{1}f_{λ_{1}} + ⋯ a_{n}f_{λ_{n}} = 0, if and only if a_{1} = ⋯ = a_{n} = 0.

5.2.2. Orthonormal bases. Orthogonality implies linear independence (see Exercise 5.20), and if our geometric intuition is right, orthogonality is in some sense the “most” linearly independent a set could be. In a finite-dimensional vector space of dimension N, if we locate N linearly independent vectors we have located a basis of the space. Similarly if we have an N-dimensional inner-product vector space and we find N orthonormal vectors, we have found an orthonormal basis.

The trigonometric functions {e_{n}(θ) := e^{inθ}}_{n∈ℤ} are an orthonormal set of L^{2}(𝕋). Having an infinite orthonormal family (hence a linearly independent family) tells us that the space is infinite-dimensional. So far, there is no guarantee that there are no other functions in the space L^{2}(𝕋) orthogonal to the trigonometric functions, or in other words that the system of trigonometric functions is complete. We will see that the completeness of the set of trigonometric functions in L^{2}(𝕋) is equivalent to the mean-square convergence of partial Fourier sums, and also to Parseval’s Identity.

Definition 5.21. Let {f_{n}}_{n∈ℕ} be an orthonormal family in L^{2}(𝕋). We say that the family is complete, or that {f_{n}}_{n∈ℕ} is a complete orthonormal system in L^{2}(𝕋), or that the vectors in the family form an orthonormal basis, if given any function f ∈ L^{2}(𝕋), f can be expanded into a series of the basis elements that is convergent in the L^{2}-norm. That is, there exists a sequence of complex numbers {a_{n}}_{n∈ℕ} such that

lim_{N→∞} ∥f − ∑_{n=1}^{N} a_{n} f_{n}∥_{L^{2}(𝕋)} = 0.

This is the same as saying that f = ∑_{n=1}^{∞} a_{n}f_{n} where equality holds in the L^{2}-sense.

The coefficients are uniquely determined by pairing the function with the basis functions.

Lemma 5.22. (Uniqueness of the coefficients). If {f_{n}}_{n∈ℕ} is an orthonormal basis in L^{2}(𝕋), and f ∈ L^{2}(𝕋), then there exists complex numbers {a_{n}}_{n∈ℕ} such that

f = ∑_{n=1}^{∞} a_{n}f_{n},

and the coefficients have no other choice than to be a_{n} = ⟨f, f_{n}⟩ for all n ∈ ℕ.

Proof. Take the expansion of f in the basis elements, and pair it with f_{k}, interchange inner-product and summation, and use the orthonormality of the system to get,

⟨f, f_{k}⟩ = ⟨∑_{n=1}^{∞} a_{n}f_{n}, f_{k}⟩

= ∑_{n=1}^{∞} a_{n}⟨f_{n}, f_{k}⟩
= a_{k}.

The interchange of the inner-product and the summation is justified because the inner-product is a “continuous map” see Corollary 5.17. □

In the case of the trigonometric system, which is an orthonormal system, if the system is complete, the coefficients must be the Fourier coefficients. It will take us some work to verify exactly that, which we now state as a theorem, still to be proven.

Theorem 5.23. The trigonometric system is a complete orthonormal system in L^{2}(𝕋). Hence if f ∈ L^{2}(𝕋) then

f(θ) = ∑_{n∈ℤ} f̂(n)e^{−inθ},

where the equality is in the L^{2}-sense.

5.3. Completeness of the trigonometric system

The trigonometric functions {e_{n}(θ) := e^{inθ}}_{n∈ℤ} are an orthonormal set in L^{2}(𝕋). Our goal is to show that the system is complete, that is Theorem 5.23. We will show that it is equivalent to showing that we have mean-square convergence of the partial Fourier sums S_{N}f to f, that is Theorem 5.1,

∥f − S_{N}f∥_{2} → 0 as N → ∞, for f ∈ L^{2}(𝕋).

Aside 5.24. To simplify notation, we use ∥f∥_{2} instead of ∥f∥_{L^{2}(𝕋)}.

More precisely, the Fourier coefficients of f ∈ L^{2}(𝕋) are given by the inner product of f against the corresponding trigonometric function,

f̂(n) = \frac{1}{2π} ∫_{−π}^{π} f(θ)\overline{e^{inθ}} dθ = ⟨f, e_{n}⟩.

Hence the partial Fourier sums are given by

S_{N}f = ∑_{|n|≤N} f̂(n) e_{n} = ∑_{|n|≤N}⟨f, e_{n}⟩ e_{n}.

Now it is clear, by Definition 5.21, that to verify completeness of the trigonometric system suffices to check the mean-square convergence of the partial Fourier sums,

lim_{N→∞} ∥f − ∑_{|n|≤N}⟨f, e_{n}⟩ e_{n}∥_{2} = lim_{N→∞} ∥f − S_{N}f∥_{2} = 0.

To prove Theorem 5.23 and hence Theorem 5.1, we will first show that S_{N}f is the trigonometric polynomial of degree N that best approximates f in the L^{2}- norm. In other words, S_{N}f is the orthogonal projection of f onto the subspace of trigonometric polynomials of degree less than or equal to N. Next we will show that the partial Fourier sums converge to f in L^{2}(𝕋) for continuous functions f. Finally, an approximation argument will allow us to conclude the same for square-integrable functions.

5.3.1. S_{N}f is the closest N^{th}- trigonometric polynomial to f. The first observation is that (f − S_{N}f) is orthogonal to the subspace P_{N}(𝕋) generated by {e_{n}}_{|n|≤N}. That subspace is exactly the subspace of trigonometric polynomials of degree less than or equal to N, in other words functions of the form ∑_{|n|≤N} c_{n}e_{n}, for complex numbers c_{n}.

Lemma 5.25. Given f ∈ L^{2}(𝕋), (f − S_{N}f) is orthogonal to all trigonometric polynomials of degree less than or equal to N.

Proof. It suffices to show that (f − S_{N}f) is orthogonal to

∑_{|n|≤N} c_{n}e_{n}

for all choices of complex numbers {c_{n}}_{|n|≤N}.

First note that e_{j} ⊥ (f − S_{N}f) for each j such that |j| ≤ N, because

⟨f − S_{N}f, e_{j}⟩ = ⟨f − ∑_{|n|≤N} f̂(n)e_{n}, e_{j}⟩
= ⟨f, e_{j}⟩ − ∑_{|n|≤N} f̂(n)⟨e_{n}, e_{j}⟩  (linearity of the inner product)
= ⟨f, e_{j}⟩ − f̂(j)  (orthonormality of the trigonometric system)
= 0.

Now plug in any linear combination ∑_{|j|≤N} c_{j}e_{j} instead of e_{j}, and use the fact that the inner product is conjugate linear in the second variable, and what we just showed, to get

⟨f − S_{N}f, ∑_{|j|≤N} c_{j}e_{j}⟩ = ∑_{|j|≤N} \overline{c_{j}} ⟨f − S_{N}f, e_{j}⟩ = 0.

Thus (f − S_{N}f) ⊥ ∑_{|j|≤N} c_{j}e_{j} for every such linear combination, as claimed. □

Since the partial sum S_{N}f = ∑_{|n|≤N} f̂(n)e_{n} is a trigonometric polynomial of degree N, then by Lemma 5.25, it is orthogonal to (f − S_{N}f), and by the Pythagorean Theorem 5.11,

∥f∥_{2}^{2} = ∥f − S_{N}f∥_{2}^{2} + ∥S_{N}f∥_{2}^{2}.

Lemma 5.26. If f ∈ L^{2}(𝕋), then

∥f∥_{2}^{2} = ∥f − S_{N}f∥_{2}^{2} + ∑_{|n|≤N} |f̂(n)|^{2}.

Proof. All we need to verify is that ∥S_{N}f∥_{2}^{2} = ∑_{|n|≤N} |f̂(n)|^{2}. In fact, by linearity in the first variable and conjugate linearity in the second variable of the inner product, and by the orthonormality of the trigonometric functions,

∥S_{N}f∥_{2}^{2} = ∥ ∑_{|n|≤N} f̂(n)e_{n}∥_{2}^{2}
= ⟨ ∑_{|n|≤N} f̂(n)e_{n}, ∑_{|j|≤N} f̂(j)e_{j}⟩

= ∑_{|n|≤N} ∑_{|j|≤N} f̂(n)\overline{f̂(j)}⟨e_{n}, e_{j}⟩

= ∑_{|n|≤N} f̂(n)\overline{f̂(n)}
= ∑_{|n|≤N} |f̂(n)|^{2},

as required. Notice that this is nothing more that the Pythagorean theorem for 2N + 1 orthogonal summands. □

As a corollary we obtain Bessel’s well-known inequality^{4} for trigonometric functions.

Lemma 5.27 (Bessel’s Inequality). For all f ∈ L^{2}(𝕋),

(5.4)  ∑_{n∈ℤ} |⟨f, e_{n}⟩|^{2} ≤ ∥f∥_{2}^{2}.

Equality (Plancherel’s Identity!) holds exactly when the orthonormal system is complete, which we have yet to prove.

It turns out that the partial Fourier sum S_{N}f is the orthogonal projection of f onto the subspace P_{N}(𝕋) of trigonometric polynomials of degree less than or equal to N.

Among all trigonometric polynomials of degree less than or equal to N, the N^{th} partial Fourier sum S_{N}f is the one that best approximates f in the L^{2}-norm.

Here are the precise statement and proof of the above statement.

Lemma 5.28 (Best Approximation Lemma ). Take f ∈ L^{2}(𝕋). Then for each N ≥ 0 and for all trigonometric polynomials P ∈ P_{N}(𝕋), we have that

∥f − S_{N}f∥_{2} ≤ ∥f − P ∥_{2}.

Equality holds if and only if P = S_{N}f.

Proof. Let P ∈ P_{N}(𝕋). Then there exist complex numbers {c_{n}}_{n=1}^{N} such that

P = ∑_{|n|≤N} c_{n}e_{n}. Then

f − P = f − ∑_{|n|≤N} c_{n}e_{n} − S_{N}f + S_{N}f
= \underbrace{f − S_{N}f} + \underbrace{∑_{|n|≤N} (f̂(n) − c_{n})e_{n}} .

4This inequality is named after the German mathematician Friedrich Wilhelm Bessel (1784–

Note that the underbraced terms are orthogonal to each other by Lemma 5.25. Hence, by the Pythagorean theorem,

∥f − P ∥_{2}^{2} = ∥f − S_{N}f∥_{2}^{2} + \underbrace{∥ ∑_{|n|≤N} (f̂(n) − c_{n})e_{n}∥_{2}^{2}}_{≥0} .

Therefore

∥f − P ∥_{2}^{2} ≥ ∥f − S_{N}f∥_{2}.

Clearly equality holds if and only if the underbraced term is equal to zero, but by the argument in the proof of Lemma 5.26,

∥ ∑_{|n|≤N} (f̂(n) − c_{n})e_{n}∥_{2}^{2} = ∑_{|n|≤N} |f̂(n) − c_{n}|^{2}.

The right-hand side is equal to zero if and only if f̂(n) = c_{n} for all |n| ≤ N. That is, P = S_{N}f as required for the equality to hold.

The lemma is proved. □

Aside 5.29. Every lemma in this section has a counterpart for any orthonormal system X = {ψ_{n}}_{n∈ℕ} in L^{2}(𝕋), where the subspace P_{N}(𝕋) of trigonometric polynomials of degree less than or equal to N is replaced by the subspace generated by the first N functions in the system,

W_{N} = {f ∈ L^{2}(𝕋) : f = ∑_{n=1}^{N} a_{n}ψ_{n}, a_{n} ∈ ℂ}.

In fact all the theorems are valid for any inner-product vector space and for any orthonormal system.

Exercise 5.30. State and prove the analogues of Lemma 5.25, Lemma 5.26, Bessel’s Inequality 5.27 and Best Approximation Lemma 5.28, for X any orthonormal system in L^{2}(𝕋), not necessarily the trigonometric system.

5.3.2. Mean-square convergence for continuous functions.

Proof of Theorem 5.1 for Continuous Functions. We are ready to prove that

∥f − S_{N}f∥_{2} → 0 as N → ∞,

for continuous functions f.

Fix ε > 0.

By Fejér’s Theorem, the Cesàro sums σ_{N}f converge uniformly to f on 𝕋. So there is a trigonometric polynomial of degree M say, namely P (θ) = σ_{M}f(θ), such that

|f(θ) − P (θ)| < ε for all θ ∈ 𝕋.

This implies that

∥f − P ∥_{2}^{2} = \frac{1}{2π} ∫_{−π}^{π} |f(θ) − P (θ)|^{2} dθ < ε^{2},

and so ∥f − P ∥_{2} ≤ ε.

Notice that if M ≤ N, then P ∈ P_{N}(𝕋). We can now use the Best Approximation Lemma (Lemma 5.28) to conclude that

∥f − S_{N}f∥_{2} ≤ ∥f − P ∥_{2} < ε for all N ≥ M.

Therefore, for continuous functions f,

lim_{N→∞} ∥f − S_{N}f∥_{2} = 0

as claimed.

□

5.3.3. Mean-square convergence for L^{2}-functions. We are now ready to prove the mean-square convergence for any function f ∈ L^{2}(𝕋), and hence to verify the completeness of the trigonometric system in L^{2}(𝕋).

Proof of Theorem 5.1. We now know that for f ∈ C(𝕋)

lim_{N→∞} ∥f − S_{N}f∥_{2} = 0.

The continuous functions on 𝕋 are dense in L^{2}(𝕋), see Theorem 2.66. This means that given a function f ∈ L^{2}(𝕋), and given ε > 0, there exists a continuous function g on 𝕋 such that

∥f − g∥_{2} ≤ ε.

Since g is continuous, we can find a trigonometric polynomial P such that

∥g − P ∥_{2} ≤ ε.

Therefore, by the triangle inequality,

∥f − P ∥_{2} ≤ ∥f − g∥_{2} + ∥g − P ∥_{2} ≤ 2ε.

Let M be the degree of P. By Lemma 5.28, for all N ≥ M we have

∥f − S_{N}f∥_{2} ≤ ∥f − P ∥_{2} ≤ 2ε.

Lo and behold, for all f ∈ L^{2}(𝕋),

lim_{N→∞} ∥f − S_{N}f∥_{2} = 0,

as claimed. □

5.3.4. Parseval’s Identity. Given the completeness of the trigonometric system, we can now prove Parseval’s Identity, namely

∥f∥_{2}^{2} = \frac{1}{2π} ∫_{−π}^{π} |f(θ)|^{2} dθ = ∑_{n=−∞}^{∞} |f̂(n)|^{2}.

Proof of Parseval’s Identity 5.2. Recall Lemma 5.26,

(5.5)  ∥f∥_{2}^{2} = ∥f − S_{N}f∥_{2}^{2} + ∑_{|n|≤N} |f̂(n)|^{2}.

Therefore the series on the right-hand side converges (all terms are nonnegative and the sum is bounded above by ∥f∥_{2}^{2}),

∑_{n=−∞}^{∞} |f̂(n)|^{2} ≤ ∥f∥_{2}^{2}.

We have just shown that if f ∈ L^{2}(𝕋) then ∥f − S_{N}f∥_{2} → 0 as N → ∞. In fact,

∥f∥_{2}^{2} = lim_{N→∞} ∑_{|n|≤N} |f̂(n)|^{2} =: ∑_{n=−∞}^{∞} |f̂(n)|^{2}.

□

5.4. Equivalent conditions for completeness in L^{2}(𝕋)

How can we tell whether a given orthonormal family X in L^{2}(𝕋) is actually an orthonormal basis? One criterion for completeness is that the only function orthogonal to all the functions in L^{2}(𝕋) is the zero function. Another is that Plancherel’s Identity, which is an infinite-dimensional Pythagorean Theorem, must hold. These ideas are summarized in Theorem 5.31, which holds for all innerproduct vector spaces V over ℂ, see Appendix B. We will prove Theorem 5.31 for the particular case when X is the trigonometric system. Showing that the same proof extends to any orthonormal system in L^{2}(𝕋) is left as an exercise.

Theorem 5.31. Let X = {f_{n}}_{n∈ℕ} an orthonormal family in L^{2}(𝕋). Then the following are equivalent:

(i) X is a complete system, hence an orthonormal basis of L^{2}(𝕋).

(ii) If f ⊥ X then f = 0 in L^{2}.

(iii) (Plancherel’s Identity) For all f ∈ L^{2}(𝕋),

∥f∥_{L^{2}(𝕋)}^{2} = ∑_{n=1}^{∞} |⟨f, f_{n}⟩|^{2}.

Proof for the trigonometric system. When X is the trigonometric system, we have shown that completeness implies Parseval’s Identity, which in this context coincides with Plancherel’s Identity. That is, we have shown that (i) ⇒ (iii).

If f ∈ L^{2}(𝕋) were orthogonal to all trigonometric functions, then necessarily f̂(n) = ⟨f, e_{n}⟩ = 0 for all n ∈ ℤ. In that case the partial Fourier sums S_{N}f would be identically equal to zero for all N ≥ 0. The completeness of the trigonometric system now implies

∥f∥_{2} = lim_{N→∞} ∥f − S_{N}f∥_{2} = 0.

Therefore f = 0. That is the completeness of the trigonometric system implies that the only L^{2}-function orthogonal to all trigonometric functions is the zero function, that is (i) ⇒ (ii) for the particular case of X being the trigonometric system.

If we had established Parseval’s Identity by independent means, and knowing that {e_{n}(θ) = e^{inθ}}_{n∈ℤ} is an orthonormal system, then we can deduce the completeness of the trigonometric system. In fact, equation (5.5) holds for all N > 0, and so we can take the limit as N → ∞ on both sides to get

∥f∥_{2}^{2} = lim_{N→∞} ∥f − S_{N}f∥_{2}^{2} + ∑_{n∈ℤ} |f̂(n)|^{2}.

Now use Parseval’s Identity to cancel the left-hand side and the rightmost summand. We conclude that for all f ∈ L^{2}(𝕋)

lim_{N→∞} ∥f − S_{N}f∥_{2} = 0.

5.4. EQUIVALENT CONDITIONS FOR COMPLETENESS IN L^{2}(𝕋) 87

To recapitulate: for X the trigonometric system we have shown the equivalence of conditions (i) and (iii) in Theorem 5.31. We also know that (i) implies (ii) in this case. To close the loop it suffices to argue that (ii) implies (iii).

We will proceed by the contrapositive. (Notice that we are not assuming the trigonometric system to be complete, only orthonormal). Assume Parseval’s Inequality does not hold for some function g ∈ L^{2}(𝕋). Then for such a function, the following strict inequality holds (Bessel’s Inequality):

∑_{n∈ℤ} |⟨g, e_{n}⟩|^{2} < ∥g∥_{2}^{2} < ∞.

Define a function h := ∑_{n∈ℤ}⟨g, e_{n}⟩e_{n}. Then h ∈ L^{2}(𝕋), ⟨h, e_{k}⟩ = ⟨g, e_{k}⟩ by the continuity of the inner-product (see Corollary 5.17), and

∥h∥_{2}^{2} = ∑_{n∈ℤ} |⟨g, e_{n}⟩|^{2},

by Exercise 5.19.

Now, let f = g −h ∈ L^{2}(𝕋). The first observation is that f ≠ 0 in the L^{2}-sense, otherwise Parseval will hold for g. The second observation is that f is orthogonal to e_{n} for all n ∈ ℤ, this is just a calculation,

⟨f, e_{n}⟩ = ⟨g − h, e_{n}⟩ = ⟨g, e_{n}⟩ − ⟨h, e_{n}⟩ = 0.

If Parseval does not hold there is a non-zero function f orthogonal to all trigonometric functions. We have shown then that in the case of the trigonometric system, (ii) ⇒ (iii). □

Exercise 5.32. Prove Theorem 5.31 for any orthonormal system X in L^{2}(𝕋).

5.4.1. Orthogonal projection onto a closed subspace. Lemma 5.28 states that S_{N}f is the best approximation to f in the subspace of trigonometric polynomials of degree N in the L^{2}-norm.

More is actually true. We first recall that a subspace W of L^{2}(𝕋) is said to be closed if every convergent sequence in W converges to a point in W.

Theorem 5.33 (Orthogonal Projection). Given any closed subspace W of the complete inner-product vector space L^{2}(𝕋) and given f ∈ L^{2}(𝕋), there exists a unique function P_{W}f ∈ W that minimizes the distance in L^{2}(𝕋) to W. That is, for all g ∈ W

∥f − g∥_{2} ≥ ∥f − P_{W}f∥_{2} for all g ∈ W.

Furthermore, the vector f − P_{W}f is orthogonal to P_{W}f.

If B = {g_{n}}_{n∈ℕ} is any orthonormal basis of W, then

P_{W}f = ∑_{n∈ℕ}⟨f, g_{n}⟩g_{n}.

We proved Theorem 5.33 for the subspaces P_{N}(𝕋) of trigonometric polynomials of degree less than or equal to N. They are finite dimensional (of dimension 2N +1) and therefore automatically closed.

The theorem holds for every complete inner-product vector space V and every closed subspace. Theorem 5.33 says that we can draw pictures as we do in Euclidean space of a vector and its orthogonal projection to a closed subspace (represented by a plane), and the difference of these two vectors minimizes the distance to the subspace; see Figure 5.1.

[figure: W; x; P_{W}x; ∥x − P_{W}x∥]

Figure 5.1. Orthogonal projection of a vector x onto a closed subspace W of a vector space V. The vector space V is represented here by three-dimensional space. The subspace W is represented by a plane. The vector x is pictured, and its orthogonal projection onto W is represented by the vector P_{W}x. The distance from x to W is the length ∥x − P_{W}x∥ of the difference vector x − P_{W}x.

We will use this orthogonal projection theorem for the Hilbert space L^{2}(ℝ) instead of L^{2}(𝕋) in Section 8.3 when we discuss the Haar basis on ℝ, and in Chapter 9 when we talk about wavelets and multiresolution analysis.

Exercise 5.34. Prove Theorem 5.33 for a closed subspace W of L^{2}(𝕋). Knowing that the subspace has an orthonormal basis B = {g_{n}}_{n∈ℕ}, hence

W = {f ∈ L^{2}(𝕋) : f = ∑_{n∈ℕ} a_{n}g_{n}, ∑_{n∈ℕ} |a_{n}|^{2} < ∞}.