CHAPTER 9 From Fourier to Haar

In this chapter we give a brief survey of the windowed Fourier transform, also known as the Gabor transform, and introduce the newest member of the family, wavelet analysis. We also discuss the Haar basis with great care, and compare Fourier and Haar analysis, as well as Fourier and Haar basis in L^{p} spaces.

9.1. The windowed Fourier transform, and Gabor bases

The continuous Fourier transform provides a tool for analyzing a function defined on the whole real line ℝ, but the exponentials cannot be viewed as a “countable basis” any more, since there is one for each ξ ∈ ℝ. Also, the trigonometric functions {e^{2πiξx}} are not even in L^{2}(ℝ), although of course their restrictions to 𝕋 were in L^{2}(𝕋). The windowed Fourier transform addresses this problem.

How can we obtain an orthonormal basis for L^{2}(ℝ)? A simple solution would be to split the line into unit segments [k, k +1) indexed by k ∈ ℤ, and on each segment use the periodic Fourier basis for that segment. Let χ_{A} denote the characteristic function of a given set A ⊂ ℝ:

χ_{A}(x) := \begin{cases}1, & if x ∈ A; \\ 0, & otherwise. \end{cases}

The functions

g_{n,k}(x) = e^{2πinx}χ_{[k,k+1)}(x), for n, k ∈ ℤ,

form an orthonormal basis for L^{2}(ℝ). They give us the so-called windowed Fourier transform, defined by

Gf(n, k) := ⟨f, g_{n,k}⟩ = ∫_{k}^{k+1} f(x)e^{−2πinx} dx,

and the corresponding reconstruction formula, where equality holds in the L^{2}-sense,

f(x) = ∑_{j,k} Gf(n, k)g_{n,k}(x).

Notice that now our signal f is a function of the continuous variable x ∈ ℝ, while its windowed Fourier transform Gf is a function of two discrete (integer) variables, n and k.

Exercise 9.1. Verify that the family {g_{n,k}}_{n,k∈ℤ} is an orthonormal basis in L^{2}(ℝ).

We can think of each function χ_{[k,k+1)} giving us a window through which to view the behavior of f on the interval [k, k + 1). Both the function χ_{[k,k+1)} and the associated interval [k, k + 1) are commonly called windows.

Figure 9.1. The figure shows g_{n,k} for some values of n and k, and for g = χ_{[0,1)}.

We could have used windows of varying sizes. More precisely, given an arbitrary partition {a_{k}}_{k∈ℤ} of ℝ into bounded intervals [a_{k}, a_{k+1}), k ∈ ℤ, let L_{k} = a_{k+1} −a_{k}, and on each window use the corresponding L_{k}-Fourier basis. Then the functions

\frac{1}{\sqrt{L_{k}}} e^{−2πinx/L_{k}}χ_{[a_{k},a_{k+1})}(x), for n, k ∈ ℤ,

form an orthonormal basis of L^{2}(ℝ). This generalization provides some adaptability of the basis to the function to be analyzed: for instance, if the behavior of f changes a lot in some region of ℝ we may want to use many small windows there, however wherever the function does not fluctuate much, we may use wider windows. We can get a fairly accurate reconstruction of f while retaining only a few coefficients. On the wider windows, a few low frequencies should contain most of the information. On the smaller windows, retaining a few big coefficients might not be that accurate but hopefully it is only on a few small intervals. On the other hand, by adapting the windows to the function, we might lose the translation structure provided by having all the windows the same size.

Because of their discontinuities at k and k + 1, the windows χ_{[k,k+1)} are called sharp windows. It turns out that in numerical calculations, these sharp windows produce at the edges the same artifacts that are seen when analyzing periodic functions at discontinuity points (Gibb’s phenomenon, or “kinks” at the divisions between windows).

Exercise 9.2. Consider the hat function

f(x) = \{ \begin{matrix}1 − |x| & −1 ≤ x < 1 \\ 0 & otherwise \end{matrix} .

Compute its windowed Fourier transform with windows the intervals [k, k + 1), and plot using Matlab successive approximations. You should see “kinks” at x = −1, 0, 1. Do the same with windows on the intervals [2k − 1, 2k + 1), do you see any kinks? How about windows [k/2, (k + 1)/2)?

To avoid Gibb’s phenomenon, smoother windows are desirable. The sharp windows, χ_{[0,1)} and its modulated integer translates e^{2πinx}χ_{[k,k+1)}(x), can be replaced by a smooth window g and its modulated integer translates

(9.1)  g_{n,k}(x) = g(x − k)e^{2πinx},  for n, k ∈ ℤ.

When a function g ∈ L^{2}(ℝ) is such that {g_{n,k}}_{n,k∈ℤ} is an orthonormal basis for L^{2}(ℝ), we call the function a Gabor function, and the basis a Gabor basis. In 1946, Gabor considered systems of this type and proposed using them in communication theory [Gab].

Figure 9.2. The figure shows g_{n,k} for some values of n and k, and for g a smooth window.

The polarization formula (7.26) for square integrable functions (⟨f, g⟩ = ⟨f̂, ĝ⟩) shows that a family of functions is an orthonormal basis if and only if the family of their Fourier transforms is itself an orthonormal basis. In other words,

{ψ_{n}}_{n∈ℕ} is an o.n. basis in L^{2}(ℝ) ⟺ {ψ̂_{n}}_{n∈ℕ} is an o.n. basis in L^{2}(ℝ).

Orthonormality holds on one side if and only if it holds on the other, because for all n, m ∈ ℕ, ⟨ψ_{n}, ψ_{m}⟩ = ⟨ψ̂_{n}, ψ̂_{m}⟩. The same is true for completeness, because f ⊥ ψ_{n} if and only if f̂ ⊥ ψ̂_{n}.

In particular, given a Gabor basis {g_{n,k}}_{n,k∈ℤ}, generated by g ∈ L^{2}(ℝ) according to the scheme in equation (9.1), the family of their Fourier transforms {ĝ_{n,k}}_{n,k∈ℤ} forms an orthonormal basis, by the above observation. What is remarkable is that the basis given by the Fourier transforms of a Gabor basis is again a Gabor basis. With the time–frequency dictionary in mind, we see on closer examination that the precise form of the modulated integer translates in equation (9.1) is exactly what is needed to achieve this property, since the Fourier transform converts translation to modulation, and vice versa. More precisely, notice that the Fourier transforms of the Gabor basis elements can be calculated using the time–frequency dictionary:

(9.2)  ĝ_{n,k}(ξ) = ĝ(ξ − n)e^{−2πikξ} = (ĝ)_{−k,n}(ξ).

Exercise 9.3. Verify (9.2).

To sum up:

A function g ∈ L^{2}(ℝ) generates a Gabor basis, in other words {g_{n,k}} forms an orthonormal basis in L^{2}(ℝ), if and only if ĝ ∈ L^{2}(ℝ) generates a Gabor basis, in other words {(ĝ)_{n,k}} forms an orthonormal basis in L^{2}(ℝ).

Example 9.4. Since g = χ_{[0,1)} generates a Gabor basis, so does its Fourier transform

ĝ(ξ) = (χ_{[0,1)})^{∧}(ξ) = e^{−iπξ} \frac{sin (πξ)}{πξ} = e^{−iπξ} sinc(ξ).

This is an example of a differentiable window ĝ, in contrast with our first window which was not even continuous. However, unlike g, ĝ is not compactly supported.

♢

Which other functions g ∈ L^{2}(ℝ) generate a Gabor basis? Can we find a Gabor function that is simultaneously smooth and compactly supported? The limitations of the Gabor analysis are explained by the following result.

Theorem 9.5 (Balian–Low Theorem). For g ∈ L^{2}(ℝ), if {g_{n,k}}_{n,k∈ℤ} is an orthonormal basis, then either

∫_{ℝ} x^{2}|g(x)|^{2} dx = ∞,  or  ∫_{ℝ} ξ^{2}|ĝ(ξ)|^{2} dξ = ∫_{ℝ} |g′(x)|^{2} dx = ∞.

Exercise 9.6. (Examples of Balian–Low) Verify the Balian–Low Theorem for the two examples discussed so far: g(x) = e^{2πinx}χ_{[k,k+1)}(x) and g(x) = e^{−inx} sinc x.

A proof of Theorem 9.5 can be found in [Dau92, p. 108, Theorem 4.1.1]. The theorem implies that a Gabor window or bell cannot be simultaneously compactly supported and smooth. The first example, g(x) = χ_{[0,1)}(x), is perfectly localized in time but is not even continuous, and the second example, g(x) = e^{−iπx} sinc(x), is the opposite. In particular the slow decay of the sinc function reflects the lack of smoothness of the characteristic function χ_{[0,1)}. This phenomenon is an incarnation of Heisenberg’s Uncertainty Principle. However, if the exponentials are replaced by appropriate sines and cosines one can obtain Gabor-type bases with smooth bell functions. These are the so-called local sine and cosine bases, described by Coifman and Meyer [CM], but first discovered by Malvar [Malv]. For a very good discussion see the text by Hernández and Weiss [HW].

Exercise 9.7. Show that the Balian-Low Theorem implies that a Gabor function cannot be smmoth and compactly supported.

There is a continuous Gabor transform as well, where the parameters are now real numbers instead of integers. Let g be a real and symmetric window, normalized so that ∥g∥_{2} = 1, let

g_{ξ,u}(x) = g(x − u)e^{2πiξx}, u, ξ ∈ ℝ.

The Gabor transform is then

Gf(ξ, u) = ∫_{ℝ} f(x)g(x − u)e^{−2πiξx} dx = ⟨f, g_{u,ξ}⟩.

The multiplication by the translated window localizes the Fourier integral in a neighborhood of u. The following inversion formula holds for f ∈ L^{2}(ℝ),

(9.3) f(x) = ∫_{ℝ} ∫_{ℝ} Gf(u, ξ)g(x − u)e^{2πixξ} dξ du = ∫_{ℝ^{2}}⟨f, g_{ξ,u}⟩g_{ξ,u}(x) dξ du.

These formulae are similar in spirit to the Fourier transform and the inverse Fourier transform integral formulae in ℝ.

Exercise 9.8. Verify the inversion formula (9.3) for f, g ∈ S(ℝ), g real-valued, symmetric (g(x) = g(−x)), and with L^{2}-norm 1.

Hint: Verify that (9.3) is equivalent to

f(x) = ∫ g_{ξ,0} ∗ g_{ξ,0} ∗ f(x) dξ,

and now verify that the Fourier transform of the right-hand-side is f̂ using the time–frequency dictionary.

9.2. The wavelet transform

Gabor bases give partial answers to the localization issues. A problem is that the size of the windows is fixed (whether all the windows have the same size or not, once a partition of ℝ has been chosen it cannot be changed). Variable widths are the new ingredient added by wavelet analysis.

The wavelet transform involves translations (as in the Gabor basis) and scalings (instead of modulations). The resulting zooming mechanism lies behind the multiresolution structure of these bases that we will explore in detail in Chapter 10.

The goal is to find functions ψ ∈ L^{2}(ℝ) so that the family

(9.4)  ψ_{j,k}(x) = 2^{j/2}ψ(2^{j}x − k),  j, k ∈ ℤ,

forms an orthonormal basis of L^{2}(ℝ).

Exercise 9.9. (Fourier Transform of a Wavelet) Suppose ψ ∈ L^{2}(ℝ). Use the time–frequency dictionary to compute ψ̂_{j,k}(ξ).

The family of Fourier transforms of a wavelet basis is another orthonormal basis, but it is not a wavelet basis. It is generated from one function ψ̂ by scalings and modulations, rather than by scalings and translations.

The orthogonal wavelet transform is given by

W f(j, k) = ⟨f, ψ_{j,k}⟩ = ∫_{ℝ} f(x)\overline{ψ_{j,k}(x)} dx,

and the following reconstruction formula holds in the L^{2}-sense,

f(x) = ∑_{j,k∈ℤ}⟨f, ψ_{j,k}⟩ψ_{j,k}(x).

The oldest example of a wavelet basis is the Haar basis, introduced by Alfréd Haar in 1910 [Haa]. He succeeded in finding an orthonormal basis on L^{2}([0, 1)) that, unlike the trigonometric basis, provides uniform convergence of its partial sums for continuous functions.

Example 9.10. (The Haar Basis) The Haar function h(x) on the unit interval is given by

h(x) := −χ_{[0,1/2)}(x) + χ_{[1/2,1)}(x).

The family {h_{j,k}(x) = 2^{j/2}h(2^{j}x − k)}_{j,k∈ℤ} is an orthonormal basis for L^{2}(ℝ). We will study this example in depth in Section 9.3. ♢

Exercise 9.11. Show that {h_{j,k}} is an orthonormal set:

⟨h_{j,k}, h_{j′,k′}⟩ = \begin{cases}1, & if j = j′ and k = k′; \\ 0, & otherwise. \end{cases}

Also show that the functions h_{j,k} have zero integral: ∫ h_{j,k} = 0.

Wavelet theory can sometimes be thought of as the search for smoother wavelets. The Haar function is discontinuous, perfectly localized in time, but not in frequency.

Exercise 9.12. Find the Fourier transform of the Haar function h(x).

Example 9.13. (The Shannon Basis) At the other end of the spectrum, one finds the Shannon basis. Let ψ be given on Fourier side by

ψ̂(ξ) := e^{2πiξ}χ_{[−1,−1/2)∪[1/2,1)}(ξ).

The family {ψ_{j,k}}_{j,k∈ℤ} is an orthonormal basis for L^{2}(ℝ). ♢

Exercise 9.14. Show that the Shannon functions {ψ_{j,k}} form an orthonormal set, and have zero integral. Furthermore, they are a basis. Hint: Work on Fourier side using the polarization formula (7.26). It can be seen that on Fourier side we are dealing with a windowed Fourier basis, with double-paned windows ω_{j} := [−2^{j+1}, −2^{j}) ∪ [2^{j}, 2^{j+1}) of combined length 2^{j}, for j ∈ ℤ, that are congruent^{1} modulo 2^{j} with the interval [0, 2^{j}). The collection of double-paned windows ω_{j}, provide a partition of ℝ\{0}. On each double-paned window, the trigonometric functions

\frac{1}{2^{j/2}} e^{2πikx2^{−j}}χ_{ω_{j}}(x), k ∈ ℤ,

form an orthonormal basis of L^{2}(ω_{j}).

The Shannon wavelets are perfectly localized in frequency, but not in space. The goal is to find functions that are simultaneously localized in time and in

frequency (within the limits imposed by the Heisenberg Principle). Compactly supported wavelets with arbitrary smoothness were constructed by I. Daubechies^{2} in a fundamental paper in wavelet theory [Dau88]. We can associate to most wavelets a sequence of numbers, known as a filter. The filters of compactly supported wavelets are zero except for finitely many entries. In the engineering community, such filters are called finite impulse response (FIR) filters. The connections to filter bank theory and the possibility of implementing FIR filters opened the door to widespread use of wavelets in applications. We will explore this connection in Section 10.3.

We can develop the theory of wavelets in ℝ^{N} or ℂ^{N} (linear algebra!), in the same way as we built a finite Fourier theory and introduced the discrete Haar basis in Chapter 6. This is done in full detail in the book by Frazier [Fra]. This is what ends up being implemented. Not only is there a Fast Fourier Transform (FFT), but it turns out that there is also a Fast Wavelet Transform (FWT) which has been instrumental in the success of wavelets in the “real world”. Let us just mention two of the most popular applications: both the FBI fingerprint data base and retrieval system, and the JPEG 2000 Standard for image compression, are based on wavelets.

As in the Fourier and Gabor cases, there is a continuous wavelet transform. In this case we have continuous translation and scaling parameters, s ∈ ℝ^{+} (that is, s > 0), u ∈ ℝ, and a family of time–frequency atoms that is obtained by rescaling by s and shifting by u a normalized wavelet ψ ∈ L^{2}(ℝ) with zero average (∫ ψ = 0).

Let

ψ_{s,u}(x) = \frac{1}{\sqrt{s}}ψ (\frac{x − u}{s}) , (ψ_{s,u})^{∧}(ξ) = \sqrt{s}e^{−2πiuξ}ψ̂(sξ).

The continuous wavelet transform is then defined by

W f(s, u) = ⟨f, ψ_{s,u}⟩ = ∫_{ℝ} f(x)\overline{ψ_{s,u}(x)} dx.

1The intervals [−1/2, 0) ∪ [1/2, 1) are not congruent modulo 1 to the interval [0, 1).

2Ingrid Daubechies, 1954–, Belgian mathematician.

If ψ is real valued, and it is localized near 0 with spread 1, then ψ_{s,u} is localized near u with spread s. The wavelet transform measures the variation of f near u at scale s (in the orthonormal case, u = k2^{−j} and s = 2^{−j}). As the scale s goes to zero, or equivalently as j goes to infinity, the decay of the wavelet coefficients characterizes the regularity of f near u (in the discrete case k2^{−j}). On the other hand, if ψ̂ is localized near 0 with spread 1, then ψ̂_{s,u} is localized near 0 with spread 1/s. That is, the Heisenberg boxes or phase-plane portrait of the wavelets are rectangles of area one and dimensions s × 1/s.

Under very mild assumptions on the wavelet ψ, we obtain a reconstruction formula. For any f ∈ L^{2}(ℝ),

(9.5)  f(x) = \frac{1}{C_{ψ}} ∫_{0}^{∞} ∫_{−∞}^{+∞} W f(s, u)ψ_{s,u}(x)\frac{duds}{s^{2}},

provided that ψ satisfies Calderón’s admissibility condition [Cal]:

C_{ψ} := ∫_{0}^{∞} \frac{|ψ̂(ξ)|^{2}}{ξ} dξ < ∞.

The reconstruction formula can be traced back to the famous Calderón reproducing formula given by,

(9.6)  f(x) = \frac{1}{C_{ψ}} ∫_{0}^{∞} ψ_{s,0} ∗ \overline{ψ̃_{s,0}} ∗ f(x) \frac{ds}{s^{2}},

where ψ̃(x) = ψ(−x), and ∗ denotes convolution in ℝ.

Exercise 9.15. Verify that Calderón’s reproducing formula (9.6), and the reconstruction formula (9.5) are the same. Verify now that (9.6) holds for f, ψ ∈ S(ℝ) such that c_{ψ} < ∞, by checking that the Fourier transform of the right-hand-side coincides with f̂.

9.3. Haar analysis

In this section we discuss the Haar basis in detail. In particular we do show that the Haar functions form a complete orthonormal system. Verifying the orthonormality of the system is reduced to understanding the geometry of the dyadic intervals. Verifying the completeness of the system is reduced to understanding that the limit in L^{2}(ℝ) of the averaging operators over intervals as the interval shrinks to a point x ∈ ℝ is the identity operator, and the limit as the intervals grow to be infinitely long is the zero operator.

For these to make sense we have to first define the dyadic intervals, and describe their geometry. We then define the expectation (or averaging) and difference operators, and reduce the question of completeness of the Haar system to understanding the limiting behaviour of the averaging operators. We prove the required limit results for continuous and compactly supported functions, finally an approximation argument coupled with some uniform bounds give the desired result.

9.3.1. The dyadic intervals. Intervals of the form

I_{j,k} = [k2^{−j}, (k + 1)2^{−j}), for integers j, k,

are called dyadic intervals. For instance, [5/8, 3/4) and [−16, −12) are dyadic intervals, while [3/8, 5/8) is not. Each dyadic interval is half-open: I_{j,k} contains its left endpoint but not its right. The collection of all dyadic intervals is denoted by D, and D_{j} denotes the collection of dyadic intervals I of length 2^{−j}, also called the j^{th} generation. It is clear that each D_{j} forms a partition of the real line, and that

D = ⋃_{j∈ℤ} D_{j}.

Given two distinct intervals I, J ∈ D, then either I and J are disjoint or one is contained in the other. Each dyadic interval I is in a unique generation D_{j}, and there are exactly two subintervals of I in the next generation D_{j+1}. These are the children of I, denoted by I_{l} and I_{r} for the left and right child respectively. See Figure 9.3. Clearly, I = I_{l} ∪ I_{r}. We denote by |I| the length of the interval I.

I I_{l}  I_{r}

Figure 9.3. Parent dyadic interval I and its children I_{l} and I_{r}.

Exercise 9.16. (Dyadic Intervals Are Nested Or Disjoint) Show that if I, J ∈ D, then exactly one of the following is true: I ⋂ J = ∅, or I ⊆ J, or J ⊂ I.

Given J ∈ D, we denote by D(J) the collection of dyadic intervals that are contained in J.

Given x ∈ ℝ and j ∈ ℤ, there is a unique interval I ∈ D_{j} such that x ∈ I. We denote this unique interval by I_{j}(x).

9.3.2. The Haar basis. Associated to each dyadic interval I there is a Haar function h_{I} defined by

h_{I}(x) := \frac{1}{|I|^{1/2}} [χ_{I_{r}}(x) − χ_{I_{l}}(x)].

Figure 9.4 shows the graphs of two Haar functions.

Exercise 9.17. Let I_{j,k} = [k2^{−j}, (k + 1)2^{−j}). Show that

h_{I_{j,k}}(x) = 2^{j/2}h(2^{j}x − k) = h_{j,k}(x), where h = h_{[0,1)}.

In Exercise 9.11, you were asked to check that {h_{j,k}}_{j,k∈ℤ} is an orthonormal set in L^{2}(ℝ). Exercise 9.17 shows that {h_{I}}_{I∈D} is the same set of functions; hence it is an orthonormal set as well. However we can prove directly now that the Haar functions indexed on the dyadic intervals form an orthonormal family using the fact that the dyadic intervals are nested or disjoint (Exercise 9.16). In fact, consider I, J ∈ D, and I ≠ J, then either they are disjoint or one is strictly contained in the other. If I and J are disjoint then clearly ⟨h_{I}, h_{J}⟩ = 0 because the supports are disjoint. If, say I is strictly contained in J, then necessarily I is contained in one Figure 9.4. Graphs of the two Haar functions defined by h_{[−2,−1.5)}(x) = 2 [χ_{[−1.75,−1.5)}(x) − χ_{[−2,−1.75)}(x)] and h_{[0,2)}(x) = \sqrt{2} [χ_{[1,2)}(x) − χ_{[0,1)}(x)].

of J^{th} children, say, to fix ideas the right child, but then, the inner product is the constant value of h_{J} on its right child times the integral of h_{I} which vanishes, i.e.

⟨h_{I}, h_{J}⟩ = ∫_{I} h_{I}(x)h_{J}(x) dx = \frac{1}{|J|^{1/2}} ∫ h_{I}(x) dx = 0.

Not only the Haar system is orthonormal it is also a complete orthonormal system, hence a basis of L^{2}(ℝ).

Theorem 9.18. The Haar functions {h_{I}}_{I∈D} form an orthonormal basis in L^{2}(ℝ).

In Chapter 6 we discussed the discrete Haar basis on ℂ^{N}. We had N Haar vectors that were proven to be orthonormal, and hence a basis. In the finitedimensional case, we can count the elements in the orthonormal set, and if that number coincides with the dimension, we know we have a basis. In infinite-dimensional space we do not have that luxury. To prove Theorem 9.18, we must make sure the set is complete. This means that for all f ∈ L^{2}(ℝ), the following identity must hold in the L^{2}-sense^{3}:

(9.7)  f = ∑_{I∈D}⟨f, h_{I}⟩h_{I}.

An alternative will be to show that the only square integrable function orthogonal to all Haar functions is the zero function (see Theorem B.24). We will show that both arguments boil down to checking some limit properties of the expectation operators defined in Section 9.3.3 below.

3Recall that equation (9.7) holds in the L^{2}-sense if ∥f − ∑_{I∈D} ⟨f, h_{I}⟩h_{I}∥_{2} = 0.

Before we go on to prove the completeness of the Haar system, let us play the Devil’s advocate.

• First, consider the function f(x) = 1. Then ⟨f, h_{I}⟩ = ∫ h_{I} = 0 for all I ∈ D. We have found a function that is orthogonal to all the Haar functions, so how can the system be complete? Are we contradicting the theorem? No, because the function that is identically equal to 1 on ℝ is not in L^{2}(ℝ)!

• Second, how can it be true that functions that have zero integral (the Haar functions) can reconstruct functions that do not have zero integral?^{4}

If the Haar system is complete, then for any f ∈ L^{2}(ℝ), equation (9.7) holds. Integrating on both sides and interchanging the sum and the integral, we see that

∫_{ℝ} f(x) dx = ∫_{ℝ} ∑_{I∈D}⟨f, h_{I}⟩h_{I}(x) dx

= ∑_{I∈D}⟨f, h_{I}⟩ ∫_{ℝ} h_{I}(x) dx
= 0,

where the last equality holds because the Haar functions have integral zero. We seem to be implying that all functions in L^{2}(ℝ) must themselves have integral zero. But we know this is not true, since for example χ_{[0,1)} ∈ L^{2}(ℝ) and has integral one. What’s wrong? Perhaps the Haar system is not complete after all. Or is there something wrong in the above calculation? The Haar system is complete; it turns out that what is illegal above is the interchange of sum and integral.

For comfort, let us check that ∑_{I∈D}⟨χ_{[0,1)}, h_{I}⟩h_{I} coincides with χ_{[0,1)} pointwise,

furthermore we will see that the partial sums converge uniformly to χ_{[0,1)}.

Exercise 9.19. Verify that for I ∈ D

⟨χ_{[0,1)}, h_{I}⟩ = \begin{cases}∫_{0}^{1} h_{I}(x) dx, & if [0, 1) ⊂ I, I ≠ [0, 1); \\ 0, & otherwise. \end{cases}

Which dyadic intervals I strictly contain the interval [0, 1)? Only those dyadic intervals of the form I = [0, 2^{n}) =: I^{n}, for n ≥ 1. Notice that |I^{n}| = 2^{n}, and that the unit interval is contained on the left half of I^{n} for all n ≥ 1, that is [0, 1) ⊂ I_{l}^{n} = [0, 2^{n−1}). So if x ∈ [0, 1), then h_{I^{n}}(x) = −1/\sqrt{2^{n}}. This observation and Exercise 9.19 show that

⟨χ_{[0,1]}, h_{I}⟩ = \begin{cases}−\frac{1}{\sqrt{2^{n}}} , & if I = I^{n}, n ≥ 1; \\ 0, & otherwise. \end{cases}

4This question was posed by Lindsay Crowl, one of the participants in the 2004 Program for Women in Mathematics where we gave the lectures that lead to this book. We decided it was a perfectly natural concern, and furthermore it was very illustrative of the dangers of interchanging limiting operations!

Bearing in mind that h_{I^{n}}(x) = \frac{1}{\sqrt{2^{n}}} [ − χ_{[0,2^{n−1})}(x) + χ_{[2^{n−1},2^{n})}(x)], we can now evaluate pointwise the series:

∑_{I∈D}⟨χ_{[0,1]}, h_{I}⟩h_{I}(x) = ∑_{n=1}^{∞}⟨χ_{[0,1]}, h_{I^{n}}⟩h_{I^{n}}(x)

(9.8)  = ∑_{n=1}^{∞} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)].

We claim that the last term in equation (9.8) is equal to χ_{[0,1)}(x) for each x ∈ ℝ. To justify this, let us evaluate the sum term by term for different values of x.

For x < 0: Clearly h_{I^{n}}(x) = 0 for all n ≥ 0, and so

∑_{n=1}^{∞}⟨χ_{[0,1]}, h_{I^{n}}⟩h_{I^{n}}(x) = 0.

For 0 ≤ x < 1: We have already mentioned that [0, 1) sits on I_{l}^{n}, the left half of I^{n}, for all n ≥ 1. Hence

∑_{n=1}^{∞} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)] = ∑_{n=1}^{∞} \frac{1}{2^{n}} = 1.

For 1 ≤ x < 2: This time [1, 2) = I_{r}^{1}, and [1, 2) ⊂ I_{ℓ}^{n} for all n ≥ 2. Therefore

∑_{n=1}^{∞} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)] = −\frac{1}{2} + ∑_{n=2}^{∞} \frac{1}{2^{n}} = 0.

For 2 ≤ x < 2^{2}: Here [2, 2^{2}) ∩ I^{1} = ∅, [2, 2^{2}) = I_{r}^{2}, and [2, 2^{2}) ⊂ I_{ℓ}^{n} for all n ≥ 3. Therefore

∑_{n=1}^{∞} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)] = −\frac{1}{2^{2}} + ∑_{n=3}^{∞} \frac{1}{2^{n}} = 0.

The pattern is now clear.

For 2^{k} ≤ x < 2^{k+1}: We have [2^{k}, 2^{k+1}) ∩ I^{n} = ∅ for n < k, [2^{k}, 2^{k+1}) = I_{r}^{k}, and [2^{k}, 2^{k+1}) ⊂ I_{ℓ}^{n} for all n ≥ k + 1. Therefore

∑_{n=1}^{∞} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)] = −\frac{1}{2^{k}} + ∑_{n=k+1}^{∞} \frac{1}{2^{n}} = 0.

Hence we obtain that, pointwise,

∑_{I∈D}⟨χ_{[0,1]}, h_{I}⟩h_{I}(x) = χ_{[0,1)}(x).

Furthermore, we have uniform convergence on ℝ of the functions

f_{N}(x) := ∑_{n=1}^{N} \frac{1}{2^{n}} [χ_{[0,2^{n−1})}(x) − χ_{[2^{n−1},2^{n})}(x)],

to χ_{[0,1)}(x).

Exercise 9.20. Verify that

f_{N}(x) = \{ \begin{matrix}1 − 2^{−N} & x ∈ [0, 1) \\ −2^{−N} & x ∈ [1, 2^{N}) \\ 0 & otherwise \end{matrix} .

Check that ∫_{ℝ} f_{N}(x) dx = 0.

Exercise 9.21. Verify that f_{N} → χ_{[0,1)} uniformly on ℝ.

It is clear that we cannot interchange limit and integral despite having the uniform convergence of f_{N},

lim_{N→∞} ∫_{ℝ} f_{N}(x) dx = 0 ≠ 1 = ∫_{ℝ} χ_{[0,1)}(x) dx.

But wait, doesn’t uniform convergence guarantee the interchange of the limit and the integral? NO, only when integrating on a compact set, and the line is not compact.

Exercise 9.22. (i) Show that f_{N} cannot converge to χ_{[0,1)} in L^{1}(ℝ).

(ii) Show that f_{N} converges to χ_{[0,1)} in L^{2}(ℝ), that is

∥f_{N} − χ_{[0,1)}∥_{L^{2}(ℝ)} → 0.

More precisely, verify that

∥f_{N} − χ_{[0,1)}∥_{L^{2}(ℝ)} = 2^{−N/2}.

There is a very important result in the Lebesgue integration theory on ℝ that we stated for intervals in Chapter 2, the Lebesgue Dominated Convergence Theorem. This theorem states that if a sequence of measurable functions f_{n} converges pointwise almost everywhere to a function f (which must be measurable because pointwise a.e. limits preserve measurability), AND there is an integrable function g that dominates pointwise all the functions, namely, |f_{n}(x)| ≤ g(x), then the interchange of limit and integral is legal, i.e. lim_{n→∞} ∫ f_{n} = ∫ f. We know in our case, that the interchange does not hold, so it better be that there is NO dominating function g. That is the content of the following exercise.

Exercise 9.23. Verify that if g : ℝ → [0, ∞) has the property that |f_{N}(x)| ≤ g(x) for all x ∈ ℝ and for all N > 1, then g(x) ≥ g_{0}(x), where g_{0}(x) = 1 if x ∈ [0, 1), g_{0}(x) = 2^{−j} if x ∈ [2^{j−1}, 2^{j}) for all j ≥ 1, and zero when x < 0. Now verify that g_{0} is not integrable, hence no dominating function g can be integrable.

9.3.3. The expectation and difference operators, P_{j} and Q_{j}. We will introduce here two important operators^{5} that will help us to understand the zooming properties of the Haar basis.

The expectation operators P_{j} : L^{2}(ℝ) → L^{2}(ℝ), j ∈ ℤ are nothing more than averages over dyadic intervals at generation j:

(9.9)  P_{j}f(x) := \frac{1}{|I|} ∫_{I} f(t) dt,

where I is the unique interval of length 2^{−j} containing x.

5An operator is a mapping from a space of functions into another space of functions. The input is a function and so is the output. The Fourier transform is an operator, and we discussed in Section 8 its mapping properties for different functional spaces. In particular its action on L^{p}-spaces is summarized in the table on page 153.

Notice that the new function P_{j}f is a step function with steps on dyadic intervals I ∈ D_{j}. Figure 9.5 shows the graph of a specific function f together with the graph of P_{j}f. In this example j = −1.

Exercise 9.24. Verify that

P_{j}f(x) = ∑_{I∈D_{j}} m_{I}f χ_{I}(x),

where m_{I}f denotes the average of f on the interval I.

Figure 9.5. Graphs of f and P_{n}f. [***Add the formula for this f, also n = j.]

As j → ∞, the size of the steps goes to zero, and we would expect P_{j}f to be a better and better approximation of f. We will make this precise in Section 9.3.4 below.

The difference operators Q_{j} : L^{2}(ℝ) → L^{2}(ℝ), j ∈ ℤ are given by

(9.10)  Q_{j}f(x) := P_{j+1}f(x) − P_{j}f(x).

These operators Q_{j} encode the information necessary to go from the approximation P_{j}f at resolution j of f to the better approximation P_{j+1}f at resolution j + 1. Figure 9.6 shows the graph of the same function f as in Figure 9.5, together with the graph of Q_{j}f, again for j = −1. Figure 9.7 shows the graphs of f, P_{j}f, and P_{j+1}f = P_{j}f + Q_{j}f.

Notice that when we superimpose the pictures of P_{j+1}f and P_{j}f, the averages at the coarser scale j seem to be sitting exactly halfway between the averages at the finer scale, so that Q_{j}f seems to be a linear combination of the Haar functions at scale j. This is indeed the case, and the next Lemma takes care of the details of this impressionistic comment.

170 9. FROM FOURIER TO HAAR

Figure 9.6. Graphs of f and Q_{n}f.

Figure 9.7. Graphs of f, P_{n}f, and P_{n+1}f. [***n = j]

Lemma 9.25. For f ∈ L^{2}(ℝ),

Q_{j}f(x) = ∑_{I∈D_{j}}⟨f, h_{I}⟩h_{I}(x).

Proof. Recall that m_{I}f denotes the average of the function f over the interval I:

m_{I}f = \frac{1}{|I|} ∫_{I} f(x) dx.

By definition of the Haar functions, and noting that |I| = 2|I_{r}| = 2|I_{l}|, we see that

⟨f, h_{I}⟩h_{I}(x) = \frac{|I|^{1/2}}{2} (\frac{1}{|I_{r}|} ∫_{I_{r}} f − \frac{1}{|I_{l}|} ∫_{I_{l}} f) h_{I}(x) = \frac{|I|^{1/2}}{2} (m_{I_{r}}f − m_{I_{l}}f)h_{I}(x).

Since h_{I}(x) = |I|^{−1/2} if x ∈ I_{r}, and h_{I}(x) = −|I|^{−1/2} if x ∈ I_{l}, we conclude that if x ∈ I, then

(9.11)  ⟨f, h_{I}⟩h_{I}(x) = \begin{cases}\frac{1}{2}(m_{I_{r}}f − m_{I_{l}}f), & if x ∈ I_{r}; \\ −\frac{1}{2}(m_{I_{l}}f − m_{I_{r}}f), & if x ∈ I_{l}. \end{cases}

On the other hand, if x ∈ I ∈ D_{j}, then P_{j}f(x) = m_{I}f, and

P_{j+1}f(x) = \begin{cases}m_{I_{r}}f, & if x ∈ I_{r}; \\ m_{I_{l}}f, & if x ∈ I_{l}. \end{cases}

Hence

Q_{j}f(x) = \begin{cases}m_{I_{r}} − m_{I}f, & if x ∈ I_{r}; \\ m_{I_{l}} − m_{I}f, & if x ∈ I_{l}. \end{cases}

Here is a useful averaging property of integral averages on dyadic intervals:

(9.12)  m_{I}f = \frac{m_{I_{l}}f + m_{I_{r}}f}{2} .

In other words, the average m_{I}f of f over I sits half-way between the averages m_{I_{l}}f and m_{I_{r}}f over the children. Informally, the integral average over a parent interval is the average of the integral averages over its children.

This averaging property implies that

m_{I_{r}}f − m_{I}f = \frac{m_{I_{r}}f − m_{I_{l}}f}{2} = m_{I}f − m_{I_{l}}f.

Hence

Q_{j}f(x) = \begin{cases}\frac{1}{2}(m_{I_{r}}f − m_{I_{l}}f), & if x ∈ I_{r}; \\ −\frac{1}{2}(m_{I_{l}}f − m_{I_{r}}f), & if x ∈ I_{l}. \end{cases}

Comparing to (9.11) we conclude that

Q_{j}f(x) = ⟨f, h_{I}⟩h_{I}(x) for x ∈ I ∈ D_{j}.

This certainly proves the lemma. □

Exercise 9.26. Verify the averaging property in equation (9.12).

9.3.4. Completeness of the Haar system. The Haar system of functions is complete if, for all f ∈ L^{2}(ℝ), we have

f(x) = ∑_{I∈D}⟨f, h_{I}⟩h_{I}(x).

By Lemma 9.25, this condition is equivalent to the condition

f(x) = lim_{M,N→∞} ∑_{−M≤j<N} Q_{j}f(x).

A telescoping series argument shows that

(9.13) P_{N}f(x) − P_{M}f(x) = ∑_{M≤j<N} (P_{j+1}f(x) − P_{j}f(x)) = ∑_{M≤j<N} Q_{j}f(x).

Therefore, verifying completeness of the Haar system reduces to checking that

f(x) = lim_{N→∞} P_{N}f(x) − lim_{M→−∞} P_{M}f(x),

where all the above equalities hold in the L^{2}-sense.

We need only prove the following theorem.

Theorem 9.27. For f ∈ L^{2}(ℝ),

(9.14)  lim_{M→−∞} ∥P_{M}f∥_{2} = 0

and

(9.15)  lim_{N→∞} ∥P_{N}f − f∥_{2} = 0.

Exercise 9.28. Use Theorem 9.27 to show that, if f ∈ L^{2}(ℝ) is orthogonal to all Haar functions, then f must be zero in L^{2}(ℝ). (We already observed that f(x) = 1 is orthogonal to all Haar functions, however is not a square integrable function.)

Recall that I_{j}(x) is the unique dyadic interval in D_{j} that contains x. Figure 9.8 shows part of the “tower” of dyadic intervals {I_{j}(x)}_{j∈ℤ} shrinking to {x} as j goes to infinity. Equation (9.15) says that given x ∈ ℝ, the averages of the function over the dyadic intervals {I_{j}(x)}_{j∈ℤ} converge to f in the L^{2}-sense as the intervals shrink:

lim_{j→∞} \frac{1}{|I_{j}(x)|} ∫_{I_{j}(x)} f(t) dt = f(x).

It turns out that the convergence also holds almost everywhere^{6} (a.e.). This is the content of the celebrated Lebesgue Differentiation Theorem in ℝ, which we now quote,

Theorem 9.29 (Lebesgue Dominated Convergence Theorem in ℝ). If f ∈ L^{1}(ℝ) (locally integrable suffices), then

lim_{x∈I,|I|→0} \frac{1}{|I|} ∫_{I} f(y) dy = f(x) a.e. x ∈ ℝ.

Here I denotes any interval (dyadic or not) containing x.

We stated in Chapter 4 this theorem for intervals centered at x, [x − h, x + h], and h → 0. For a proof see [SS2, Chapter 3].

6In other words, the convergence holds except on a set of measure zero.

⋮  ⋮ ⋮  ⋮ I_{n}(x) I_{n+1}(x) I_{n+2}(x) I_{n+3}(x) x k/2^{n}  (k + 1)/2^{n}

Figure 9.8. Part of the tower of nested dyadic intervals ⋯ ⊃ I_{n}(x) ⊃ I_{n+1}(x) ⊃ I_{n+2}(x) ⊃ I_{n+3}(x) ⊃ ⋯ containing the point x, with k/2^{n} ≤ x ≤ (k + 1)/2^{n}. Here n and k are integers.

Exercise 9.30. Verify the Lebesgue Differentiation Theorem for continuous functions, and show that for continuous functions the pointwise convergence holds everywhere. That is, for all x ∈ ℝ,

(9.16)  lim_{[a,b]→\{x\}} \frac{1}{b − a} ∫_{a}^{b} f(t) dt = f(x).

Here the intervals [a, b] need not be dyadic.

Theorem 9.27 is a consequence of the following lemmata.

Lemma 9.31. For every function f ∈ L^{2}(ℝ) and every integer j, the operators P_{j} are uniformly bounded in L^{2}(ℝ). More precisely,

∥P_{j}f∥_{2} ≤ ∥f∥_{2}.

Lemma 9.32. If g is continuous and has compact support on the interval [−K, K], then Theorem 9.27 holds.

Lemma 9.33. The continuous functions with compact support are dense in L^{2}(ℝ). More precisely, given f ∈ L^{2}(ℝ), for any ε > 0 there exist functions g and h, such that f = g + h, where g is a continuous function with compact support on an interval [−K, K], and h ∈ L^{2}(ℝ) with small L^{2}-norm, ∥h∥_{2} < ε.

We first prove the theorem, and then the lemmata.

Proof of Theorem 9.27. By Lemma 9.33, given ε > 0 we can decompose f = g + h, where g is continuous with compact support on [−K, K], and h ∈ L^{2}(ℝ) with ∥h∥_{2} < ε/4.

By Lemma 9.32, we can choose N large enough so that for all j > N,

∥P_{−j}g∥_{2} ≤ ε/2.

Now apply the triangle inequality, and use Lemma 9.31 to conclude that

∥P_{−j}f∥_{2} ≤ ∥P_{−j}g∥_{2} + ∥P_{−j}h∥_{2} ≤ \frac{ε}{2} + ∥h∥_{2} ≤ ε.

This proves equation (9.14).

Similarly, by Lemma 9.32, we can choose N large enough that for all j > N,

∥P_{j}g − g∥_{2} ≤ ε/2.

Now apply the triangle inequality, and use Lemma 9.31 to conclude that

∥P_{j}f − f∥_{2} ≤ ∥P_{j}g − g∥_{2} + ∥P_{j}h − h∥_{2} ≤ \frac{ε}{2} + 2∥h∥_{2} ≤ ε.

This proves equation (9.15). □

We have twice used a very important principle from functional analysis: If a sequence of linear operators is uniformly bounded in a Banach space, and the sequence converges to a bounded operator on a dense subset of the Banach space, then it converges in the whole space to the same operator. This principle is called the Uniform Boundedness Principle, or the Banach–Steinhaus Theorem In our case the Banach space is L^{2}(ℝ), the dense subset is the set of continuous functions with compact support, the linear operators are P_{j}, and the uniform bounds are provided by Lemma 9.31. In one case the operators converge to the zero operator (as n → −∞), and in the other case they converge to the identity operator (as n → ∞).

Here is a precise statement of the Uniform Boundedness Principle that you can find in [Sch, Chapter III],

Theorem 9.34 (Uniform Boundedness Principle). Let W be a family of bounded linear operators T : X → Y from a Banach space X into a normed space Y, such that for each x ∈ X,

sup_{T ∈W} ∥T x∥_{Y} < ∞.

Then the operators are uniformly bounded, that is, there exists a constant C > 0 such that for all T ∈ W, and all x ∈ X,

∥T x∥_{Y} ≤ C∥x∥_{X}.

In particular if the operators {T_{n}}_{n≥1} are uniformly bounded, that is ∥T_{n}x∥_{Y} ≤ C∥x∥_{X}, and lim_{n→∞} ∥T_{n}x∥_{Y} = 0, for all x on a dense subset A ⊂ X, then lim_{n→∞} ∥T_{n}x∥_{Y} = 0 for all x ∈ X. Why? Consider a point x ∈ X. There are points x_{m} ∈ A such that lim_{m→∞} ∥x_{m} − x∥_{X} = 0, and for those we know that ∥T_{n}x_{m}∥_{Y} → 0 as n → ∞ for all m. Now, x = (x − x_{m}) + x_{m}, the operators are linear, hence T_{n}x = T_{n}(x − x_{m}) + T_{n}(x_{m}). We can estimate the norm of T_{n}x using the triangle inequality, and the uniform boundedness,

∥T_{n}x∥_{Y} ≤ ∥T_{n}(x − x_{m})∥_{Y} + ∥T_{n}x_{m}∥_{Y} ≤ C∥x − x_{m}∥_{Y} + ∥T_{n}x_{m}∥_{Y} .

Both terms on the right hand side can be made arbitrarilly small. First by choosing M large enough we can ensure that for a given ϵ

∥x − x_{m}∥_{Y} < ϵ/2C, for all m > M.

second fix m > M and now choose N large enough so that

∥T_{n}x_{m}∥_{Y} < ϵ/2.

We conclude then that for all ϵ > 0, and for all n > N, ∥T_{n}x∥_{Y} < ϵ, that is

lim_{n→∞} ∥T_{n}x∥_{Y} = 0.

A beautiful application of the Uniform Boundedness Principle is to show the existence of a real-valued continuous periodic function whose Fourier series diverges at a given point x_{0}. We indicate in the next exercise how to deduce this.

Exercise 9.35. Let X be the Banach space of all real-valued continuous functions of period 2π with uniform norm, that is X = C(𝕋), and let Y = ℂ. Let f ∈ C(𝕋), define T_{N}(f) = S_{N}f(0) ∈ ℂ, where S_{N}f denotes the N^{th} partial Fourier sum of f. Recall that S_{N}f = D_{N} ∗ f where D_{N} denotes the periodic Dirichlet kernel (see Chapter 4). Verify that if C_{N} > 0 is such that

|T_{N}f| ≤ C_{N}∥f∥_{∞},

then necessarily C_{N} ≥ c∥D_{N}∥_{L^{1}(𝕋)}. But we showed in Chapter 4 that ∥D_{N}∥_{L^{1}(𝕋)} ∼ log N. This implies that the operators T_{N} cannot be uniformly bounded, hence there must exist f ∈ C(𝕋) such that

sup_{N≥0} |S_{N}f(0)| = ∞.

Therefore for this function f the partial Fourier sums do not converge at x = 0.

Proof of Lemma 9.31. For x ∈ I ∈ D_{j},

|P_{j}f(x)|^{2} = |\frac{1}{|I|} ∫_{I} f(t) dt|^{2} ≤ \frac{1}{|I|^{2}} (∫_{I} 1^{2} dt) (∫_{I} |f(t)|^{2} dt) = \frac{1}{|I|} ∫_{I} |f(t)|^{2} dt.

The inequality is a consequence of the Cauchy–Schwarz inequality.

Now integrate over the interval I to obtain

∫_{I} |P_{j}f(x)|^{2}dx ≤ ∫_{I} |f(t)|^{2}dt,

and add over all intervals in D_{j} (this is a disjoint family that covers the whole line!):

∫_{ℝ} |P_{j}f(x)|^{2}dx = ∑_{I∈D_{n}} ∫_{I} |P_{j}f(x)|^{2} dx ≤ ∑_{I∈D_{n}} ∫_{I} |f(t)|^{2}dt = ∫_{ℝ} |f(t)|^{2} dt.

The lemma is proved. □

Proof of Lemma 9.32. The function g is continuous and has support on the interval [−K, K]. If j is large enough so that K < 2^{j} and x ∈ [0, 2^{j}) ⊂ D_{−j}, then

|P_{−j}g(x)| = \frac{1}{2^{j}} ∫_{0}^{K} |g(t)|dt

≤ \frac{1}{2^{n}} (∫_{0}^{K} 1^{2} dt)^{1/2} (∫_{0}^{K} |g(t)|^{2} dt)^{1/2}
≤ \frac{1}{2^{j}} \sqrt{K}∥g∥_{2},

where the last inequality is another application of the Cauchy–Schwarz inequality. The same inequality holds for x < 0. If |x| ≥ 2^{j} then P_{−j}g(x) = 0.

We can now estimate the L^{2}-norm of P_{−j}g:

∥P_{−j}g∥_{2}^{2} = ∫_{−2^{j}}^{2^{j}} |P_{−j}g(x)|^{2} dx ≤ \frac{1}{2^{2j}} K∥g∥_{2}^{2} ∫_{−2^{j}}^{2^{j}} 1 dx = 2^{−j+1}K∥g∥_{2}^{2}.

By choosing N large enough, we can make 2^{−N+1}K∥g∥_{2}^{2} < ε^{2}. That is, given ε > 0, there is an N > 0 such that for all j > N,

∥P_{−j}g∥_{2} ≤ ε.

This proves equation (9.14) for continuous functions with compact support.

We are assuming that g is continuous and supported on the compact interval [−K, K] ⊂ [2^{−M}, 2^{M}]. But then g is uniformly continuous. That is, given ε > 0 there exists δ > 0 such that

|g(y) − g(x)| < ε/\sqrt{2^{M+1}}whenever |y − x| < δ.

Now choose N > M large enough that 2^{−j} < δ for all j > N. Each point x is contained in a unique I ∈ D_{j}, with |I| = 2^{−j} < δ. Therefore |y − x| ≤ δ for all y ∈ I, and

|P_{j}g(x) − g(x)| ≤ \frac{1}{|I|} ∫_{I} |g(y) − g(x)| dy ≤ \frac{ε}{\sqrt{2^{M+1}}} .

Squaring and integrating over ℝ, we get

∫_{ℝ} |P_{j}g(x) − g(x)|^{2}dx = ∫_{−2^{M}}^{2^{M}} |P_{j}g(x) − g(x)|^{2}dx < \frac{ε^{2}}{2^{M+1}} ∫_{−2^{M}}^{2^{M}} 1 dx = ε^{2}.

Notice that if |x| > 2^{M}, then for n > N ≥ M, P_{j}g(x) is the average over an interval I ∈ D_{j} that is completely outside the support of g. For such x and j, P_{j}g(x) = 0, and therefore there is zero contribution to the integral from |x| > 2^{M}.

Lo and behold, we have shown that given ε > 0, there is an N > 0 such that for all n > N,

∥P_{j}g − g∥_{2} ≤ ε.

This proves equation (9.15) for continuous functions with compact support. □

Proof of Lemma 9.33. This lemma is an example of an approximation theorem in L^{2}(ℝ). How do we achieve it? We choose K large enough so that the “tail” of f has very small L^{2}-norm, in other words ∥fχ_{{x∈ℝ:|x|>K}}∥_{2} ≤ ε/3. Next we recall that on compact intervals, the continuous functions are dense in L^{2}([−K, K]); see Theorem 2.66. (For example, polynomials are dense, and so are trigonometric polynomials, by the Weierstrass approximation theorem (Theorem 3.4).) Now choose g_{1} continuous on [−K, K] so that ∥fχ_{[−K,K]} − g_{1}∥_{2} ≤ ε/3. It could happen that g_{1} is continuous on [−K, K], but when extended to be zero outside the interval, it is not continuous on the line. That can be fixed by giving yourself some margin at the endpoints: define g to coincide with g_{1} on [−K + δ, K − δ] and to be zero outside [−K, K], and connect these pieces with straight segments, so that g is continuous on ℝ. Finally, choose δ small enough so that ∥g_{1} − g∥_{2} ≤ ε/3. Now let h = f − g = fχ_{{x∈ℝ:|x|>K}} + fχ_{[−K,K]} − g_{1} + g_{1} − g. By the triangle inequality,

∥h∥_{2} ≤ ∥fχ_{{x∈ℝ:|x|>K}}∥_{2} + ∥fχ_{[−K,K]} − g_{1}∥_{2} + ∥g_{1} − g∥_{2} ≤ ε.

□

We have shown (Lemma 9.32) that the step functions can approximate continuous functions with compact support in the L^{2}-norm. Lemma 9.33 shows that we can approximate L^{2}-functions by continuous functions with compact support. Therefore, we can approximate L^{2}-functions by step functions, in the L^{2}-norm. Furthermore, we can choose the steps to be dyadic intervals of a fixed generation for any prescribed accuracy.

Exercise 9.36. (Approximation by Step Functions) Show that continuous functions with compact support can be approximated in the uniform norm by step functions. Furthermore, one can choose the intervals where the approximating function is constant to be dyadic intervals of a fixed generation for any prescribed accuracy. More precisely, show that given f continuous on ℝ, and ε > 0, there exists N > 0 such that for all j > N, and for all x ∈ ℝ,

|P_{j}f(x) − f(x)| < ε.

Exercise 9.37. Show that the set {h_{I}}_{I∈D([0,1])} is not a complete set in L^{2}([0, 1]). What are we missing? Can you complete the set?

9.4. Haar vs Fourier

We give two examples to illustrate how the Haar basis can outperform the Fourier basis when dealing with localized data. See [Kra, Section 7.4, p.285].

The first example is a caricature of the problem: What is the most localized “function” we could consider? The delta distribution. If we could find its Fourier series, we would see that it has a very slowly decreasing tail that extends well beyond the highly localized support of the delta function. However, its Haar transform is very localized; although the Haar transform still has a tail, the tail decays faster than that of the Fourier series.

Consider the following approximation of the delta distribution:

f_{N}(x) = 2^{N}χ_{[0,2^{−N}]}.

Each of these functions has mass 1, and they converge in distribution sense to the delta distribution, that is

lim_{N→∞} T_{f_{N}} (ϕ) = lim_{N→∞} ∫ f_{N}(x)ϕ(x) dx = ϕ(0) = δ(ϕ),

by the Lebesgue Differentiation Theorem for continuous functions, Exercise 9.30.

Exercise 9.38. Compute the Fourier transform of f_{N}. Check that if we view f_{N} as a periodic function on [0, 1), and if its M^{th} partial Fourier sum is

S_{M}(f_{N})(x) = ∑_{|m|≤M} f̂(m)e^{2πimx},

then

∥f_{N} − S_{M}(f_{N})∥_{L^{2}([0,1))} ∼ M^{−1/2}.

Exercise 9.39. Compute the Haar coefficients of f_{N}. Check that the partial Haar sum

P_{M}(f_{N}) = ∑_{j<M} Q_{j}(f_{N}) = ∑_{|j|<M} Q_{j}(f_{N}) + P_{−M+1}(f_{N})

has an exponential decay rate:

∥f_{N} − P_{M}(f_{N})∥_{2} = 2^{−M/2}.

Hint: we already saw this phenomena in Exercise ??, for N = 1.

The exponential decay rate exhibited in Exercise 9.39 is much better than the square root decay exhibited in Exercise 9.38. Suppose we wanted an approximation of F_{N} with an L^{2}-error of magnitude less than 10^{−5}. In the Fourier case it will mean that we need a partial Fourier sum of order M ∼ 10^{10}. In the Haar case. It suffices to consider M = 10 log_{2} 10 < 40.

The localization properties of wavelets allow us to worry only about the wavelets that are supported near where the action occurs in the function. For example, if the function is constant on an interval, then the Haar coefficients of those wavelets supported in the interval vanish, because wavelets share the zero integral property of Haar functions, namely ∫ ψ = 0. This is not the case with the trigonometric functions, whose support spreads over the whole real line.

Exercise 9.40. (Truncated Cosine) Consider the function f defined on the real line by

f(x) = cos(πx) χ_{[0,1]}(x).

Compute the Fourier transform of f and then reconstruct f, with the aid of a computer if necessary. With the aid of the computer, compute the Haar decomposition of f, and compare with the Fourier approximation.

In Exercise 9.40, if we had considered the function cos(πx)χ_{[−1,1]}(x) over a full period of the cosine function, then the Fourier series would have been the function itself, and no doubt that would have been the right way of doing the analysis. However, by chopping off the function we force other Fourier coefficients to come into play.

9.5. Unconditional bases and the martingale transform

The Haar basis happens to be a so-called unconditional basis for L^{p}(ℝ), 1 < p < ∞. Roughly speaking, this means that we can approximate a function in the L^{p}-norm with finite linear combinations of Haar functions (basis). Furthermore, the coefficients corresponding to h_{I} must be ⟨f, h_{I}⟩, and we can recover the L^{p}- norm of the function from knowledge about the absolute value of those coefficients, that is, using some formula involving only |⟨f, h_{I}⟩|. No information about the sign or argument of ⟨f, h_{I}⟩ is necessary. In particular, if f ∈ L^{p} and

f = ∑_{I∈D}⟨f, h_{I}⟩h_{I},

then the new functions defined by

(9.17)  T_{σ}f := ∑_{I∈D} σ_{I}⟨f, h_{I}⟩h_{I},

where σ_{I} = ±1 (or more generally |σ_{I}| = 1), are also in L^{p} and their norms are comparable to that of f. There exist constants c, C > 0 such that for all choices σ of signs,

(9.18)  c∥f∥_{p} ≤ ∥T_{σ}f∥_{p} ≤ C∥f∥_{p}.

For a given sequence σ = {σ_{I}}, the operator T_{σ} in equation (9.17) is called the martingale transform. It is an example of a constant Haar multiplier.

Let us illustrate the above paragraphs in the case p = 2. We already know that the Haar functions provide an orthonormal basis in L^{2}(ℝ). In particular, the Plancherel formula holds:

∥f∥_{2}^{2} = ∑_{I∈D} |⟨f, h_{I}⟩|^{2}.

To compute the L^{2}-norm of the function, we must add the squares of the absolute values of the Haar coefficients. Since each |σ_{I}|^{2} = 1, we have

∥T_{σ}f∥_{2} = ∥f∥_{2},

and so inequality (9.18) holds with c = C = 1. Therefore the martingale transform is an isometry in L^{2}(ℝ).

We need to introduce a new operator, the dyadic square function S^{d}, defined by

(9.19)  S^{d}(f)(x) = (∑_{I∈D} \frac{|⟨f, h_{I}⟩|^{2}}{|I|} χ_{I}(x))^{1/2} .

It turns out that in the case of the Haar functions, one has the following norm equivalence, that can be considered as an L^{p} substitute of Plancherel for Haar functions.

(9.20)  c_{p}∥f∥_{p} ≤ ∥S^{d}(f)∥_{p} ≤ C_{p}∥f∥_{p}.

Exercise 9.41. Verify that ∥S^{d}(f)∥_{2} = ∥f∥_{2}.

Exercise 9.42. Verify that

S^{d}(f)(x) = (∑_{j∈ℤ} |Q_{j}f(x)|^{2})^{1/2} .

Where Q_{j} is the difference operator defined in (9.10) (Lemma 9.25 will be useful).

The definition of the dyadic square function ( 9.19) and inequality (9.20) imply inequality (9.18), because

S^{d}(f) = S^{d}(T_{σ}f).

We can estimate the L^{p}-norm of f from the knowledge of the absolute values of its Haar coefficients alone.

In the case of general wavelets we also have some averaging and difference operators, P_{j} and Q_{j}, and a corresponding square function. The same norm equivalence (9.20) holds in L^{p}(ℝ). As it turns out, wavelet bases will provide unconditional bases for a whole zoo of function spaces (Sobolev, Hölder, etc). There are many references, in particular [Dau92, Chapter 9].

The trigonometric system is an orthonormal basis in L^{2}([0, 1]), however it does not provide an unconditional basis in L^{p}([0, 1]) for p ≠ 2. This reflects a lack of “orthogonality” of the trigonometric system in L^{p}([0, 1]). There is a square function that plays the same role as in wavelets, but it involves more than just the absolute value of the Fourier coefficients,

Sf(x) = (∑_{j} |Δ_{j}f(x)|^{2})^{1/2} ,

where

Δ_{j}f(x) = ∑_{2^{j}≤|n|<2^{j+1}} f̂(j)e^{2πinx}.

It is true that ∥f∥_{p} is comparable to ∥S(f)∥_{p} in the sense of inequality (9.20). We are allowed to change the signs of the Fourier coefficients of f on the dyadic blocks of frequency. If we denote by T_{δ}f the function reconstructed with the modified coefficients, that is,

T_{δ}f(x) := ∑_{j≥0} δ_{j}Δ_{j}f(x), δ_{j} = ±1,

then S(T_{δ}f) = S(f), and so their L^{p} norms are the same and are both equivalent to ∥f∥_{p}. But there is no guarantee that the same will be true if we change some but not all signs inside a given dyadic block! In that case, S(f) does not have to coincide with S(T_{δ}f). In fact, the innocent-seeming operation of changing signs inside the dyadic blocks can cause us to “jump out” of L^{p}([0, 1]).

The study of square functions is known as Littlewood–Paley Theory; it is a widely used tool in harmonic analysis. For an introduction to Dyadic Harmonic Analysis see the lecture notes by the first author [Per].