Harmonic Analysis: from Fourier to Haar

María Cristina Pereyra

Lesley A. Ward

Department of Mathematics and Statistics, MSC03 2150, 1 University of New Mexico, Albuquerque, NM 87131-0001, USA

E-mail address: crisp@math.unm.edu

School of Mathematics and Statistics, University of South Australia, Mawson Lakes SA 5095, Australia

E-mail address: Lesley.Ward@unisa.edu.au

2000 Mathematics Subject Classification. Primary 42-xx

Contents

List of Figures ix

List of Tables xiii

Introduction xv

Chapter 1. Fourier series: some motivation 1 1.1. Some examples and key definitions 1 1.2. Main questions 5 1.3. Fourier series and Fourier coefficients 7 1.4. A little history, and motivation from the physical world 11

Chapter 2. Interlude 17 2.1. Nested classes of functions on bounded intervals 17 2.2. Modes of convergence 28 2.3. Interchanging limit operations 34 2.4. Density 38

Chapter 3. Pointwise convergence of Fourier series 41 3.1. Pointwise convergence: why do we care? 41 3.2. Smoothness vs convergence 44 3.3. A suite of convergence theorems 50

Chapter 4. Summability methods: Dirichlet kernels and Fejér kernels 55 4.1. Partial Fourier sums and the Dirichlet kernel D_{N} 55 4.2. Properties of the Dirichlet kernel 57 4.3. Convolution 59 4.4. Good kernels, or approximations of the identity 67 4.5. Fejér kernels and Cesàro means 72 4.6. Poisson kernels and Abel means 75 4.7. Excursion into L^{p}(𝕋) 76

Chapter 5. Mean-square convergence of Fourier series 79 5.1. Basic Fourier theorems in L^{2}(𝕋) 79 5.2. Geometry of the Hilbert space L^{2}(𝕋) 81 5.3. Completeness of the trigonometric system 85 5.4. Equivalent conditions for completeness in L^{2}(𝕋) 90

Chapter 6. A whirlwind tour of discrete Fourier and Haar analysis 95 6.1. Fourier Series: a summary 95 6.2. The discrete Fourier basis 96 6.3. A parenthesis on bases and dual bases in ℂ^{N} 100 6.4. The discrete Fourier transform and its inverse 101 6.5. The Fast Fourier Transform (FFT) 102 6.6. The discrete Haar basis 107 6.7. The Discrete Haar Transform 110 6.8. The Fast Haar Transform 111

Chapter 7. The Fourier transform in paradise 115 7.1. From Fourier series to Fourier integrals 116 7.2. The Fourier transform on the Schwartz class S(ℝ) 117 7.3. The time–frequency dictionary for S(ℝ) 119 7.4. The Schwartz class is closed under the Fourier transform 122 7.5. Convolution and approximations of the identity on ℝ 125 7.6. An interlude on kernels 128 7.7. The Fourier Inversion Formula, and Plancherel’s Identity 130 7.8. L^{p}-norms on S(ℝ) 133

Chapter 8. Beyond paradise 137 8.1. Functions of moderate decrease 137 8.2. Tempered distributions 140 8.3. The time–frequency dictionary for S′(ℝ) 143 8.4. The delta distribution 147 8.5. Some applications of the Fourier transform 149 8.6. L^{p}(ℝ) as distributions 155

Chapter 9. From Fourier to Haar 161 9.1. The windowed Fourier transform, and Gabor bases 161 9.2. The wavelet transform 165 9.3. Haar analysis 167 9.4. Haar vs Fourier 181 9.5. Unconditional bases, martingale transforms, and square functions 183

Chapter 10. Zooming properties of wavelets, and applications 185 10.1. Multiresolution analyses (MRAs) 185 10.2. The Haar multiresolution analysis 187 10.3. From MRA to wavelets: Mallat’s Theorem 192 10.4. Mallat’s algorithm revisited 197 10.5. Cascade algorithm and filter banks 200 10.6. Design features 204 10.7. A catalog of wavelets 206 10.8. Wavelet packets 207 10.9. Two-dimensional wavelets 209 10.10. Basics of compression and denoising 210

Chapter 11. The Hilbert transform 213 11.1. The Hilbert transform on the frequency domain: Fourier multipliers 213 11.2. The Hilbert transform on the time domain: singular integrals 215 11.3. Boundedness properties of the Hilbert transform 217 11.4. The function space weak-L^{1}(ℝ) 220 11.5. Interpolation theorems 228 11.6. A festival of inequalities 233 11.7. Some history to conclude our journey 239 11.8. Disclaimer 244

Chapter 12. Projects for students 245 12.1. Fourier analysis on finite groups 246 12.2. Two extremal problems on curves 249 12.3. Weyl’s equidistribution theorem 249 12.4. The Gibbs phenomenon for Fourier series and for wavelets 250 12.5. Linear and nonlinear approximation in Fourier and Haar bases 252 12.6. Monsters 253 12.7. The power of averaging or summability methods 255 12.8. Local cosine and sine bases 257 12.9. The twin dragon and fractals in wavelets 257 12.10. Fourier analysis and wavelets on graphs 258 12.11. The discrete and finite Hilbert transforms and Cotlar’s Lemma 259 12.12. Edge Detection 260 12.13. The Hilbert transform of characteristic functions 261 12.14. The Calderón–Zygmund decomposition and weak-L^{1} estimates 262 12.15. BMO, John-Nirenberg and Stopping time 266 12.16. The Hilbert transform as an average of Haar multipliers 268

Appendix A. Some useful facts from analysis 271

Appendix B. Vector spaces, norms, inner products 273 B.1. Vector spaces 273 B.2. Normed spaces 274 B.3. Inner-product vector spaces 274

Bibliography 281

Name Index 285

Subject Index 287

CHAPTER 1 Fourier series: some motivation

In this book we discuss three types of Fourier analysis: first, Fourier series, in which the input is a periodic function on ℝ, and the output is a two-sided series where the summation is over n ∈ ℤ; second, finite Fourier analysis, where the input is a vector of length N with complex entries, and the output is another vector in ℂ^{N}; and third, the Fourier transform, where the input is a function on ℝ, and the output is another function on ℝ.

As an aside, we note that one can also do Fourier analysis on ℝ^{n}, on abelian groups, on graphs, and in still more abstract settings. See [SS1, Chapter 7]. Fourier analysis is also connected with representation theory. Some of the projects in Chapter 12 are related to these abstract settings.

1.1. Some examples and key definitions

The central idea of Fourier analysis is to break a function into a combination of simpler functions. We think of these simpler functions as building blocks. We will also be interested in reconstructing the original function from the building blocks. Here is a colorful analogy: a prism or raindrop can break a ray of (white) light into all colors of the rainbow. The analogy is quite apt: the different colors correspond to different wavelengths/frequencies of light. We will see that our simpler functions can also correspond to pure frequencies. For example, we will shortly consider sine and cosine functions of various frequencies as our first example of building blocks. When played aloud, a given sine or cosine function produces a pure tone, or note, or harmonic, at a single frequency. The term harmonic analysis evokes this idea of separation of a sound, or in our terms a function, into pure tones.

Expressing a function as a combination of building blocks is also called decomposing the function. In Chapters 9 and 10 we will study so-called time–frequency decompositions, in which each building block encodes information about time as well as about frequency, very much like musical notation does.

We begin with an example.

Example 1.1. (Toy Model of Voice Signal) Suppose Amanda is in Baltimore, and she calls her mother in Vancouver, saying ‘Hi Mom, it’s Amanda, and I can’t wait to tell you what happened today’. What happens to the sound? As Amanda speaks, she creates waves of pressure in the air, which travel toward the phone receiver. The sound has duration, say about five seconds in this example, and intensity or loudness, which varies over time. It also has many other qualities that make it sound like speech rather than music, say. The sound of Amanda’s voice becomes a signal, which travels along the phone wire or via satellite, and at the other end is converted back into a recognizable voice.

Let us try our idea of breaking Amanda’s voice signal into simpler building blocks. Suppose the signal looks like the function f(t) plotted in Figure 1.1. (In reality, her voice signal would be much more complicated.)

Figure 1.1. Toy voice signal f(t).

In Figure 1.1 the horizontal axis represents time t in seconds, and the vertical axis represents the intensity of the sound, so that when y = f(t) is near zero the sound is soft, and when y = f(t) is large (positive or negative) the sound is loud. In this particular signal, there are two sorts of wiggling going on; see Figure 1.2.

We recognize the large, slow wiggle as a multiple of sin t. The smaller wiggle is oscillating much faster; counting the oscillations, we recognize it as a multiple of sin(16t). Here sin t and sin(16t) are our first examples of building blocks. They are functions of frequency 1 and 16 respectively, meaning that they complete respectively 1 and 16 full oscillations as t runs through 2π units of time.

Next we need to know how much of each building block is present in our signal. The maximum amplitudes of the large and small wiggles are 3 and 0.5 respectively, giving us the terms 3 sin t and \frac{1}{2} sin 16t. We add these together to build the original signal:

f(t) = 3 sin t + \frac{1}{2} sin(16t).

We have written our signal f(t) as a sum of constant multiples of the two building blocks. The right-hand side is our first example of a Fourier decomposition; it is the Fourier decomposition of our signal f(t).

Let us get back to the phone company. When Amanda calls her mother, the signal goes from her phone to Vancouver. How will the signal be encoded? For instance, the phone company could take a collection of say 100 equally-spaced times, record the strength of the signal at each time, and send the resulting 200 numbers to Vancouver, where they will know how to put them back together. (By the way,

Figure 1.2. Building blocks of toy voice signal f(t).

can you find a more efficient way of encoding the timing information?) But for our signal, we can achieve the same result with just four numbers. We simply send the frequencies 1 and 16, and the corresponding strengths or amplitudes 3 and 1/2. Once you know the code, that the numbers represent sine wave frequencies and amplitudes, these four numbers are all you need to know to rebuild our signal exactly.

Note that for a typical, more complicated signal, one would need more than just two sine functions as building blocks. Some signals would require infinitely many sine functions, and some would require cosine functions instead or as well. The collection

{sin(nt) : n ∈ ℕ} ∪ {cos nt : n ∈ ℕ ∪ {0}}

of all the sine and cosine functions whose frequencies are positive integers, together with the constant function with value 1, is an example of a basis. We will return to the idea of a basis later; informally, it means a given collection of building blocks that is able to express every function in some given class. Fourier series use sines and cosines; other types of functions can be used as building blocks, notably in the wavelet series we will see later (Chapters 9 and 10).

Now let us be more ambitious. Are we willing to sacrifice a bit of the quality of Amanda’s signal in order to send it more cheaply? Maybe. For instance, in our signal the strongest component is the big wiggle. What if we send only the single frequency 1 and its corresponding amplitude 3? In Vancouver, only the big wiggle will be reconstructed, and the small fast wiggle will be lost from our signal. But Amanda’s mother knows the sound of Amanda’s voice, and the imperfect reconstructed signal may still be recognizable. This is our first example of compression of a signal.

Figure 1.3. Plot of an actual voice saying ‘Hi Mom, it’s Amanda, and I can’t wait to tell you what happened today’. Explain the units: time in seconds along the horizontal axis, a proxy for volume/intensity along the vertical axis. Check how long it lasts and correct the estimate (currently ‘five seconds’) in the first paragraph of this example.

To sum up: We analyzed our signal f(t), determining which building blocks were present and with what strength. We compressed the signal, discarding some of the frequencies and their amplitudes. We transmitted the remaining frequencies and amplitudes. (At this stage we could also have stored the signal.) At the other end they reconstructed the compressed signal.

Again, in practice one wants the reconstructed signal to be similar to the original signal. There are many interesting questions about which building blocks, and how many, can be thrown away while retaining a recognizable signal. This is part of the subject of signal processing, in electrical engineering.

In the interests of realism, Figure 1.3 shows a plot of an actual voice saying ‘Hi Mom, it’s Amanda, and I can’t wait to tell you what happened today’. ♢

In another analogy, we can think of the decomposition as a recipe. The building blocks, distinguished by their frequencies, correspond to the different ingredients in your recipe. You also need to know how much sugar, flour, and so on you have to add to the mix to get your cake f; that information is encoded in the amplitudes or coefficients.

The function f in Example 1.1 is especially simple. The intuition that sines and cosines were sufficient to describe many different functions was gained from the experience of the pioneers of Fourier analysis with physical problems (for instance heat diffusion and the vibrating string) in the early eighteenth century; see Section 1.4. This intuition leads to the idea of expressing a periodic function f(θ) as an infinite linear combination of sines and cosines, also known as a trigonometric series:

(1.1)  f(θ) ∼ ∑_{n=0}^{∞} [b_{n} sin(nθ) + c_{n} cos(nθ)] .

We use the symbol ∼ to indicate that the right-hand side is the trigonometric series associated with f. We don’t use the symbol = here since in some cases the left- and right-hand sides of (1.1) are not equal, as discussed below.

We can rewrite the right-hand side of (1.1) as a linear combination of exponential functions. To do so, we use Euler’s Formula^{1}: e^{iθ} = cos θ + i sin θ, and the corresponding formulas for sine and cosine in terms of exponentials, applied to nθ for each n,

1This formula is named after the Swiss mathematician Leonhard Euler (pronounced “oiler”), (1707–1783). We assume the reader is familiar with basic complex number operations and

cos θ = \frac{e^{iθ} + e^{−iθ}}{2} , sin θ = \frac{e^{iθ} − e^{−iθ}}{2i} .

We will use the following version throughout:

(1.2)  f(θ) ∼ ∑_{n=−∞}^{∞} a_{n}e^{inθ}.

The right-hand side is called the Fourier series of f. We also say that f can be expanded in a Fourier series. The coefficient a_{n} is called the n^{th} Fourier coefficient of f, and is often denoted by f̂(n) to emphasize the dependence on the function f. The coefficients {a_{n}} are determined by the coefficients {b_{n}} and {c_{n}} in the sine/cosine expansion.

Exercise 1.2. Write a_{n} in terms of b_{n} and c_{n}. ♢

1.2. Main questions

Our brief sketch immediately suggests several questions.

• How can we find the Fourier coefficients a_{n} from the corresponding function f?

• Given the {a_{n}}, how can we reconstruct f?

• In what sense does the Fourier series converge? Pointwise? Uniformly? Or in some other sense? If it does converge in some sense, how fast does it converge?

• When the Fourier series does converge, is its limit equal to the original function f?

• Which functions can we express with a trigonometric series

∑_{n=−∞}^{∞} a_{n}e^{inθ}?

For example, must f be continuous, or Riemann integrable?

• What other building blocks could we use besides sines and cosines, or

equivalently exponentials? We develop answers to these questions in the following pages, but before do-

ing that let us compare to the more familiar problem of power series expansions. An infinitely-often differentiable function f can be expanded into a Taylor series^{2} centered say at x = 0,

(1.3) Taylor series and coefficients:  ∑_{n=0}^{∞} c_{n} x^{n},  where c_{n} = \frac{f^{(n)}(0)}{n!}.

notation. For example if v = a + ib, w = c + id, then v + w = (a + c) + i(b + d) and vw = (ac − bd) + i(bc + ad). The algebra is done as it would be for real numbers, with the extra fact that the imaginary unit i has the property that i^{2} = −1. The absolute value of a complex number v = a + ib is defined to be |v| = \sqrt{a^{2} + b^{2}}. See [Tao2, Section 15.6] for a quick review of complex numbers.

2Named after the English mathematician Brook Taylor (1685–1731). Sometimes the Taylor series centered at x = 0 is called the Maclaurin series, named after the Scottish mathematician Colin Maclaurin (1698–1746).

The Taylor series always converges to f(0) = c_{0} when evaluated at x = 0. In the 18^{th} century mathematicians discovered that the traditional functions of calculus (sin x, cos x, ln(1 + x), \sqrt{1 + x}, e^{x}, etc.) can be expanded in Taylor series, and that for these examples the Taylor series converges to the function in an open interval containing x = 0, and they were very good at manipulating power series and calculating with them. That led them to believe that the same would be true for all functions, which at the time meant infinitely-often differentiable functions. That dream was shattered by Cauchy’s^{3} discovery in 1821 of a counterexample.

Example 1.3. (Cauchy’s Counterexample) The function

f(x) = \begin{cases}e^{−1/x^{2}}, & if x ≠ 0; \\ 0, & if x = 0 \end{cases}

is infinitely often differentiable at every x, and f^{(n)}(0) = 0 for all n ≥ 0. Thus its Taylor series is identically equal to zero. Therefore the Taylor series will converge to f(x) only for x = 0. ♢

Exercise 1.4. Verify that Cauchy’s function is infinitely differentiable. Concentrate on what happens at x = 0. ♢

The Taylor polynomial of order N of a function f that can be differentiated at least N times is given by the formula

(1.4)  P_{N}(f, 0) = f(0) + f′(0)x + \frac{f′′(0)}{2!}x^{2} + ⋯ + \frac{f^{(N)}(0)}{N!}x^{N}.

Exercise 1.5. Verify that if f is a polynomial of order less than or equal to N, then f coincides with its Taylor polynomial of order N. ♢

Definition 1.6. A trigonometric^{4} polynomial of degree M is a function of the form

f(θ) = ∑_{n=−M}^{M} a_{n}e^{inθ},

where a_{n} ∈ ℂ for n = −M, …, M.

Exercise 1.7. Verify that if f is a trigonometric polynomial then its coefficients {a_{n}} can be calculated with the following formula:

a_{n} = \frac{1}{2π} ∫_{−π}^{π} f(θ)e^{−inθ} dθ.

♢

In both the Fourier and Taylor series, the problem is how well and in what sense we can approximate a given function by using its Taylor polynomials (very specific polynomials) or by using its partial Fourier sums (very specific trigonometric polynomials).

3The French mathematician Augustin Louis Cauchy (1789–1857).

4To be more precise, this particular trigonometric polynomial is 2π-periodic; see Section 1.3.1.

1.3. Fourier series and Fourier coefficients

We begin to answer our questions. The Fourier coefficients f̂(n) = a_{n} are calculated using the formula suggested by Exercise 1.7:

(1.5)  f̂(n) = a_{n} := \frac{1}{2π} ∫_{−π}^{π} f(θ)e^{−inθ} dθ.

Aside 1.8. The notation := indicates that we are defining the term on the left of the :=, in this case a_{n}. Occasionally we may need to use =: instead, when we are defining a quantity on the right, as in Theorem 3.6 below.

One way to explain the appearance of formula (1.5) is to assume that the function f is equal to a trigonometric series,

f(θ) = ∑_{n=−∞}^{∞} a_{n}e^{inθ},

and then to proceed formally^{5}, operating on both sides of the equation. Multiply both sides by an exponential function, and then integrate, taking the liberty of interchanging the sum and the integral:

\frac{1}{2π} ∫_{−π}^{π} f(θ)e^{−ikθ} dθ = \frac{1}{2π} ∫_{−π}^{π} ∑_{n=−∞}^{∞} a_{n}e^{inθ}e^{−ikθ} dθ

= ∑_{n=−∞}^{∞} a_{n} \frac{1}{2π} ∫_{−π}^{π} e^{inθ}e^{−ikθ} dθ = a_{k}.

The last equality holds because

(1.6)  \frac{1}{2π} ∫_{−π}^{π} e^{inθ}e^{−ikθ} dθ = \frac{1}{2π} ∫_{−π}^{π} e^{i(n−k)θ} dθ = δ_{n,k},

where the Kronecker^{6} delta δ_{n,k} is defined by

(1.7)  δ_{n,k} = \begin{cases}1, & if k = n; \\ 0, & if k ≠ n. \end{cases}

We explore the geometric meaning of equation (1.6) in Chapter 5. In language we will meet there, the equation says that the exponential functions {e^{inx}}_{n∈ℕ} form an orthonormal set.

Aside 1.9. The usual rules of calculus apply to complex-valued functions. For example, if u and v are the real and imaginary parts of the function f : [a, b] → ℂ, that is f = u + iv where u and v are real-valued, then f′ := u′ + iv′. Likewise, for integration, ∫ f = ∫ u + i ∫ v. Here we are assuming that the functions u and v are differentiable in the first case, and integrable in the second.

5Here the term formally means that we work through a computation ‘following our noses’, without stopping to justify every step. In this example we don’t worry about whether the integrals or series converge, or whether it is valid to exchange the order of the sum and the integral.

A rigorous justification of our computation could start from the observation that it is valid to exchange the sum and the integral if the Fourier series converges uniformly to f (see Chapters 2 and 4 for some definitions). Formal computations are often extremely helpful in building intuition.

6Named after Leopold Kronecker, German mathematician and logician (1823–1891).

Notice that a necessary condition on f for the integral in formula (1.5) for the coefficients to exist is that the complex-valued function f(θ)e^{−inθ} should be an integrable^{7} function on [−π, π). In fact, if |f| is integrable, then so is f(θ)e^{−inθ} and

|a_{n}| = |\frac{1}{2π} ∫_{−π}^{π} f(θ)e^{−inθ} dθ|
≤ \frac{1}{2π} ∫_{−π}^{π} |f(θ)e^{−inθ}| dθ  (since |∫ g| ≤ ∫ |g|)
= \frac{1}{2π} ∫_{−π}^{π} |f(θ)| dθ  (since |e^{−inθ}| = 1)
< ∞  (since f is integrable).

Aside 1.10. It can be verified that the absolute value of the integral of an integrable complex-valued function is less than or equal to the integral of the absolute value of the function, the so-called triangle inequality for integrals, not to be confused with the triangle inequality for complex numbers: |a + b| ≤ |a| + |b|, or the triangle inequality for integrable functions: ∫ |f + g| ≤ ∫ |f| + ∫ |g|. They are all animals in the same family.

We now explore in what sense the Fourier series (1.2) approximates the original function f. We begin with some examples.

Exercise 1.11. Find the Fourier coefficients and the Fourier series for the trigonometric polynomial

f(θ) = 3e^{−2iθ} − e^{−iθ} + 1 + e^{iθ} − πe^{4iθ} + \frac{1}{2} e^{7iθ}.

♢

Example 1.12. (Periodic Ramp Function) Compute the Fourier coefficients and the Fourier series for the function

f(θ) = θ, for −π ≤ θ < π.

The n^{th} Fourier coefficient is given by

f̂(n) = a_{n} := \frac{1}{2π} ∫_{−π}^{π} f(θ)e^{−inθ} dθ

= \frac{1}{2π} ∫_{−π}^{π} θe^{−inθ} dθ

= \begin{cases}(−1)^{n+1}/(in), & if n ≠ 0; \\ 0, & if n = 0. \end{cases}

7A function g(θ) is said to be integrable on [−π, π) if ∫_{−π}^{π} g(θ) dθ is well-defined; in particular the value of the integral is a finite number. In harmonic analysis the notion of integral used is the Lebesgue integral. We expect the reader to be familiar with Riemann-integrable functions meaning that the function g is bounded and the integral exists in the sense of Riemann. Riemann-integrable functions are Lebesgue integrable. In Chapter 2 we will briefly review the Riemann integral. See [Tao1, Chapters 11], [Tao2, Chapters 19].

These integration methods are named after the French mathematician Henri Lebesgue (1875– 1941), and the German mathematician Georg Friedrich Bernhard Riemann (1826–1866).

using integration by parts in the last line. Thus the Fourier series of f(θ) = θ is given by

f(θ) ∼ ∑_{\{n∈ℤ:n≠0\}} \frac{(−1)^{n+1}}{in} e^{inθ} = 2 ∑_{n=1}^{∞} \frac{(−1)^{n+1}}{n} sin (nθ).

Note that both f and its Fourier series are odd functions. It turns out that the function and the series coincide for all θ ∈ (−π, π). At the endpoints θ = ±π the series converges to zero (the midpoint of the jump), since sin (nπ) = 0 for all n ∈ ℕ. Notice that in this example the function and the series do not coincide at the endpoints where the jump discontinuity occurs. ♢

Exercise 1.13. Fill in the details in the integration-by-parts argument in the calculation in Example 1.12. ♢

1.3.1. 2π-Periodic functions. Note that we are considering functions

f : [−π, π) → ℂ

that are complex-valued, and defined on a bounded half-open interval in the real line. We extend the function periodically to the whole of ℝ. For example, Figure 1.4 shows part of the graph of the periodic extension to ℝ of the real-valued function f defined on [−π, π) in Example 1.12.

Figure 1.4. Periodized ramp function on [−3π, 3π].

Exercise 1.14. Run the following Matlab script to reproduce the plot in Figure 1.4.

  clear all
  plot([-3*pi,-pi],[-pi,pi],'k',[-pi,-pi],[-pi,pi],'k',...
    [-pi,pi],[-pi,pi],'k',[pi,pi],[-pi,pi],'k',[pi,3*pi],[-pi,pi],'k')

  axis([-10,10,-8,8])
  xlabel('\theta')
  ylabel('Periodized ramp function f(\theta)')
  legend('f(\theta)')}

♢

Exercise 1.15. Modify the Matlab script from Exercise 1.14 to plot over [−3π, 3π] the periodic extension to ℝ of the function defined by f(x) = x^{2} for x ∈ [−π, π). ♢

In mathematical language, a function defined on ℝ is 2π-periodic if f(x+2π) = f(x) for all x ∈ ℝ. The building blocks we are using are 2π-periodic functions: sin(nθ), cos(nθ), e^{−inθ}. Finite linear combinations of 2π-periodic functions are 2π-periodic.

Let 𝕋 denote the unit circle:

(1.8)  𝕋 := {z ∈ ℂ : |z| = 1} = {z = e^{iθ} : −π ≤ θ < π}.

We can identify a 2π-periodic function f on ℝ with a function g : 𝕋 → ℂ defined on the unit circle 𝕋 as follows. Given z ∈ 𝕋 there is a unique θ ∈ [−π, π) such that z = e^{iθ}. Define

g(z) := f(θ).

Note that if g is to be continuous on the unit circle 𝕋, we will need f(−π) = f(π). When we say f : 𝕋 → ℂ and f ∈ C^{k}(𝕋), we mean that the function f is differentiable k times, and that f, f′, …, f^{(k)} are all continuous functions on 𝕋. In words, f is a k times continuously differentiable function from the unit circle into the complex plane. In particular f^{(ℓ)}(−π) = f^{(ℓ)}(π) for all 0 ≤ ℓ ≤ k.

Note that if f is a 2π-periodic and integrable function, then the integral of f over each interval of length 2π takes the same value. For example,

∫_{−π}^{π} f(θ) dθ = ∫_{0}^{2π} f(θ) dθ = ∫ \frac{5π}{2} ; \frac{π}{2} f(θ) dθ = · · · .

Exercise 1.16. Verify that if f is 2π-periodic and integrable, then for all a ∈ ℝ

∫_{a}^{a+2π} f(θ) dθ = ∫_{−π}^{π} f(θ) dθ.

♢

In general, if f is a 2π-periodic trigonometric polynomial of degree M, in other words a function of the form

f(θ) = ∑_{k=−M}^{M} a_{k}e^{ikθ},

then its Fourier series coincides with f itself. See Exercises 1.7 and 1.18.

1.3.2. 2π- and L-periodic Fourier series and coefficients. So far, we have mostly considered functions defined on the interval [−π, π) and their 2π-periodic extensions. Later we will also use functions defined on the unit interval [0, 1), or a symmetric version of it [−1/2, 1/2), and their 1-periodic extensions. More generally, we can consider functions defined on a general interval [a, b) of length L = b − a, and extended periodically to ℝ with period L.

An L-periodic function f defined on ℝ is a function such that f(x) = f(x + L) for all x ∈ ℝ. We can define the Fourier coefficients and the Fourier series for an L-periodic function f. The formulae come out slightly differently to account for the rescaling:

L-Fourier coefficients  f̂ ^{L}(n) = \frac{1}{L} ∫_{a}^{b} f(θ)e^{−2πinθ/L}dθ,

L-Fourier series  ∑_{n=−∞}^{∞} f̂ ^{L}(n)e^{2πinθ/L}.

In the earlier case [a, b) = [−π, π), we had 2π/L = 1. Note that the building blocks are now the L-periodic exponential functions e^{2πinθ/L}, while the L-periodic trigonometric polynomials are finite linear combinations of these building blocks.

Exercise 1.17. Verify that for each n ∈ ℤ, the function e^{2πinθ/L} is L-periodic. ♢

In Chapters 7, 9, and 11, we will consider 1-periodic functions. Their Fourier coefficients and series will read

f̂(n) = ∫_{0}^{1} f(x)e^{−2πinx} dx,  ∑_{n=−∞}^{∞} f̂(n)e^{2πinx}.

Why is it important to consider functions defined on a general interval [a, b)? A key distinction in Fourier analysis is between functions f that are defined on a bounded interval, or equivalently periodic functions on ℝ, and functions f that are defined on ℝ but that are not periodic. For non-periodic functions on ℝ, it turns out that the natural “Fourier quantity” is not a Fourier series but another function, known as the inverse Fourier transform of f. One way to develop and understand the Fourier transform is to consider Fourier series on symmetric intervals [−L/2, L/2) of length L, and then let L → ∞; see Section 7.1.

Exercise 1.18. Let f be an L-periodic trigonometric polynomial of degree N, that is

f(θ) = ∑_{n=−M}^{M} a_{n}e^{2πinθ/L}.

Verify that f coincides with its L-Fourier series. ♢

1.4. A little history, and motivation from the physical world

‘It was this pliability which was embodied in Fourier’s intuition, commonly but falsely called a theorem, according to which the trigonometric series “can express any function whatever between definite values of the variable.” This familiar statement of Fourier’s “theorem,” taken from Thompson and Tait’s “Natural Philosophy,” is much too broad a one, but even with the limitations which must to-day be imposed upon the conclusion, its importance can still be most fittingly described as follows in their own words: The theorem “is not only one of the most beautiful results of modern analysis, but may be said to furnish an indispensable instrument in the treatment of nearly recondite question [sic] in modern physics. To mention only sonorous vibrations, the propagation of electric signals along a telegraph wire, and the conduction of heat by the earth’s crust, as subjects in their generality intractable without it, is to give but a feeble idea of its importance.” ’

Edward B. Van Vleck

Address to the American Association for the Advancement of Science, 1913. Quoted in [Bre, p. 12].

[figure: 0; π; x]

Figure 1.5. Sketch of a one-dimensional rod.

For the last 200 years, Fourier Analysis has been of immense practical importance, both for theoretical mathematics (especially in the subfield now called harmonic analysis, but also in number theory), and in understanding, modeling, predicting, and controlling the behavior of physical systems. Older applications in mathematics, physics and engineering include the way that heat diffuses in a solid object, and the wave motion undergone by a string or a two-dimensional surface such as a drumhead. Recent applications include telecommunications (radio, wireless phones) and more generally data storage and transmission (the JPEG format for images on the Internet).

The mathematicians involved in the discovery, development and applications of trigonometric expansions include, in the 18th, 19th, and early 20th centuries: Brook Taylor (1685–1731), Daniel Bernoulli (1700–1782), Jean Le Rond d’Alembert (1717–1783), Leonhard Euler (1707–1783) (especially for the vibrating string), Joseph Louis Lagrange (1736–1813), Jean Baptiste Joseph Fourier (1768–1830), Peter Gustave Lejeune Dirichlet (1805–1859), and Charles De La Vallée Poussin (1866–1962), among others. See the books by Ivor Grattan-Guinness [Grat, Chapter 1] and by Thomas Körner [Kor] for more on historical development.

We describe briefly some of the physical models that these mathematicians tried to understand, and whose solutions led them to believe that generic functions should be expandable in trigonometric series.

1.4.1. Temperature distribution in a one-dimensional rod. Our task is to find the temperature u(x, t) in a rod of length π, where x ∈ [0, π] represents the position of a point on the rod, and t ∈ [0, ∞) represents time. See Figure 1.5. We assume that the initial temperature (when t = 0) is given by a known function f(x) = u(x, 0), and that at both endpoints of the rod the temperature is held at zero, giving the boundary conditions u(0, t) = u(π, t) = 0. There are many other plausible boundary conditions.

The physical principle governing the diffusion of heat is expressed by the Heat Equation:

(1.9)  \frac{∂u}{∂t} = \frac{∂^{2}u}{∂x^{2}}.

We want u(x, t) to be a solution of the linear partial differential equation (1.9), with initial and boundary conditions

(1.10)  u(x, 0) = f(x),  u(0, t) = u(π, t) = 0.

We refer to equation (1.9) together with the initial conditions and boundary conditions (1.10) as an initial value problem.

As Fourier did in his Théorie Analytique de la Chaleur, we assume (or, more accurately, guess) that there will be a separable solution, meaning a solution that is the product of a function of x and a function of t:

u(x, t) = α(x)β(t).

Then

\frac{∂u}{∂t}(x, t) = α(x)β′(t), \frac{∂^{2}u}{∂x^{2}} = α′′(x)β(t).

By equation (1.9), we conclude that

α(x)β′(t) = α′′(x)β(t),

and so

(1.11)  \frac{β′(t)}{β(t)} = \frac{α′′(x)}{α(x)}.

Because the left side of equation (1.11) depends only on t and the right side depends only on x, both sides of the equation must be equal to a constant. Call the constant k. We can decouple equation (1.11) into a system of two linear ordinary differential equations, whose solutions we know:

\{ \begin{matrix}β′(t) = kβ(t) & ⟶ β(t) = C_{1}e^{kt}, \\ α′′(x) = kα(x) & ⟶ α(x) = C_{2} sin(\sqrt{−k} x) + C_{3} cos(\sqrt{−k} x). \end{matrix}

The boundary conditions tell us which choices to make:

α(0) = 0 ⟶ α(x) = C_{2} sin(\sqrt{−k} x),
α(π) = 0 ⟶ \sqrt{−k} = n ∈ ℕ,  and thus k = −n^{2}, with n ∈ ℕ.

We have found the following separable solutions of the initial value problem. For each n ∈ ℕ, there is a solution of the form

u_{n}(x, t) = Ce^{−n^{2}t} sin(nx).

Aside 1.19. A note on notation: Here we are using a typical convention from analysis, where C denotes a constant that depends on constants from earlier in the argument. In this case C depends on C_{1} and C_{2}. An extra complication in this example is that C_{1}, C_{2} and therefore C may all depend on the natural number n; it would be more illuminating to write

u_{n}(x, t) = C_{n}e^{−n^{2}t} sin(nx).

Finite linear combinations of these solutions will also be solutions of equation (1.9), since ∂/∂t and ∂^{2}/∂x^{2} are linear. We would like to say that infinite linear combinations

u(x, t) = ∑_{n=1}^{∞} b_{n}e^{−n^{2}t} sin(nx)

are also solutions.

To satisfy the initial condition u(x, 0) = f(x) at time t = 0, we would have

f(x) = u(x, 0) = ∑_{n=1}^{∞} b_{n} sin(nx).

So as long as we can write the initial temperature distribution f as a superposition of sine functions, we will have a solution to the equation. Here is where Fourier suggested that all functions have expansions into sine and cosine series. However, it was a long time before this statement was fully explored. See Section 3.3 for some highlights in the history of this quest.

1.4.2. Other physical models. In Table 1.1 we present schematically several physical models. In each case, one can do an heuristic analysis similar to the one performed in Section 1.4.1 for the heat equation in a one-dimensional rod.

Exercise 1.20. Find solutions to the physical models presented in Table 1.1 by an analysis similar to the one used in Section 1.4.1. ♢

The initial or boundary conditions in the table specify the value of u at time t = 0, that is u(x, 0) = f(x).

1.4. A LITTLE HISTORY, AND MOTIVATION FROM THE PHYSICAL WORLD 15

Table 1.1. Physical models.

Model Initial or Boundary Condition Solution

Vibrating string

f(x) = ∑_{n} b_{n} sin(nx)  u(x, t) = ∑_{n} b_{n}e^{int} sin(nx)

u_{tt} = u_{xx}

u(0, t) = u(π, t) = 0 Temperature in a rod

f(x) = ∑_{n} c_{n} cos(nx)  u(x, t) = ∑_{n} c_{n}e^{−n^{2}t} cos(nx)

u_{t} = u_{xx}

u_{x}(0, t) = u_{x}(π, t) Steady-state temp.

in a disk

f(θ) = ∑_{n} a_{n}e^{inθ}  u(r, θ) = ∑_{n} a_{n}r^{|n|}e^{inθ}

u_{rr} + r^{−1}u_{r} + r^{−2}u_{θθ} = 0

Steady-state temp.

in a rectangle

f_{0}(x) = ∑_{k} A_{k} sin(kx)  u(x, y) = ∑_{k} (\frac{sinh[k(1 − y)]}{sinh k}A_{k}
f_{1}(x) = ∑_{k} B_{k} sin(kx)  +\frac{sinh(ky)}{sinh k}B_{k}) sin(kx)

u_{xx} + u_{yy} = 0

Bibliography

[Abb] S. Abbott, Understanding analysis, Undergraduate Texts in Mathematics, Springer, 2001.

[Ari] J. Arias de Reyna, Pointwise convergence of Fourier series, Lecture Notes in Math., vol. 1785, Springer–Verlag, Berlin, 2002.

[Bart1] R. G. Bartle, The elements of integration and Lebesgue measure, John Wiley & Sons, Inc., 1966.

[Bart2] R. G. Bartle, A modern theory of integration, Graduate Studies in Mathematics, vol. 32, Amer. Math. Soc., Providence, RI, 2001.

[Bea] J. C. Bean, Engaging ideas: The professor’s guide to integrating writing, critical thinking, and active learning in the classroom, Jossey–Bass, San Francisco, CA, 2001.

[Beck] Inequalities in Fourier Analysis on ℝ^{n}., Proceedings of the National Academy of Sciences 72 (1975) 638-641.

[Bre] D. M. Bressoud, A radical approach to real analysis, Math. Assoc. of Amer., Washington, DC, 1994.

[Cal] A. Calderón, Intermediate spaces and interpolation, the complex method, Studia Math. 24 (1964), 113–190.

[Car] L. Carleson, On convergence and growth of partial sums of Fourier series, Acta Math. 116 (1966), 135–157.

[Chu] F. R. K. Chung, Spectral Graph Theory, CBMS Regional Conference Series in Mathematics, 92, Amer. Math. Soc. Providence, RI, 1997.

[CM] R. R. Coifman and Y. Meyer, Remarques sur l’analyse de Fourier à fenêtre, C. R. Acad. Sci. Paris, Série I 312 (1991), 259–261.

[CW] R. R. Coifman and V. Wickerhauser, Entropy-based algorithms for best basis selection, IEEE Trans. Information Theory 38 (1992), 713–718.

[CT] J. W. Cooley and J. W. Tukey, An algorithm for the machine calculation of complex Fourier series, Math. Comp. 19 (1965), 297–301.

[Dau88] I. Daubechies, Orthonormal bases of compactly supported wavelets, Comm. Pure and Appl. Math. 41 (1988), 909–996.

[Dau92] I. Daubechies, Ten lectures on wavelets, CBMS–NSF Lecture Notes, vol. 61, SIAM, Philadelphia, PA, 1992.

[Duo] J. Duoandikoetxea, Fourier analysis, English transl. by D. Cruz-Uribe, SFO, Graduate Studies in Mathematics, vol. 29, Amer. Math. Soc., Providence, RI, 2001.

[DM] H. Dym and H. P. McKean, Fourier series and integrals, Probability and Mathematical Statistics, vol. 14, Academic Press, New York–London, 1972.

[Fol] G. B. Folland, Real analysis: Modern techniques and their applications, Wiley, New York, NY, 1999.

[Fra] M. W. Frazier, An introduction to wavelets through linear algebra, Springer–Verlag, New York, NY, 1999.

[Gab] D. Gabor, Theory of communication, J. Inst. Electr. Eng., London 93 (III) (1946), 429– 457.

[Gar] J. B. Garnett, Bounded Analytic Functions, Academic Press, Orlando, FL, 1981.

[Grab] J. V. Grabiner, Who gave you the epsilon? Cauchy and the origins of rigorous calculus, Amer. Math. Monthly 90 No. 3 (1983), 185–194.

[Graf] L. Grafakos, Classical and modern Fourier analysis, Prentice Hall, 2004.

[Grat] I. Grattan-Guinness, The development of the foundations of mathematical analysis from Euler to Riemann, MIT Press, Cambridge, MA, 1970.

[Haa] A. Haar, Zur Theorie der orthogonalen Funktionensysteme, Math. Annal. 69 (1910), 331–371.

[HW] E. Hernandez and G. Weiss, A first course on wavelets, CRC Press, Boca Raton, FL, 1996.

[Hig] N. J. Higham, Handbook of Writing for the Mathematical Sciences, SIAM, Philadelphia, PA, 1998.

[Hun] R. Hunt, On the convergence of Fourier series, Orthogonal expansions and their continuous analogues (Proc. Conf., Edwardsville, IL, 1967), Southern Illinois Univ. Press, Carbondale, IL, 1968, pp. 235–255.

[JMR] S. Jaffard, Y. Meyer, and R. D. Ryan, Wavelets: Tools for science and technology, revised edition, SIAM, Philadelphia, PA, 2001.

[KK] J.-P. Kahane and Y. Katznelson, Sur les ensembles de divergence des séries trigonométriques, Studia Math. 26 (1966), 305–306.

[KL] J.-P. Kahane and P.-G. Lemarié-Rieusset, Fourier series and wavelets, Gordon and Breach Publishers, 1995.

[Kat] Y. Katznelson, Introduction to harmonic analysis, Dover Publications Inc., 1976.

[Kol] A. Kolmogorov, Une série de Fourier–Lebesgue divergente partout, C. R. Acad. Sci. Paris 183 (1926), 1327–1328.

[Koo] p. Koosis, Introduction the H_{p} spaces, Cambridge University Press, 1980.

[Kor] T. W. Körner, Fourier analysis, Cambridge Univ. Press, Cambridge, 1988.

[Kra] S. G. Krantz, A panorama of harmonic analysis, The Carus Mathematical Monographs, vol. 27, Amer. Math. Soc., Providence, RI, 1999.

[Mall89] S. Mallat, Multiresolution approximations and wavelet orthonormal bases for L^{2}(ℝ), Trans. Amer. Math. Soc. 315 (1989), 69–87.

[Mall98] S. Mallat, A wavelet tour of signal processing, Academic Press, San Diego, CA, 1998.

[Malv] H. Malvar, Lapped transforms for efficient transform/subband coding, IEEE Trans. Acoust. Speech Signal Process 38 (1990), 969–978.

[MP] M. Mohlenkamp and M. C. Pereyra, Wavelets, their friends, and what they can do for you, EMS Series of Lectures in Mathematics, European Mathematical Society Publishing House, Zurich, to appear.

[Mun] J. R. Munkres, Topology: A first course, Prentice Hall, Englewood Cliffs, NJ, 1975.

[Per] M. C. Pereyra, ¡¡¡¡¡¡¡ .mine Lecture notes on dyadic harmonic analysis, In “Second Summer school in analysis and mathematical physics. Topics in analysis: harmonic, complex, nonlinear and quantization,” Cuernavaca Morelos, Mexico, June 12-22, 2000. S. Pérez-Esteva, C.Villegas eds. Contemporary Mathematics 289 AMS, Ch. I, p. 1-61 (2001). ======= Lecture notes on dyadic harmonic analysis, In “Second Summer school in analysis and mathematical physics. Topics in analysis: harmonic, complex, nonlinear and quantization,” Cuernavaca Morelos, Mexico, June 12-22, 2000. S. Pérez-Esteva, C.Villegas eds. Contemporary Mathematics 289 AMS, Ch. I, p. 1-61 (2001). ¿¿¿¿¿¿¿ .r136

[Pet] S. Petermichl, Dyadic shifts and a logarithmic estimate for Hankel operators with matrix symbol, C.R. Acad. Sci. Paris, Sér. I Math 330 (2000), 455–460.

[Pin] M. A. Pinsky, Introduction to Fourier analysis and wavelets, The Brooks/Cole Series in Advanced Mathematics (Paul J. Sally, Jr, ed.), Brooks/Cole, 2001.

[Pre] E. Prestini, The evolution of applied harmonic analysis: Models of the real world, Birkhäuser, Boston, MA, 2004.

[RS] M. Reed and B. Simon, Methods of modern mathematical physics, Vol. II: Fourier analysis, self-adjointness, Academic Press, 1975.

[Roy] H. L. Royden, Real analysis, 2nd ed., MacMillan, New York, NY, 1968.

[Rud] W. Rudin, Principles of mathematical analysis, McGraw–Hill, New York, NY, 1976.

[Sad] C. Sadosky, Interpolation of operators and singular integrals, Marcel Dekker, Inc., New York, NY, 1979.

[Sch] M. Schechter, Principles of functional analysis, Graduate Studies in Mathematics, vol. 36, 2nd ed., Amer. Math. Soc., Providence, RI, 2002.

[Sha] C. E. Shannon, Communications in the presence of noise, Proc. Inst. Radio Eng. 37 (1949), 10–21.

[Spi] M. Spiegel, Schaum’s outline of Fourier analysis with applications to boundary value problems, McGraw–Hill, New York, NY, 1974.

[Ste] E. M. Stein, Singular integrals and differentiability properties of functions, Princeton Univ. Press, Princeton, NJ, 1970.

[SS1] E. M. Stein and R. Shakarchi, Fourier analysis: an introduction, Princeton Lectures in Analysis I, Princeton Univ. Press, Princeton, NJ, 2003.

[SS2] E. M. Stein and R. Shakarchi, Real analysis: Measure theory, integration, and Hilbert spaces, Princeton Lectures in Analysis III, Princeton Univ. Press, Princeton, NJ, 2005.

[SW] E. M. Stein and G. Weiss, Introduction to Fourier analysis on Euclidean spaces, Princeton Univ. Press, Princeton, NJ, 1971.

[SN] G. Strang and T. Nguyen, Wavelets and filter banks, Wellesley–Cambridge Press, Wellesley, MA, 1996.

[Str] R. S. Strichartz, The way of analysis, Jones & Bartlett Publishers; revised edition, 2000.

[Str2] R. S. Strichartz, A guide to distribution theory and Fourier transforms, Studies in Advanced Mathematics, CRC Press, Boca Raton, FL, 1994.

[Tao1] T. Tao, Analysis I, Text and Readings in Mathematics, vol. 37, Hindustan Book Agency, India, 2006.

[Tao2] T. Tao, Analysis II, Text and Readings in Mathematics, vol. 37, Hindustan Book Agency, India, 2006.

[Ter] A. Terras, Fourier analysis on finite groups and applications, London Math. Soc. Student Texts, vol. 43, Cambridge Univ. Press, Cambridge, 1999.

[Tor] A. Torchinsky, Real-variable methods in harmonic analysis, Pure and Applied Mathematics, vol. 123, Academic press, Inc., San Diego, CA, 1986.

[VK] M. Vetterli and J. Kovačević, Wavelets and subband coding, Prentice Hall, Upper Saddle River, NJ, 1995.

[Wic] V. Wickerhauser, Adapted wavelet analysis from theory to software, A. K. Peters, Wellesley, 1994.

[Woj] P. Wojtaszczyk, A mathematical introduction to wavelets, London Math. Soc. Student Texts, vol. 37, Cambridge Univ. Press, Cambridge, 2003.

[Zyg] A. Zygmund Trigonometric Series, 2nd Ed., Cambridge, UK: Cambridge University Press, 1959.