Advanced Linear Modeling: Multivariate, Time Series, and Spatial Data; Nonparametric Regression and Response Surface Maximization

Data Files

Preface to Second Edition, Preface to First Edition, Table of Contents

Preface to Second Edition

This is the second edition of Linear Models for Multivariate, Time Series and Spatial Data. It has a new title to indicate that it contains much new material. The primary changes are the addition of two new chapters: one on nonparametric regression and one on response surface maximization. As before, the presentations focus on the linear model aspects of the subject. For example, in the nonparametric regression chapter there is very little about kernal regression estimation but quite a bit about series approximations, splines, and regression trees, all of which can be viewed as linear modeling.

The new edition also includes various smaller changes. Of particular note are a subsection in Chapter 1 on modeling longitudinal (repeated measures) data and a section in Chapter 6 on covariance structures for spatial lattice data. I would like to thank Dale Zimmerman for the suggestion of incorporating material on spatial lattices. Another change is that the subject index is now entirely alphabetical.

Preface to First Edition

This is a companion volume to Plane Answers to Complex Questions: The Theory of Linear Models. It consists of six additional chapters written in the same spirit as the last six chapters of the earlier book. Brief introductions are given to topics related to linear model theory. No attempt is made to give a comprehensive treatment of the topics. Such an effort would be futile. Each chapter is on a topic so broad that an in depth discussion would require a book length treatment.

People need to impose structure on the world in order to understand it. There is a limit to the number of unrelated facts that anyone can remember. If ideas can be put within a broad, sophisticatedly simple structure, not only are they easier to remember but often new insights become available. In fact, sophisticatedly simple models of the world may be the only ones that work. I have often heard Arnold Zellner say that, to the best of his knowledge, this is true in econometrics. The process of modeling is fundamental to understanding the world.

In Statistics, the most widely used models revolve around linear structures. Often the linear structure is exploited in ways that are peculiar to the subject matter. Certainly this is true of frequency domain times series and geostatistics. The purpose of this volume is to take three fundamental ideas from standard linear model theory and exploit their properties in examining multivariate, time series and spatial data. In decreasing order of importance to the presentation, the three ideas are: best linear prediction, projections and Mahalanobis's distance. (Actually, Mahalanobis's distance is a fundamentally multivariate idea that has been appropriated for use in linear models.) Numerous references to results in Plane Answers are made. Nevertheless, I have tried to make this book as independent as possible. Typically, when a result from Plane Answers is needed not only is the reference given but also the result itself. Of course, for proofs of these results the reader will have to refer to the original source.

I want to re-emphasize that this is a book about linear models. It is not traditional multivariate analysis, time series, or geostatistics. Multivariate linear models are viewed as linear models with a nondiagonal covariance matrix. Discriminant analysis is related to the Mahalanobis distance and multivariate analysis of variance. Principle components are best linear predictors. Frequency domain time series involves linear models with a peculiar design matrix. Time domain analysis involves models that are linear in the parameters but have random design matrices. Best linear predictors are used for forecasting time series; they are also fundamental to the estimation techniques used in time domain analysis. Spatial data analysis involves linear models in which the covariance matrix is modeled from the data; a primary objective in analyzing spatial data is making best linear unbiased predictions of future observables. While other approaches to these problems may yield different insights, there is value in having a unified approach to looking at these problems. Developing such a unified approach is the purpose of this book.

There are two well known models with linear structure that are conspicuous by their absence in my two volumes on linear models. One is Cox's (1972) proportional hazards model. The other is the generalized linear model of Nelder and Wedderburn (1972). The proportional hazards methodology is a fundamentally nonparametric technique for dealing with censored data having linear structure. The emphasis on nonparametrics and censored data would make its inclusion here awkward. The interested reader can see Kalbfleisch and Prentice (1980). Generalized linear models allow the extension of linear model ideas to many situations that involve independent nonnormally distributed observations. Beyond the presentation of basic linear model theory, these volumes focus on methods for analyzing correlated observations. While it is true that generalized linear models can be used for some types of correlated data, such applications do not flow from the essential theory. McCullagh and Nelder (1989) give a detailed exposition of generalized linear models and Christensen (1990) contains a short introduction.


I would like to thank MINITAB for providing me with a copy of release 6.1.1, BMDP for providing me with copies of their programs 4M, 1T, 2T, and 4V and Dick Lund for providing me with a copy of MSUSTAT. Nearly all of the computations were performed with one of these programs. Many were performed with more than one.

I would not have tackled this project but for Larry Blackwood and Bob Shumway. Together Larry and I reconfirmed, in my mind anyway, that multivariate analysis is just the same old stuff. Bob's book put an end to a specter that has long haunted me: a career full of half-hearted attempts at figuring out basic time series analysis.

At my request, Ed Bedrick, Bert Koopmans, Wes Johnson, Bob Shumway and Dale Zimmerman tried to turn me from the errors of my ways. I sincerely thank them for their valuable efforts. The reader must judge how successful they were with a recalcitrant subject. As always, I must thank my editors Steve Fienberg and Ingram Olkin for their suggestions. Jackie Damrau did an exceptional job in typing the first draft of the manuscript.

Finally, I have to recognize the contribution of Magic Johnson. I was so upset when the 1987-88 Lakers won a second consecutive NBA title that I began writing this book in order to block the mental anguish. I am reminded of Woody Allen's dilemma: is the importance of life more accurately reflected in watching The Sorrow and the Pity or in watching the Knicks? (In my case, the Jazz and the Celtics.) It's a tough call. Perhaps life is about actually making movies and doing statistics.

Table of Contents

Advanced Linear Modeling

Buy Advanced Linear Modeling now!

Web design by Ronald Christensen (2007) and Fletcher Christensen (2008)