The Fall From Scientific Grace Of Macroeconometrics

During the 1960s, many Keynesian-type macroeconometric models were devel­oped, all with a certain scientific air. These included the Data Research Institute (DRI) model, the Wharton model, and various Federal Reserve models. As predictors of the economy, these macro models remained popular through the early 1970s, but by the mid-1970s this work was losing support. In his discussion of these models, Roy Epstein writes:

The confidence of applied econometricians did not last long into the 1970s. The economic shocks of the decade began to invalidate the forecasts from the large structural macro models and drove researchers to constant re-specifications and re-estimation of their systems. This work was accompanied by a growing number of studies that compared the forecasting qualities of large models to a new generation of univariate time series naive models. These comparisons still often showed that the structural models predicted no better than the naive models, an apparent confirmation of Friedman's predictions made in 1949.

The reasons for the criticism of the macroeconometric models were similar to the reasons economists objected to earlier work. First, the validity of classical statistical tests depends upon the theory being developed independently of the data. In reality, however, most empirical economic researchers "mine the data," looking for the "best fit"—that is, the formulation of the theory that achieves the best r2, t, and F statistics (statistics that measure the likelihood that the theory is correct). Data mining erodes the validity of the statistical tests. Second, even where statistical tests are conducted appropriately, the limited availability of data makes it necessary to designate proxies, which may or may not be appropriate. Thus, the validity of the tests depends upon the appropriateness of the proxy, but there is no statistical measure of the appropriateness of a proxy. Third, almost all economic theories include some immeasurable variables that can be, and often are, relied upon to explain statistical results that do not conform to the theory. Fourth, replication of econometric tests generally is impossible, because econo­mists can seldom (if ever) conduct a controlled experiment. This makes any result's reliability unknown and dependent upon subjective judgment.

Robert Solow, a Nobel Prize-winning macroeconomist, captured the concern of much of the profession with the formal macroeconometric models when he wrote:

I do not think that it is possible to settle these arguments econometrically. I do not think that econometrics is a powerful or usable enough tool with macroeconomic time series. And so one is reduced to a species of judgment about the structure of the economy. You can always provide models to support your position economet­rically, but that is too easy for both sides. One was never able to find common empirical ground.

Cynicism toward econometric testing has led many researchers to take a cavalier attitude toward their statistical work. The result is that many studies cannot be duplicated, much less replicated, and that mistakes in published empirical articles are commonplace. Edward Leamer, an econometrician at UCLA, summarizes this view. He writes:

the econometric modeling was done in the basement of the building and the econometric theory courses were taught on the top floor (the thitd). I was perplexed by the fact that the same language was used in both places. Even more amazing was the transmogrification of particular individuals who wantonly sinned in the base­ment and metamorphosed into the highest of high priests as they ascended to the third floor.

He suggests that one way out of the dilemma is to use Bayesian econometrics, in which a researcher's degree of belief is taken into account in any statistical test; but the process of doing this is so complicated that most researchers simply continue to do what they have always done.