Sunday, October 10, 2010

Why Are Forecasters Grumpy?

Laurence Meyer, former Fed Governor, has an interview with Mark Sniderman (Cleveland Fed) available here. Before going to the Fed in 1996, Meyer worked for my current employer, Washington University in St. Louis, and he returned to his forecasting firm, Macroeconomic Advisers, in 1992. Macroeconomic Advisers is headquartered in the municipality where I live, Clayton Missouri. Indeed, when I go to my youngest son's back-to-school night at Clayton High School (which entails making the circuit of my son's classes), I'm often sitting in a class with Jim Bullard (St. Louis Fed President) and Ken Matheny senior economist at Macro Advisers.

In his interview with Sniderman, Meyer is generally quite amiable, but there is a patch where he lets loose:
Sniderman: Should we expect to be living with our mainstream workhorse macro models for some time, and should we feel good about that? Is there enough progress there?

Meyer: I love that question! So I think we have two kinds of modeling traditions. First there is the classic tradition. I was educated at MIT. I was a research assistant to Franco Modigliani, Nobel laureate, and the director of the project on the large-scale model that was used at the time at the Federal Reserve Board. This is the beginning of modern macro-econometric model building. That’s the kind of models that I would use, the kind of models that folks at the Board use.

There’s also another tradition that began to build up in the late seventies to early eighties—the real business cycle or neoclassical models. It’s what’s taught in graduate schools. It’s the only kind of paper that can be published in journals. It is called “modern macroeconomics.”

The question is, what’s it good for? Well, it’s good for getting articles published in journals. It’s a good way to apply very sophisticated computational skills. But the question is, do those models have anything to do with reality? Models are always a caricature—but is this a caricature that’s so silly that you wouldn’t want to get close to it if you were a policymaker?

My views would be considered outrageous in the academic community, but I feel very strongly about them. Those models are a diversion. They haven’t been helpful at all at understanding anything that would be relevant to a monetary policymaker or fiscal policymaker. So we’d better come back to, and begin with as our base, these classic macro-econometric models. We don’t need a revolution. We know the basic stories of optimizing behavior and consumers and businesses that are embedded in these models. We need to go back to the founding fathers, appreciate how smart they were, and build on that.
Well, that is pretty clear. Meyer thinks:

1. There are two strands of macroeconomics. First, we have the macroeconomics of the "founding fathers." The founding fathers created large-scale macroeconometric models in the 1960s. Second, we have "modern macroeconomics," which started with the work of Kydland and Prescott in the early 1980s.

2. The approach of the founding fathers is useful. Modern macreconomics is of no practical use, particular for economic policy.

In 1979-80, I worked in the Research Department of the Bank of Canada. Like all of the other people in the Current Analysis group I worked in, I had a Masters degree from a Canadian university, and I was hired because I had the required toolkit. I knew some time series econometrics and computer science, and could run a regression as well as the next person. The group I worked in was organized like an IS-LM model. There was a consumption person, a business investment person, a prices person, a wages person, and two people who were the "foreign sector," which in Canada is another term for the United States of America. I was the residential construction person. The LM curve resided in another department: Monetary and Financial Analysis.

Part of our job was to run a forecast. Our forecasting model at the time was RDXF, which derived from the Bank of Canada's RDX2 model, which in turn stemmed directly from the work of the "founding fathers." RDXF consisted of about 400 equations. I was responsible for 6 of those equations, which determined housing starts, the price of houses, residential construction expenditures, a housing construction cost index, and two other variables I cannot remember, from variables determined elsewhere in the model. My job was to monitor the performance of those equations, and my input into the forecast was a set of "add factors" or "constant-term adjustments," i.e. quantities added to or subtracted from the variables in my purview.

Now, the people I worked with became quite cynical about what we were accomplishing with our founding fathers model. It became clear that the model, left to its own devices, would do a horrible job of forecasting. The add factors we applied in making our forecasts were not only there to correct for out-of-sample observed forecasting errors, but as a judgemental check on what the model was producing. Ultimately, the model appeared to function only as a device for making sure that national income identities held. This is true of essentially all forecasting models. At the Minneapolis Fed, the Bayesian Vector Autoregressive models developed by Robert Litterman did not forecast well without Litterman input, and I'm sure Macroeconomic Advisers uses a heavy dose of judgement in constructing a forecast.

The models of the founding fathers were essentially elaborate IS-LM models, and they shared some of the disadvantages of that modeling framework. In an IS-LM model, the theory is built sector-by-sector, and one sector has no clue what the other sectors are up to. Neil Wallace (can't remember the citation) was fond of remarking how the Tobin-Baumol framework, which was the theoretical basis for the money demand function, contained costs of transacting with the bank which, while seemingly important for money demand, were nowhere in evidence in the rest of the model. This sectoral approach was enshrined in the organizational structure by which research was done, and in how the founding-fathers models were used. Students, like Laurence Meyer, who studied with Modigliani at MIT, or with Ando at Penn (for example) would do econometric work related to some particular sector of the economy - prices, consumption, investment - and organizations which did forecasting and policy analysis (like the one I worked in) were organized, as I remarked above, like an IS-LM model.

Did I come away from my experience at the Bank of Canada wanting to build on the work of those smart founding fathers, as Myer is suggesting we do now? Not on your life. The experience was certainly useful, as it gave me insight into what Lucas was talking about here. The year after I left the Bank of Canada, Rao Aiyagari introduced me to Models of Monetary Economies, which taught me far more than I could ever learn from the founding fathers. Here were coherent general equilibrium frameworks in which one could see the interrelationships among decision-making in different sectors of the economy (neglected of course, in IS-LM), understand how financial frictions could matter for monetary policy, and evaluate the welfare consequences of alternative policies and regulations.

Now, what has happened to large-scale macroeconometric models? Clearly, they are still alive and well at forecasting firms, including Macroeconomic Advisers. If these models were not useful for forecasting, it seems unlikely that these people would continue to use them. On the policy front, however, and in economic research, founding-fathers models have for the most part been relegated to the dinosaur museum. There are die-hard outposts of founding fathers macroeoconomics, though, the most prominent of which is the Board of Governors, which continues to use the FRB/US model for forecasting and policy analysis. This reference is somewhat old, but it tells you something about what is in the FRB/US model. In spite of paying lip service to "rational expectations," the structure of the model (at least the parts that are revealed here) does not look much different from what I encountered in 1979. For example, there is a Phillips curve relationship that determines prices, and consumption is determined by something that looks like a Keynesian consumption function with appended dynamics.

In spite of what Meyer seems to think, modern macro is very much at work in current policymaking circles. You can see this in the published speeches that Jim Bullard, Narayana Kocherlakota, Jeff Lacker, and Charles Plosser are making. These people are applying the key innovations in macroeconomics of the last 40 years, including: (i) Commitment by policymakers is important; (ii) The fact that aggregate output and employment are fluctuating need not imply a role for stabilization policy; (iii) Observed correlations in macroeconomic data need not reveal structural relationships; and many more. In none of the regional Federal Reserve Banks where I have had occasion to visit does discussion of founding-fathers models appear to enter the policy discussion, and for good reason. A 400-equation model has ceased to be a model. By definition a model is simple enough to help us understand something, and these models just do not do the job. When you are a dinosaur, I guess it can make you grumpy, but Meyer should get over it, and stick to his forecasting.


  1. I looked through some of the sections of Models of Monetary Economics. Tobin's essay is quite critical of Wallace et al. Was that what you were referencing in your previous post regarding Tobin?

  2. Yes, that was one of the things that Tobin wrote during his own grumpy period. I met Tobin once, probably in 1989 or 1990. I thought some of Tobin's work on financial intermediation was interesting and wanted to know what he thought of the new intermediation theories that people (Doug Diamond, Diamond-Dybvig, etc.) were working on. Basically, he said he thought it was crap. Needless to say, he didn't come off well.

  3. It seems Tobin was right that overlapping generation is not crucial for money. What we know right now is record keeping.

    So what did Tobin think about financial intermediation?

  4. Tobin actually had more in common with Wallace than he realized. They shared the idea that we shouldn't draw some line between assets that are "monetary" and those that are not. In the OG model, just as in early search models or Townsend's turnpike model, the lack of recordkeeping was implicit. OG is a nice representation of the double coincidence problem. Tobin's problem seems to have to do with the abstraction in the OG framework. Basically, he didn't observe real-life consumers saving for retirement by holding fiat money, therefore he thought the model was bad.

  5. "They shared the idea that we shouldn't draw some line between assets that are "monetary" and those that are not."

    Would you be referring to Wallace's idea that bonds could be monetary assets, except legal restrictions prevent them from being so? Or there is more to the idea of not drawing a line between monetary/nonmonetary assets than that?

  6. It's that, and in Sargent and Wallace (1982), there's the idea that currency and some intermediary liabilities could look like they are playing similar roles, but they are really entirely different. That paper is quite negative on monetarism, as was Tobin.

  7. Is that their article on the real bills doctrine or is it "Some Unpleasant Monetarist Arithmetic"?

  8. It's this one: