Predicting stock market returns

Robert J. Shiller, Project Syndicate

Historically, the stock market has performed well. In his celebrated 2002 book Stocks for the Long Run, Jeremy Siegel shows that the American stock market returned 6.9 percent per year in real terms between 1802 and 2001. Though the return varied by decade, even turning negative in some decades, overall it performed fairly consistently. This 6.9 percent annual average return has since been referred to as "Siegel's constant," as if Siegel had discovered a new law of nature.

The idea that stocks will perform well in the future has many promoters today, especially among those trying to sell investments in stocks. In the United States, President George W. Bush's Commission to Strengthen Social Security cited Siegel for its claim that the government should encourage people to invest in stocks. Bush has been traveling the country promoting a plan to introduce personal retirement accounts invested in stocks and bonds. The plan assumes a 6.5 percent real return for stocks -- only slightly below Siegel's constant -- in future decades.

But most people don't believe that the stock market will perform so well in the future. Siegel himself recently projected only a 6 percent average real return for U.S. stocks over the next four decades. Others have lower expectations.

I have been conducting surveys of U.S. investors under the auspices of the Yale School of Management, asking what percentage change they expect for the Dow Jones Industrial Average. The expected one-year increase in the Dow in 2005 averages 4.8 percent for institutional investors and 4.3 percent for individual investors.

On closer inspection, the idea that the market will yield a real 6.9 percent a year in the future appears suspect. Think about it: Investing in the stock market at 6.9 percent a year, and reinvesting any dividends, means that, in a tax-free account, the real value of the investment will double every ten years. At that rate, a 20-year-old who in 1960 invested $4,000 a year in a tax-free account in the U.S. stock market would have one million dollars by age 65 today. Should we expect to be able to do that in the future?

Obviously, most people didn't invest this way in 1960. But could most people have? If so, how would the economy, with the labor and material resources at its disposal, provide the large houses, luxury cars, and high-end services that millionaires expect? It is natural to suppose that it could not.

In fact, statistics on past stock market performance mislead because of what statisticians call "selection bias," which occurs when the sample from which a statistic is derived is not representative of all the data. Several kinds of selection bias must be considered when we look at Siegel's constant.

The most fundamental problem is that, in examining stock market investments, we are selecting an economic activity because it was a consistent success in the past. We are trying to extrapolate the past experience of a small fraction of the world population that we have chosen to examine because they made a lot of money. Of course, if one looks at many different investment strategies and many different countries, one can find something that performed spectacularly in the past, even if there is no strategy that can be expected to do so well in the future.

The U.S. had one of the world's most successful stock markets in the twentieth century. Elroy Dimson, Paul Marsh, and Mike Staunton wrote in their book Triumph of the Optimists that of 15 countries that have advanced economies today, the U.S. stock market ranked fourth in its rate of return between 1900 and 2000, behind Australia, Sweden, and South Africa. The geometric average real return of the U.S. stock market was 6.7 percent, but the median geometric real return for the other countries was only 4.7 percent.

But even this comparison involves a selection bias. Countries with more successful markets are more likely to have complete data on both prices and dividends for a hundred years.

A study by Philippe Jorion and William Goetzmann found 39 countries with reliable stock price data -- though not dividend data -- for a good part of the twentieth century. Their sample included countries in Latin America and Asian countries beyond Japan. They found that the median real stock price appreciation from 1920 to 1996 for all these countries was only 0.8 percent, compared to 4.3 percent for the U.S. The U.S. was actually ranked first among the 39 countries.

Of course, even looking at these countries entails selection bias, for it excludes countries without price data for much of the twentieth century, notably China and Russia, where communist revolutions terminated the stock markets, resulting in -100 percent returns for investors. The particular problems that prevented us from observing the returns on these stock markets will never be repeated, but it is wrong to assume that problems of that scale will not recur.

There is also the selection bias that we infer from looking at the twentieth century, the most successful in terms of economic growth in human history. The twenty-first century will be different in ways that we cannot fathom today.

Of course, investing in stocks is not a bad thing. Indeed, the stock market is an important component of any modern economy. But we should not make plans that rely on high returns, as many (including some governments) appear to be doing.

The writer is Professor of Economics at Yale University, Director at Macro Securities Research LLC, and author of Irrational Exuberance and The New Financial Order: Risk in the 21st Century.