Why we don't need better forecasts: Kemp

One of my first assignments as a newly-minted graduate in the late 1990s was to produce a 30-year forecast for the Bangladesh taka against the U.S. dollar on behalf of a client contemplating a multi-billion dollar investment.

Even then, I realised this was not a sensible question. I wasn't even sure how much of Bangladesh would still be above sea level in 30 years, let alone the exchange rate. If the economics of the project depended on an exchange rate forecast in three decades time, it was not a very good project.

But the client is always right, so we produced a forecast, and everyone was happy. The client got an independent third-party forecast to use in their financial calculations and we got paid. If the forecast turned out wrong, the clients could blame their independent forecaster, and we could blame an unforeseen change in circumstances.

Like many other analysts, I have spent most of my working life producing forecasts of one sort or another -- ranging from interest rates and exchange rates to metal prices, oil prices and politics.

For an analyst, the solution to any problem is always more information, more data and more sophisticated models, so decisions can be based on more accurate predictions about the future.

But the truth is that we are not very good at it.

In some areas, our forecasting has become amazingly accurate. We understand the laws of physics well enough to land a spacecraft on a comet just 4 km across and hurtling through space at 38 km per second after a journey of more than ten years.

Employing enormous super-computers, we can make short-term weather forecasts that are far more accurate than 30 years ago. Britain's four-day weather forecast is now as accurate as the one-day forecast was in 1980, according to the Meteorological Office.

But once we move beyond physical systems to systems involving human agents who adapt and learn from their surroundings, or push the forecasting horizon further into the future, forecasting accuracy breaks down quickly.

With hundreds of PhDs and lots of fancy dynamic stochastic general equilibrium (DSGE) models about how the economy works, the world's top central banks cannot reliably predict unemployment, inflation and growth even six months ahead let alone spot looming recessions and financial crises.

Large-scale weather and climate models quickly break down once the forecasting horizon is pushed further into the future. For much of the first part of this year, the major models have been predicting an El Nino in the Pacific which has still not occurred.

None of the world's intelligence agencies and international relations specialists successfully predicted the sudden collapse of the Soviet Union, or the fall of the Berlin Wall, or the Arab Spring.

And no one has consistently and successfully predicted bond yields, stock market valuations or oil prices with a track record that cannot be explained simply by random chance.

These are all examples of complex adaptive systems (CAS) in which all the components are tightly linked to one another, can adapt to past outcomes, and are sometimes very sensitive to even a small change in conditions deep within the system, which occasionally produces an enormous change in outcomes through cascading effects and reinforcing feedback.

When it comes to complex and adaptive systems, our forecasting abilities have not improved much since classical times, when Paris of Troy pronounced Aphrodite the most beautiful goddess and abducted Helen of Sparta without realising it would set in motion a complex chain of events that would end with the destruction of his home city.

COPE WITH UNCERTAINTY

The common response is to redouble our efforts, collect more information and build more sophisticated models. But perhaps it makes more sense to focus on managing uncertainty about the future rather than trying to predict it out of existence.

In a powerful paper on "Dealing with femtorisks in international relations," published in the Proceedings of the National Academy of Sciences, an international group of researchers argue the best way to deal with low probability/high consequence risks which occur in complex adaptive systems is to create "new ways of coping with uncertainties that do not depend on precise forecasts of the probability and consequences of future events."

Rather than employing the forecasting approach central to elementary physics and relying on "improvements in the quality of predictions about increasingly small-scale actors, low probability events and the cross-scale implications of rapidly evolving technologies" they suggest employing an approach borrowed from biology.

In evolutionary biology, the most robust and resilient systems are those which exhibit diversity, flexibility, redundancy and adaptability to cope with unknown threats.

"Rather than seek to design robust institutions and strategies that may fare well only against specified futures, risk measurement and management professionals would benefit from imitating how evolution has dealt with unknowable challenges," the researchers urged.

"This approach shifts ... policy options away from optimal, often brittle solutions that require accurate predictions in favour of resilient solutions that can be adapted in response to new information and experiences."

There is nothing new in the idea that the best strategies are the ones which maximise flexibility and adaptability. The concept is central to John Maynard Keynes' theory about investment in the "General Theory of Employment, Interest and Money" (1936) and Frank Knight's theory about the role of the entrepreneur in "Risk, Uncertainty and Profit" (1921).

It is central to the idea of defence in depth employed by the designers of nuclear power plants. It has been taught for decades in business schools as real options theory.

It is essential to the strategy of successful traders: no investor should ever put on a large position without knowing how they would get out of it and preferably having several options for doing so. Traders and executives look for real options and natural hedges to protect them against an uncertain future.

Flexibility, adaptability and options are central to success. Systems, organisations and individuals that adapt and evolve survive. Those which cannot or do not die.

None of this is meant to disparage the work that forecasters do or suggest it is not valuable. Forecasts can tell us a lot about simple systems in the short term. Even for longer time horizons and complex systems, building forecasts can enforce intellectual discipline and consistency, sharpen our thinking about causal relationships, and help us focus on risks and opportunities.

But it is a powerful reason to be humble about our predictive abilities and realise the best way to face an uncertain future is to focus on creating flexible, adaptive and innovative systems, avoiding rigidity, rather than simply throwing money and effort at ever more complex forecasting systems.

I can't remember what 30-year currency forecast we provided back in the late 1990s. At the time, the taka was trading around 42 to the U.S. dollar. Almost twenty years later it is trading at nearly 80. I still have no idea where it will be in ten years time. And I've learned not to worry about it.

By John Kemp