Neil Cantle and David Ingram highlight the perils of modelling outcomes and show how to avert danger with systems thinking
Then someone asked "What would happen if the collision wasn't head on?". It turned out that crashing the same car at a different angle caused fatal intrusion into the cabin. Cars are now designed to withstand collisions at a range of angles, making passengers safer.
You can imagine what they had to go through to challenge the current crash test model. First, someone had to have the 'outside-the-box' idea. Then they would have studied accident information and built a model to simulate a large number of impacts at various angles. Lastly, they would have used that research to change their testing process. This concept of planning for resilience rather than optimisation is often appropriate for complex systems and the best actuarial work goes through a similar pattern.
At the same time, however, researchers have been studying decision-making processes with valuable insights. Actuaries can and should learn from this, particularly as they move into new areas such as risk management, where spotting and interpreting new trends is important.
Statistics in general and actuarial techniques in particular have proven to be immensely powerful to find and exploit situations where either combining or separating risks can be advantageous. Statistics is all about data: collecting it, organising it, analysing it and interpreting it. In the hands of actuaries, studies of mortality, health, natural disasters, motor accidents and so on have all yielded to statistical analyses, permitting individual risks to be seen as sufficiently predictable for companies to accept them for a suitable premium. Much of the world is highly complex, so statistical approaches have helped understanding, even where detailed knowledge of mechanics of the problem is absent.
All about the outcome
Statistics form a major part of the scientific revolution that has transformed human existence over the past several hundred years. They are commonly used as a part of a rational decision-making (RDM) process. Optimising is accomplished by selecting the course of action that has the best expected value.
The apparent success of statistical approaches has arguably focused a generation of practitioners on modelling outcomes, but often forgetting the assumptions that are made at the start of the statistical process. While studying the trend in outcomes is a powerful way to understand a process, it does not provide insight into what causes the outcomes.
This can reduce credibility and also causes modellers to miss when the real world has diverted from the modelled world. This can be particularly dangerous when making predictions about rare events where you cannot directly observe conflicting evidence.
Other sciences have found that complex behaviours and outcomes are not necessarily the result of complex rules and that interactions between factors are crucially important to the understanding of such systems. Applying a systems approach to understanding a problem requires the modeller first to appreciate the whole, before applying reductionist methods to replicate it.
This discipline of remaining open-minded about the driving processes and then updating beliefs is a good way to prevent a model from being seen as the unassailable truth. An unhealthy belief in models arguably contributed to the recent financial crisis.
For a long time, psychologists have been studying how people make decisions. The work of Gerd Gigerenzer explains that fast and frugal heuristics (FFH) is the best representation of how the bulk of humanity thinks. With FFH, people will naturally form decision rules by automatically evaluating thousands of possible clues and quickly isolating the few that are needed to find a reasonable solution.
By contrast, in the RDM approach frequently taught in schools, a decision-maker will identify potential solutions to a problem and then evaluate the characteristics of the outcomes under those solutions to find the optimal choice. Herbert Simon (1957) identified a major flaw in this RDM approach: there is no natural limit to the analysis needed to reach the optimal conclusion. Simon suggested that RDM required 'unbounded rationality' - in other words, potentially infinite data and infinite analysis.
Simon went on to develop the idea of 'satisficing' as a decision criterion. Instead of seeking an optimal outcome, satisficing concludes the decision process when a satisfactory outcome has been found.
'Outside-the-box' thinking is usually based on systems thinking. This is a world view that problems cannot be addressed by reduction of the system and that system behaviour is about interactions and relationships and the emergent behaviours they produce. It is also a process, or methodology, for understanding complex system behaviour to help you see both the forest and the trees. In most situations, people just assume that tomorrow's weather will be the same as today. It is more efficient for a brain to spot patterns than to think too deeply about the underlying causes, so it is not surprising that most people default to assessing problems that way too.
Statistical models are by themselves insufficient to capture the range of possibilities of a complex adaptive system. Systems thinking can provide the insights that allow actuaries and other quantitative analysts to look around the corner of the situation they are modelling.
Experts tend not to follow RDM in practice as they find it more effective to spend most of their time considering the problem with one solution in mind at a time rather than considering all the possible solutions - this is known as natural decision-making (NDM). Most business decisions are based upon a set of both heuristic and NDM processes.
In the areas where actuaries often work, statistical analysis can work better than simple heuristic methods. This is usually because the number of potentially important variables is much larger than can be dealt with by heuristics. So the heuristic approach may well call on an actuary for statistical analysis.
However, when even with additional statistical analysis the heuristic approach is not living up to expectations, the actuary can think of their work as part of a cycle to update both the statistical analysis and the heuristic approach using systems thinking.
While the areas of heuristic processes, NDM and systems thinking may be reflected in some actuarial processes, it is likely to be in an underdeveloped way. Bringing together these different approaches can therefore be used to improve actuarial processes.
Actuaries usually identify when model results no longer ring true. Systems thinking can provide the tools for more consistently finding the right route through problems and identifying the gaps in existing approaches. NDM and heuristic methods can provide ways for actuaries to better communicate their findings to others. As actuaries move into fields with complex phenomena like wider risk, it is important that they use tools to help others listen to their conclusions.
Gerd Gigerenzer, Rationality for Mortals: How People Cope with Uncertainty (2008)
Herbert Simon, A Behavioral Model of Rational Choice (1957)