Skip to main content
The Actuary: The magazine of the Institute and Faculty of Actuaries - return to the homepage Logo of The Actuary website
  • Search
  • Visit The Actuary Magazine on Facebook
  • Visit The Actuary Magazine on LinkedIn
  • Visit @TheActuaryMag on Twitter
Visit the website of the Institute and Faculty of Actuaries Logo of the Institute and Faculty of Actuaries

Main navigation

  • News
  • Features
    • General Features
    • Interviews
    • Students
    • Opinion
  • Topics
  • Knowledge
    • Business Skills
    • Careers
    • Events
    • Predictions by The Actuary
    • Whitepapers
    • Moody's - Climate Risk Insurers series
    • Webinars
    • Podcasts
  • Jobs
  • IFoA
    • CEO Comment
    • IFoA News
    • People & Social News
    • President Comment
  • Archive
Quick links:
  • Home
  • The Actuary Issues
  • November 2014
11

Top 10 forecasting fallacies

Open-access content Wednesday 29th October 2014

Whether they realise it or not, most actuaries are at risk of some form of bias in their modelling work. Michael Ortmann explores the most common pitfalls

2

"You have been biased!" - what an unpleasant rebuke for an actuary. Yet, this characterisation may not be too far-fetched. Exhibiting some form of bias is all too human, but hardly any quantitative analyst would admit to it.

Risk analysis and risk modelling have become a core competence of actuaries. In particular, actuaries make numerous predictions about the future with respect to pensioners' longevity, mortality and morbidity incidence rates and solvency capital requirements to name but a few. On the other hand, the traps an actuary can easily fall into give rise to particular concern.


Perception versus reality 

Perception is another hurdle to overcome. 

A recent 'Reality Cheque' survey conducted by Hymans Robertson showed that individuals underestimate life expectancy at retirement by some five to eight years on average (Figure 2, below). This is possibly a function of personal anchor points, the inability to conceptualise living in old age, and the media reporting 'period' rather than 'cohort' life expectancies (the former not allowing for future mortality improvements). 


As expected, the perception gap has been shown to close as individuals consider life expectancy at older ages. This requires us to take different approaches for different age groups. 


10 Incentives

It is unheard of for any capital actuary to have been fired for inaccurate projections of the future. In general, there is no reward - and no reprimand either - for forecasts duly conducted, unless there is gross negligence at work. On the contrary, remuneration and bonuses are based on enhancing the risk framework by building internal Solvency II models, applying state-of-the-art statistical techniques and encompassing ever more risks. In other words, effort matters. The incentive bias refers to the misalignment of incentives and desired outcomes.


9 Who's in charge?

Problems can arise if the actuary has a superior who asserts undue pressure and influence over the actuary's work. Sadly, unquestioning acceptance of authority implies fatally simplified thinking. This bias refers to being overly submissive to authority so it affects the actuary's impartial modelling. What happens if the company is not able to afford the solvency capital that the risk actuary deems necessary? Not surprisingly, the dangers for the profession invoked by authority bias are part of the IFoA's professionalism courses.


8 Telling stories

Whatever the outcome of a forecast, the actuary's superiors need a story to buy into it. A convincing story that explains projected results and renders them plausible is vital. In fact, it is often seen by some as more important than the forecast itself. The actuary puts results into a commercial context. The story bias refers to the tendency to be more easily convinced by a narrative than numbers. In an extreme scenario, management may buy into a capital requirement figure that is not robust, simply because there is a convincing story to support it. 


7 Anchoring to the past

A seasoned Solvency II actuary carefully weighs up projected results against previous forecasts and market benchmarks. The anchor bias postulates that we always relate new results to past established results. In fact, an actuary has to come up with a good reason to overwrite anything key stakeholders have bought into previously. As a consequence, past mistakes that have nested are very difficult to correct.


6 Does the answer look right?

Undoubtedly, for any stochastic forecast for example, there is an abundance of parameters that the actuary in charge needs to set carefully. Most practical actuaries would concur that modelling is a mix of art and science. Quite often, in a process of tinkering and twiddling with the assumptions, the actuary arrives at a projection that looks reasonable. This tendency is called outcome bias. Thus, the actuary risks judging a model on the basis of its results. Not surprisingly, the best guess as well as likely outcomes will comply with mainstream forecasts. 


5 Where's my data?

When data is scarce, capital model actuaries risk making incorrect assumptions with reference to the applicability of the normal distribution, stochastic independence, correlation matrices, the Black-Scholes formula, and so on. Likewise, actuaries tend to use data that is readily available in order to evaluate certain risks where data is scarce, such as operational risk, for instance. As a consequence, results have to be taken with a pinch of salt. Any insights gained from such a model exhibit the availability bias. 


4 Lots of effort, must be right!

Actuarial models are becoming ever more sophisticated. The complexity bias refers to the tendency to overly appreciate the labour that has gone into a forecast model. Most tend to believe less in simple plausibility calculations than in complex computations that have required a good deal of mental effort, such as internal models for Solvency II. Interestingly, some academic research has revealed that during an acute crisis, financial markets behave like a single factor model, as all assets become co-monotonic. Stress scenario models do not necessarily need to be complex per se.


3 But the model's been back-tested!

Comprehensive mortality forecasting, in particular, entails a number of different back-testing exercises. These tests have in common that model parameters are calibrated in order to forecast the past based on previous experience. If the model works fine for a number of such ex-post evaluations, we deduce by induction that it will always work. Such reasoning is called induction bias. There is, however, no underlying law of cause and effect in actuarial science as there is in natural science. A forecast based on induction can only work if things stay as they are. The implied stationarity of the model has an innocuous but vital consequence - a stationary model tends towards its equilibrium and will always predict this stable equilibrium. 


2 With hindsight…

The most well-known forecasting pitfall relates to hindsight bias. Actuaries analyse and model historic data with a view to forecasting the future. A fundamental assumption is that past experience is a reliable basis for predicting future experience. But there is inherent uncertainty in so doing, since the future may be affected by events that have never before been witnessed, such as the financial crisis that unfolded in 2007. In retrospect, its development over the following years is compelling and logical. There is a heap of experts who can fully explain the whole story from beginning to end - with the benefit of hindsight. However, at no point in time did anybody correctly forecast the subsequent course of global economics. Thus, it must be a misperception to feel knowledgeable about what has gone on. Such an error in reasoning makes us believe that we are better at projecting the future than we actually are.


1 I'll take this dataset, please

The most important drawback of all is called confirmation bias. This phenomenon refers to the tendency to actively select and over-emphasise evidence that supports the aspired results. Likewise, the actuary may ignore and under-emphasise any data that point to the contrary. To be more specific, a reasonable Solvency II actuary usually tries to render a capital forecast plausible by arriving at about the same figure in different ways. By so doing, the actuary actively searches for confirming evidence. On the other hand, an actuary does not usually take the trouble to falsify a satisfactory modelling outcome. As such, there is a risk of disconfirming evidence being discarded. 


Know your unconscious!

In a nutshell, a bias inveigles an actuary into favouring a particular forecast over any alternative projected result. The model outcome may be no more than a common sense conclusion. Erroneous beliefs in a seemingly objective model may seriously affect an actuary's impartial judgement. Unconscious attitudes play an important role in this respect. Implicit fallacies may result in overconfidence and self-delusion. Therefore, it is crucially important to be aware of potential errors in reasoning.


This article appeared in our November 2014 issue of The Actuary.
Click here to view this issue
Filed in
11
Topics
Modelling/software

You might also like...

Share
  • Twitter
  • Facebook
  • Linked in
  • Mail
  • Print

Latest Jobs

Senior Catastrophe Analyst

London, England
£70000 - £100000 per annum
Reference
146055

Catastrophe Analyst

London, England
Up to £50000 per annum + + Bonus
Reference
146053

Principal Pricing Analyst

England, London
£60000 - £70000 per annum
Reference
146052
See all jobs »
 
 

Today's top reads

 
 

Sign up to our newsletter

News, jobs and updates

Sign up

Subscribe to The Actuary

Receive the print edition straight to your door

Subscribe
Spread-iPad-slantB-june.png

Topics

  • Data Science
  • Investment
  • Risk & ERM
  • Pensions
  • Environment
  • Soft skills
  • General Insurance
  • Regulation Standards
  • Health care
  • Technology
  • Reinsurance
  • Global
  • Life insurance
​
FOLLOW US
The Actuary on LinkedIn
@TheActuaryMag on Twitter
Facebook: The Actuary Magazine
CONTACT US
The Actuary
Tel: (+44) 020 7880 6200
​

IFoA

About IFoA
Become an actuary
IFoA Events
About membership

Information

Privacy Policy
Terms & Conditions
Cookie Policy
Think Green

Get in touch

Contact us
Advertise with us
Subscribe to The Actuary Magazine
Contribute

The Actuary Jobs

Actuarial job search
Pensions jobs
General insurance jobs
Solvency II jobs

© 2023 The Actuary. The Actuary is published on behalf of the Institute and Faculty of Actuaries by Redactive Publishing Limited. All rights reserved. Reproduction of any part is not allowed without written permission.

Redactive Media Group Ltd, 71-75 Shelton Street, London WC2H 9JQ