[Skip to content]

Sign up for our daily newsletter
The Actuary The magazine of the Institute & Faculty of Actuaries

Rewriting history

There are two certainties in life: death and taxes. Oh yes, and that actuaries always get it wrong! At first glance, perhaps a harsh comment, but, when you think about it, absolutely correct.
I’ve recently been to GIRO, where there were a series of prominent interconnected issues on offer:
– the reserving cycle and the GRIT working party;
– the thorny problem of estimating a 99.5th percentile out of an unknown distribution of events correlated in an unknown way (and with unknown exposure to little-known risks); and
– the fallibility of actuarial estimates in general.
The accuracy of our working methods and practices became a recurring topic in conversations and presentations, particularly in the light of recent criticism in the press and the Morris review. We all know that we will never get the right answers. We also know that our estimates are likely to differ from those of all our colleagues working with exactly the same information. So, how can we explain this to our clients?
This topic not only affects how we should communicate directly with our own ‘stakeholders’, but is also very relevant to the image of our profession as a whole not just in general insurance. We need to bolster our defences to guard against the growing tide of ‘name and shame’ that results when financial services stakeholders change their view of the world. It is even more important that we can defend and explain our beliefs when we receive criticism, even though both the previous estimate and the change may have been justified based on the information available at each point in time.

Best estimate or bust
Before we get to that stage, we need to get our own house in order. Ranges of reasonable best estimates, ranges of possible outcomes, and inferred percentiles are all terms we use when discussing results, but do we all mean the same thing? We know that there is a range of best estimates (ie means) because of the ‘individuality’ of actuaries, and their differing experiences and priorities. A range of possible outcomes will also be different for each actuary, potentially with extreme differences at the tails of the distribution. In the middle ground the variation in what actuaries would estimate as a 75th percentile would probably vary more than a best estimate, but less than these extreme tails. (See box.)
An obvious question to ask is whether it is realistic to try to estimate some of these more subjective amounts. Choosing a percentile, as is done for reserve adequacy purposes under the Australian regulatory regime for general insurance, would imply to a numerically literate observer that the amount booked would be adequate 75% of the time. This same number may actually lie in the range of 60% to 85% or even wider, depending on the actuary who has derived this estimate. Does this meet the original purpose of sufficient adequacy?
Would a non-numerical lay person expect reserves to be set at a level where there was approximately a 50% chance of reserve inadequacy? Wouldn’t this look as though the initial estimate was inappropriate and that the actuary had provided poor advice? Even boards may not fully understand that booking their actuaries’ best estimate means they are quite likely to post a prior-year reserve deterioration the next year.

What now?
So, what should we do? Should we start every report with the phrase ‘The figures in this report will not be the true result’ in bold, and concentrate in the rest of the report on trying to explain just how wrong we are likely to be, and why? Would this benefit the recipients of the advice, and enable the reader to understand better the uncertainty in the estimates? Would this help management make more appropriate decisions?
To help answer these questions I signed up for the Estimating and Communicating Uncertainty in Reserves Working Party. My view is that we should produce a challenging discussion document, and perhaps some recommendations. I also hope that everyone will contribute to the debate, as our methods and clarity of communication are fundamental to the work of all actuaries.
Where’s your 75th percentile?
A purely hypothetical situation: imagine a distribution of reserves being estimated by a number of actuaries based on incomplete data. I assume the distributions thus estimated have means of 100 +/- 10, and standard deviations of around 50 +/- 5.
Assuming a LogNormal distribution, admittedly very crude particularly in the tail the implied values of the reserves for a given probability of adequacy are approximately:
Mea– SD 75th 99.5th
90 45 110 270
100 50 123 302
110 55 135 330
We now look at the results of the distribution with mean 90 and consider the percentiles implied by the estimated ‘reserves’ using the distribution with mean of 110, and vice versa. The ‘75th’ percentile results give a range of percentiles of 60‡% and the ‘99.5th’ results give a range of 98.4™.9%. The mean of these distributions lies around the 59.5th percentile. The implied range of results would, of course, be wider if the different actuaries had assumed different distributions as well