Chris Hursey explores a surprising result in insurance capital modelling, and warns of the dangers of focusing too heavily on biting scenarios
Solvency II has led a number of life and general insurers to develop internal capital models. In these models, thousands of potential scenarios are produced, reflecting a range of possible outcomes for economic and insurance risks. Within each of these scenarios, the insurer revalues its balance sheet. The solvency capital requirement is then set so as to ensure solvency in all but a one-in-a-200-year event. In other words, the 'tail' of the capital distribution needs to be covered.
While the basic concepts of simulation-driven capital modelling will be familiar to many practitioners, the challenge remains as to how to revalue a balance sheet in thousands of different scenarios within a short space of time. The calculation of liabilities itself is a complicated process, and computing capacity is finite. Proxy modelling seeks to address this with simplified models. The trick is to ensure that the accuracy of the result is not compromised. The Proxy Models Working Group was set up to explore this approach. In the remainder of this article, we share some interesting early findings relating to the proxy modelling of life insurance with-profit liabilities.
The primary purpose of producing all these scenario results is to allow us to draw quantile results, in particular, the 99.5th percentile capital requirement. By ranking the scenarios in order of capital requirement we are able to derive the result at the 'biting' scenario, beyond which all scenarios produce a higher required capital than our chosen quantile.
One of the problems associated with any proxy model is how to assess accuracy reliably without testing every single scenario. If we could run every scenario then a proxy model would not be required in the first place. As a result, it is common for the proxy-model-derived biting scenario to be analysed, checked and even re-evaluated in the primary model to arrive at the final capital result. As we shall see, however, care should be taken with how this biting scenario is used.
Our aim was to compare, across thousands of scenarios, the actual value of with-profit guarantee costs to those approximated by a proxy model. For the actual value, we built a cashflow projection model combined with a Black-Scholes closed form solution for the time value of guarantees. The proxy model took the form of polynomial formulae, calibrated using least-squares regression against 200 random in-sample scenarios and tested against 20,000 out-of-sample scenarios. The product was simple enough to ensure accuracy of the actual valuation approach. This seemed like a sensible starting point, rather than delving straight into complex guarantee structures requiring nested stochastic models for actual capital values. A range of nine market and insurance risks were incorporated and over 1,000 model points of varying term to maturity and 'moneyness' of guarantees were chosen. For simplicity, we assumed that the cost of guarantees was backed by a fixed cash amount and also that the asset share liabilities were exactly matched by backing assets, so that the capital result is driven purely by the variation in cost of guarantees.
A spooky result
By most conventional measures - in which actual results are compared with proxy results for individual scenarios - the results demonstrated a poor quality of fit. This could have been improved using a variety of methods, but this was not the purpose of the exercise. Surprisingly, though, we found the quantile results to be very accurate across the whole distribution. In particular, the error at the 99.5th percentile was less than 0.3%. This is demonstrated in Figure 1 and Figure 2, which show the errors and error percentages for the ranked results (orange) compared with the ranked errors of the individual scenario results (blue).
You will see from the diagrams why we refer to this as a spooky result. The question is, how can the scenario results be so inaccurate and yet the quantile results so accurate?
What's going on?
We will try to explain this ghostly outcome, but, first, let's take a step back - or three.
It is tempting to assume that proxy model errors increase as scenarios become more extreme. This is perhaps based on our experience of approximations in one risk dimension. Figure 3 shows the error curve resulting from just such an approximation.
Beyond a certain point there are no further turning points, and errors continue to increase in magnitude. Drawing quantile results in the tails leads to increasingly large errors as the scenario being considered becomes more extreme.
In fact, this occurs not because of the extremity of the event but because of error bias in the region of interest. In this region, all errors are of one sign and for any particular error there is only one corresponding value of the risk. Therefore, the error in the quantile result is dictated by the error in that scenario result.
Multiple dimensions and the curve of constant loss sounds like an episode from
Dr Who, but really isn't that strange. In multiple dimensions - in other words, where we have a number of risk variables - the single point is replaced by the 'curve of constant loss', representing the combinations of different risk variable values that all give the same result. For example, if equity values drop by 10% and lapse rates increase by 5%, we might get the same answer as if equity values rise by 5% and lapse rates fall by 10%, and in this case these two scenarios would be on the same curve. If we could plot the errors along the path of a curve of constant loss, we would not expect the errors to be of one sign, with the proxy being greater or less than actual at different points along the path, as Figure 4 illustrates.
At specific points, the errors can be large, up to 60% in our example. However, the nature of least-squares regression, which does a good job of minimising average error, removes some of the error bias along the path of constant loss. This is by no means guaranteed though.
Our aim now is to minimise error bias along the curve of constant loss. More formally, we wish to have an expectation of error equal to zero, and we believe - while still subject to rigorous proof - that under these circumstances, and a few other conditions, our result about quantile accuracy will hold.
Returning to our example, comparing ranked results of actual, based on our cashflow plus Black-Scholes model, versus proxy, based on our polynomial formulae, we find that the scenario numbers in each of the two lists of results do not match. Not only is the biting scenario different, but almost all scenarios are different. In reality, we are unlikely to have the actual quantile result so may wish to check our proxy result by running the implied biting scenario through the actual model and the proxy model.
From here on, the inaccuracy at the biting scenario can dominate proceedings if we are not careful. In particular, it may be concluded that the correct result is £404m, significantly understating the actual result of £441m, a result that the proxy has underestimated by a mere 0.3% at £440m.
It appears then that using an inaccurate model to determine a 'biting scenario' that is then subjected to more detailed analysis might not be appropriate. While we do not have a mathematical proof as yet, the above results certainly indicate that the capital measured by a proxy might be relatively accurate even if the scenario producing it is not. It should also be considered just how much value there is in analysing a single biting scenario.
Perhaps the proxy model has already provided the correct capital result and, from a risk management perspective, we should be deriving a range of biting scenarios along the curve of constant loss in our primary models, as individual scenarios might be misleading. The Proxy Models Working Group is on the case... watch this space.
Chris Hursey is chair of The Proxy Models Working Group of the Institute and Faculty of Actuaries. The group welcomes any comments or suggestions. Please contact Chris by email at [email protected]
