Silvana Pesenti, Alberto Bettini, Pietro Millossovich and Andreas Tsanakas present their alternative approach to sensitivity analysis
As insurance businesses have become increasingly complex, so have the quantitative models used to describe them. Sensitivity analysis encompasses an array of techniques for generating insights into models, stressing assumptions and determining what the key model inputs are.
Such analyses, while crucial for model validation, can be computationally demanding. For simulation models like those used in capital modelling, sensitivity analysis entails generating repeated sets of simulations - a process that can be frustratingly time consuming. Moreover, a robust way of interpreting the results of sensitivity analyses is sometimes lacking. Consequently, the fruits of modellers' labours may end up languishing in an appendix, rather than generating value for their organisations.
We present an alternative approach to sensitivity analysis called Scenario Weights for Importance Measurement (SWIM), based on our academic research ('Reverse sensitivity testing: What does it take to break the model?' - available at bit.ly/2RZppuU). SWIM is implemented as a package in the R language and is freely available from the CRAN repository. SWIM quantifies the impact of distorting a random variable in the model (such as a risk driver) on other variables of interest (such as line-of-business losses, portfolio profit and loss). A benefit of our approach is that it works on a single set of simulated scenarios; no further simulations are needed, cutting the need for costly repeated evaluations of the model. In particular, SWIM is well suited for reverse stress testing. Furthermore, SWIM provides visualisations and metrics that help generate insight from sensitivity tests.
An introductory example
Before getting into how SWIM works, we provide a simple illustration based on a credit risk example. The model represents a loan portfolio with systematic and idiosyncratic risk, consisting of three correlated sub-portfolios. The portfolio loss is represented by L , such that L = L1+L2+L3 , where L1,L2,L3 are sub-portfolio losses (for example, in thousands of £). We have 100,000 simulations from the model, available as an example data set in the SWIM package. Summary statistics for the losses are given in Table 1.
Now, we introduce a stress to our baseline model in order to quantify how changes in assumptions impact the portfolio. Specifically, we require that the mean of L2, the second sub-portfolio, is increased by 20%, from 454 to 545.
Subsequently, using SWIM we can evaluate the impact of this change on the distribution of the portfolio loss. Figure 1 shows samples of the portfolio loss L against the sub-portfolio L2. The high positive dependence is apparent, indicating that the stress would be impactful at portfolio level. This is confirmed by Figure 2, where the values at risk (VaR) for L are shown for the baseline and stressed models.
How does SWIM work?
How can distributions be changed without re-running the simulation model? The SWIM methodology can be summarised as follows.
The starting point is a table of simulated scenarios, each column containing outcomes of a different random variable. This table forms the dataset on which SWIM bases its calculations.
A stress defines a particular modification of one or more variables. This can be a change in means and standard deviations of risk factors, in the probabilities of specific events (such as net asset value lower than a critical level), or of portfolio risk measures, such as VaR or tail value at risk (TVaR). SWIM implements these options directly and offers the possibility of designing customised stresses.
Then, SWIM calculates a set of scenario weights, acting upon the simulated scenarios. Scenario weights are derived such that the defined stress is fulfilled while keeping distortion to the baseline model to a minimum (as measured by relative entropy). This captures the essence of the SWIM approach: rather than simulating new scenarios, the probabilities of already existing scenarios are modified, impacting the distribution of all variables in the model.
Finally, with the calculated scenario weights, the impact of the stress on all model components is worked out. SWIM carries out these calculations, providing and plotting distributions and key metrics (such as VaR and TVaR) for any variable of interest.
The SWIM approach works whatever the model structure may be, including the case of non-linear aggregations - there is actually no need for the portfolio structure to be explicitly known to the user. The only limitation is that the existing scenario set contains situations relevant to the stresses applied (for example, one cannot ask for the mean of a variable to be higher than the observed maximum simulation).
Reverse stress testing
Let us now consider a reverse stress test. We study the effect that changing (the tail of) the aggregate portfolio loss has on the three sub-portfolios. In this way, we seek to assess their comparative importance. In the language of Solvency II internal models, such a stress corresponds to the question: what would the likely SCR for individual sub-portfolios be if the total SCR were to increase by, say, 20%? (Of course, one can also ask the alternative question: which sub-portfolios could drive a reduction in the total SRC?) Questions such as this can be answered with SWIM, which works out likely changes in the distributions of all variables, along with the change in the total loss distribution.
In our current example, we carry out two stresses: first, a 20% increase on the portfolio VaR at level 90% ('stress 1'); second, a 20% increase in the VaR and a 50% increase in TVaR, at the same confidence level ('stress 2'). Figure 3 displays the histogram of the portfolio loss under the baseline and the two stressed models. It is seen how stressing VaR and TVaR has a higher impact on the right tail of L, compared to stressing VaR only. This is consistent with the tail-sensitive nature of TVaR.
Next, we analyse the impact that stressing the aggregate loss has on the sub-portfolios L1, L2 and L3 by plotting their cumulative distribution functions (cdfs) in Figure 4. All stressed cdfs are below the corresponding baseline ones, indicating an increase in probability of high losses. It is seen that the stresses have no substantial impact on L1, while L2 and L3 are more affected, indicating a higher sensitivity to those two variables.
It is helpful to have a numerical measure of comparative sensitivity of sub-portfolios, so as not to rely on visual tools only - this is particularly important when considering large models with many variables of interest. SWIM implements various sensitivity metrics. Here we focus on the 'Gamma' measure, which we introduced and applied to a London market portfolio (bit.ly/2RZppuU). That measure calculates the normalised change in the mean of variables, under the applied stresses. Given the flexibility offered by SWIM, it is not hard to generate variants of that metric, for example to measure the impact of stresses on the tails, rather than the means, of distributions.
The values of the Gamma sensitivity measure are reported in Table 2. Consistently with what the distribution plots of Figure 4 showed, L2 emerges as the most important sub-portfolio, followed by L3 and L1, when the portfolio VaR and TVaR are stressed.
We now carry out an additional piece of analysis, as an example of a more complex customised stress. We suspect that the high importance of L2 is specifically due to the impact of a risk driver, representing random default probabilities for a set of loans. We now repeat the stress on the portfolio VaR, but this time fix the mean and 75th percentile of the risk driver to its baseline values ('stress 3'). This means that, while the portfolio loss is stressed, the impact of this stress on L2 via the risk driver is limited. It is seen in Table 2 that this stress test drastically reduces the sensitivity of L2, thereby demonstrating the risk driver's importance.
A range of applications
SWIM is a free and powerful tool for sensitivity analysis, which can be used to analyse the simulation output of models used by insurers. It offers both standard choices for stressing models and the ability to design custom stresses and sensitivity measures. Further examples and details about SWIM are provided in our R vignette 'Scenario Weights for Importance Measurement (SWIM) - an R package for sensitivity analysis' (bit.ly/2RDkhxf).
We can see a wide range of practical applications in insurance. Whenever insurers generate simulations from multivariate risk factors and corresponding losses, users could feed these as inputs into SWIM and ask questions such as:
What would happen to the total SCR if the mean and/or volatility of a risk driver (such as longevity risk or natural catastrophe loss) was increased by 20%? If the total SCR became 30% larger due to growth of part of the business, what would the likely SCR be for, say, credit risk? What would the impact on SCR be if joint extremes of risk factors occurred more frequently than the model currently predicts?
SWIM can provide answers to these questions in a consistent and numerically efficient way, generating insight into the dynamics of the business, as well as into the model itself, as part of a streamlined validation process.
We look forward to collaborating with insurance practitioners in order to explore the potential of SWIM and help address some of the challenges faced by actuaries and insurance organisations.
Dr Silvana Pesenti is an assistant professor in insurance risk management at the University of Toronto, Canada.
Alberto Bettini is an actuary at Assicurazioni Generali S.p.A.
Dr Pietro Millossovich is a senior lecturer in actuarial science at Cass Business School, City, University of London.
Professor Andreas Tsanakas is a professor of actuarial science at Cass Business School, City, University of London.