Skip to main content
The Actuary: The magazine of the Institute and Faculty of Actuaries - return to the homepage Logo of The Actuary website
  • Search
  • Visit The Actuary Magazine on Facebook
  • Visit The Actuary Magazine on LinkedIn
  • Visit @TheActuaryMag on Twitter
Visit the website of the Institute and Faculty of Actuaries Logo of the Institute and Faculty of Actuaries

Main navigation

  • News
  • Features
    • General Features
    • Interviews
    • Students
    • Opinion
  • Topics
  • Knowledge
    • Business Skills
    • Careers
    • Events
    • Predictions by The Actuary
    • Whitepapers
    • Moody's - Climate Risk Insurers series
    • Webinars
    • Podcasts
  • Jobs
  • IFoA
    • CEO Comment
    • IFoA News
    • People & Social News
    • President Comment
  • Archive
Quick links:
  • Home
  • The Actuary Issues
  • August 2017
08

Perfecting proxy models

Open-access content Tuesday 8th August 2017 — updated 5.50pm, Wednesday 29th April 2020

Shaun Lazzari and Oliver Bentley look at managing sources of inaccuracy in Least Squares Monte Carlo proxy model fitting

2

—


In recent years, proxy modelling techniques have become crucial components of life insurers' capital models, with such models being used to provide approximations to values of liabilities under different risk scenarios, which then are used in calculating capital requirements. Proxy models need to be 'fitted' to the actual liability model, and a particularly powerful approach to this is the Least Squares Monte Carlo (LSMC) technique. The benefits of LSMC have been well-documented, yet there has been less discussion on the different sources of inaccuracy, or 'error', that may be embedded in the fitted proxy model. Understanding these errors and how each can be quantified and mitigated is a key part of the proxy modelling process.


What is LSMC?

Life insurance liabilities often contain embedded options and guarantees, meaning their valuation requires the use of a stochastic cashflow model. The LSMC technique involves performing such valuations under thousands of different 'outer' risk factor scenarios, but for each stochastic valuation using only a few 'inner' or 'nested' economic scenario simulations instead of the thousands which would typically be used with the cashflow model. This results in a liability value for each outer scenario, which is individually very inaccurate. However, an accurate proxy model may be found by using regression techniques to fit a function through these noisy values.

For a broader summary of LSMC, see the article by Robinson and Elliot in The Actuary (April 2014).


What errors can arise? 

Sampling error

Sampling error can arise under LSMC in two forms. Firstly, note that a valuation produced by the liability cashflow model is the average of a set of discounted cashflow values based on different inner scenarios paths. This means that sampling error will arise when only a small number of inner simulations are used for a valuation. Secondly, additional sampling error may occur if a randomised method is used for selecting the outer scenario set.

A bootstrapping approach can be used to estimate the size of the sampling error: This involves re-fitting the proxy model to randomly selected subsets of the fitting data, thus producing confidence intervals for the proxy model's value under any given risk scenario. This can help in understanding how the uncertainty of the fitted proxy model varies across scenarios, and indicate what scenarios would be most useful to incorporate in future fitting exercises.


Figure 1


Spanning error

Spanning error occurs when the form of proxy model that is chosen to be fitted is not capable of fully capturing the shape of the liability profile. For example, you could find that no quadratic polynomial can adequately fit through your liabilities, yet a cubic or some non-polynomial function could come closer.

To date, identification and management of this type of error has been performed by visual inspection of fitting errors, combined with human understanding of the nature of the liabilities and lots of trial and error. A current area of research is the use of automated machine learning techniques to help identify the best building blocks of the proxy model, replacing subjective human judgments.

A related issue is overfitting; if too complex a function is used in this process, this could achieve a very good match to the fitting data but have limited power for predicting the value of the liabilities under other scenarios. This can be avoided by fitting the proxy model using goodness-of-fit metrics that penalise complexity such as the Akaike or Bayesian information criterion (AIC or BIC), and by measuring model performance with respect to an out-of-sample validation dataset.

Figure 2

Placement error

If the 'outer' risk factor scenarios were to be distributed in a different way, then the fitted proxy model would itself differ. We call the choosing of a sub-optimal set of outer scenarios placement error. This error can be particularly significant when the proxy model is sought to be a function of many risk factors, meaning that the outer scenario set must be more sparsely distributed throughout the space of possible scenarios.

This type of error can be reduced by studying the types of risk scenarios under which there is greatest uncertainty, found through the bootstrapping approaches mentioned earlier. Based on this, the placement of fitting scenarios can be modified in future fitting exercises to place more points in the areas of greatest uncertainty for the fitted model, which will have the effect of reducing this uncertainty in future fitted models.


Bifurcation error

For each outer scenario used in the LSMC technique, the corresponding set of inner simulations needs to be produced using an economic scenario generator (ESG) model. The ESG model must be calibrated to market conditions consistent with the outer scenario. It would be natural to expect that small differences between outer scenarios should give rise to small changes in liability values. However, because of the challenges in recalibrating ESG models, several model parameterisations may satisfy the model calibration targets. This could give rise to instability of the identified model parameters between outer scenarios and hence instability in the value of the liabilities. We call this instability of liability values bifurcation error, because it arises when the ESG model's parameter space may be divided into two (or more) distinct regions each providing similar fits to outer scenario calibration targets.

An example of this type of error is shown in figure 2: When the liabilities are evaluated using sets of (thousands of) scenarios from an ESG model each calibrated to different levels of interest rate volatility, bumps are seen in the relationship between liabilities and interest rate volatility for which it is unrealistic to fit a proxy model to precisely.

Bifurcation error can be detected through assessments of the stability of ESG model parameters, and by analysing how the ESG model behaves when used to evaluate 'simple' liabilities. This latter point is an example of using out-of-model testing - a technique that is well-adopted in many financial model contexts, yet generally absent from insurer's current processes.

The occurrence of this type of error can be reduced by enforcing parameter stability on the ESG model re-calibration approach, or by incorporating additional explanatory variables in the proxy model to capture dynamics of stressed ESG model.


Stress interpretation error

In some applications of LSMC, the values of risk factors assumed to be associated with an outer scenario may not precisely reflect the values of those risk factors under that scenario's ESG model. For example, a stress to a volatility risk factor's value may be applied by adjusting a particular parameter of the ESG model and assuming a linear relationship between this and the risk factor. When this assumption is invalid, the proxy model is fitted based on incorrect risk factor values. We call this stress interpretation error.

This form of error can in fact be avoided completely by ensuring that the properties of the ESG model are accurately reflected in the proxy model fitting process. In many cases that can be achieved through the use of closed-form solutions for translating model parameters to risk values, and when these solutions do not exist then approximations in the form of supporting proxy models may be used.

By their very nature, proxy model fitting techniques will produce some differences between the behaviour of the underlying cashflow model and the proxy model. We believe that insurers who use LSMC have scope to increase their understanding of the sources of such errors and how to manage these, leading to improved fitting and more accurate modelling of the balance sheet.


Shaun Lazzari
is a senior manager at L&G and a quantitative modeller

Oliver Bentley is an analyst in Deloitte's analytics & quantitative modelling team

This article appeared in our August 2017 issue of The Actuary .
Click here to view this issue

You may also be interested in...

2

Book review: Insurance in Elizabethan England

Insurance in Elizabethan England – The London Code Cambridge Studies in English Legal History
Tuesday 8th August 2017
Open-access content
2

Are you a pusher or a puller?

Ally Yates explains how to use the right strategy for the right situation when negotiating and influencing
Tuesday 8th August 2017
Open-access content
2

Fail to prepare

Fail to prepare
Tuesday 8th August 2017
Open-access content
2

A nation divided?

Are there really differences in life expectancy between Scotland and the rest of the UK? If so, how should actuaries take account of these? Phil Caine and Susan Hanlon discuss
Monday 7th August 2017
Open-access content
2

Is de-risking in members' best interests?

Traditional asset-liability modelling for defined benefit pension schemes ignores covenant risk. Integrating this risk may suggest that a higher allocation to return-seeking assets makes sense, say Robert Waugh, Nic Barnes and Robert Chestnutt
Monday 7th August 2017
Open-access content
2

Mortality improvements in decline

Jon Palin, on behalf of the CMI Mortality Projections Committee, reviews the growing evidence for a slowing down of mortality improvements in the UK, noting the trend for pension scheme members is less clear
Monday 7th August 2017
Open-access content
Filed in
08
Share
  • Twitter
  • Facebook
  • Linked in
  • Mail
  • Print

Latest Jobs

Senior Reserving Analyst

London (City of)
Negotiable
Reference
149485

Senior GI Modeler - Capital and Planning

London (Central)
£ excellent
Reference
149436

Risk Oversight Manager

Flexible / hybrid with a minimum of 2 days per week office-based
£ excellent
Reference
149435
See all jobs »
 
 

Today's top reads

 
 

Sign up to our newsletter

News, jobs and updates

Sign up

Subscribe to The Actuary

Receive the print edition straight to your door

Subscribe
Spread-iPad-slantB-june.png

Topics

  • Data Science
  • Investment
  • Risk & ERM
  • Pensions
  • Environment
  • Soft skills
  • General Insurance
  • Regulation Standards
  • Health care
  • Technology
  • Reinsurance
  • Global
  • Life insurance
​
FOLLOW US
The Actuary on LinkedIn
@TheActuaryMag on Twitter
Facebook: The Actuary Magazine
CONTACT US
The Actuary
Tel: (+44) 020 7880 6200
​

IFoA

About IFoA
Become an actuary
IFoA Events
About membership

Information

Privacy Policy
Terms & Conditions
Cookie Policy
Think Green

Get in touch

Contact us
Advertise with us
Subscribe to The Actuary Magazine
Contribute

The Actuary Jobs

Actuarial job search
Pensions jobs
General insurance jobs
Solvency II jobs

© 2023 The Actuary. The Actuary is published on behalf of the Institute and Faculty of Actuaries by Redactive Publishing Limited. All rights reserved. Reproduction of any part is not allowed without written permission.

Redactive Media Group Ltd, 71-75 Shelton Street, London WC2H 9JQ