Skip to main content
The Actuary: The magazine of the Institute and Faculty of Actuaries - return to the homepage Logo of The Actuary website
  • Search
  • Visit The Actuary Magazine on Facebook
  • Visit The Actuary Magazine on LinkedIn
  • Visit @TheActuaryMag on Twitter
Visit the website of the Institute and Faculty of Actuaries Logo of the Institute and Faculty of Actuaries

Main navigation

  • News
  • Features
    • General Features
    • Interviews
    • Students
    • Opinion
  • Topics
  • Knowledge
    • Business Skills
    • Careers
    • Events
    • Predictions by The Actuary
    • Whitepapers
    • Moody's - Climate Risk Insurers series
    • Webinars
    • Podcasts
  • Jobs
  • IFoA
    • CEO Comment
    • IFoA News
    • People & Social News
    • President Comment
  • Archive
Quick links:
  • Home
  • The Actuary Issues
  • July 2020
General Features

Model behaviour in the banking sector

Open-access content Wednesday 8th July 2020
Authors
Alan Forrest

Alan Forrest shares the importance of model risk management in the banking sector

web_p18-19_dog_shutterstock_688998025n.jpg


At the bank where I work, I manage the risk of an important high-exposure portfolio, but it is not a portfolio of loans or insurances, nor a trading book or pension fund. My portfolio is something completely different – but it is clearly a portfolio, an inventory of equally managed entities with a stochastic loss profile and correlated interactions. I am not a front office quant, nor a fund director; I am not even an actuary. I am in fact the head of the Model Risk Oversight team, and the portfolio I am talking about is a portfolio of models.

Every bank has hundreds, even thousands, of models, and uses them so it can operate compliantly, survive and compete. You can find models everywhere: sales targeting, customer account management, fraud detection, product pricing, portfolio management, credit provisions, capital management and stress testing, financial reporting and forecasts. These models inform bank decisions at all levels. A statistical regression defines the scorecard that approves credit loans; an estimated rate curve helps a committee set deposit prices; an econometric projection interacts with calibrated stress models whose estimate of unexpected loss informs the bank’s regulatory capital. Human experts make the decisions and sign off the reports, but in doing so they use and trust the outputs of models.

Do the bank’s models merit the trust of the decision-makers? What could possibly go wrong? What, in other words, is the risk of the model portfolio? Asking and addressing these questions is the job of the model risk manager.


A new risk category

Banks have increasingly come to rely on model risk teams such as mine. In their early years, such teams simply validated models independently as an assurance and compliance function. Model risk in banks truly took off as a separate category of risk when the 2008 global financial crisis played out and regulators pointed to the markets’ failure to understand and communicate model risk: in this case the model risk of overconfidence in collateralised debt obligation pricing (see Andrew Haldane’s 2012 speech ‘The dog and the frisbee’). It was a lesson well learned. From regulatory pressure and hard experience, banks have come to know that while models are needed to make the right decisions, their use comes at a risk – and every risk needs management.

To meet this need, model risk management in banks has grown strongly, forming its own networks, tools, experience and principles. However, it is not an insular profession. The US regulator’s wish to manage model risk “like other kinds of risk” [US Federal Reserve SR11-7] has opened the door to risk specialists of all kinds and had a positive influence on bank modelling culture.

“Model risk in banks took off as a separate category of risk when the global financial crisis played out”

Model risk is where modelling meets regulatory risk meets statistics meets operational risk meets stress testing: it is an essentially actuarial science. Model risk’s origin in validation puts the focus naturally on each individual model: how it performs and how it could behave under different conditions. Here, banking practice is strong and improving. Models are inventoried and model risks are classified and assessed. A growing body of research shows how to quantify model risk using ideas of sensitivity analysis and data shifts – see, for example, ‘SWIMming lessons’ by Silvana M Pesenti et al. in The Actuary, March 2020.

But to quantify the bank’s model risk, which is my job, I must aggregate the risk of the model portfolio, and this cannot be done simply by adding up the individual model risks. To illustrate this, I give two examples from my own experience. Misleading combinations The first example is about forecasting the credit risk of two loan portfolios, each with its own behaviour and market, and each requiring a model of how the economy influences the default rate. For one portfolio, say credit cards, a modeller picks out the strongest and simplest influences and finds that the default rate is driven by unemployment, interest rates and indexes of consumer confidence. For the other, a small business loan portfolio, another modeller finds that the default rate is driven also by unemployment, interest rates and variants for business confidence.

The result is a pair of independently built stress models, each optimal on its portfolio, but accidentally having many common factors that force large structural correlations between model outputs. Certainly, defaults in small businesses are correlated to credit card defaults, but the real-life observed correlation will not be as large as the combined models predict. This is an error of the pair of models, not to be found in either model singly, that could cause the bank to overestimate the impact of stress on its two portfolios combined.

The second example is about IFRS 9 regulation, which has defined how UK banks set credit risk provisions since 2018. This has increased the number and sophistication of statistical models used to determine expected credit loss (ECL). On principle, IFRS 9 seeks accurate estimates and unbiased forecasts of risk parameters at all points in time. Unlike Basel 2 IRB capital modelling, there is little room for conservatism and the damping effects of long-run average estimation. Rather, IFRS 9 relies on a rapid cycle of monitoring, correction and post-model adjustment to achieve point-in-time precision.

This requires many models, linked by intricate input-output dependencies and controlled by feedbacks of varying lags and strengths. In my experience, more than 20 models or modelled adjustments are needed to compute retail mortgage ECL, with six layers of dependency and three lags of feedback at weekly, monthly and annual scales. Even if the models were individually perfect, this complexity and feedback carries the risk of unexpected, rapid and uncontrolled effects. Intrinsically, ECL model risk is high because of its structure, and it is heightened paradoxically by the regulation’s insistence on accuracy.

The interactions between models are not limited to these examples, of course. One model’s output can be the input or adjuster for another; one model may have variants each used for different purposes; one model can be a loosely defined complex of smaller modelled parameters or assumptions. The weaknesses of one model may be amplified or corrected by its neighbouring models. A group of models can work well individually but collectively be misleading, or collectively be accurate but unusable individually for other purposes. All this carries us away from individual models and over to the unfamiliar problem of aggregating model risk.
 

Aggregating the model risk

The variety of model interactions possible shows that the aggregate model risk of a model portfolio is determined by more than a list of correlations – model risk is unlikely to have a simple value at risk (VaR) metric. Rather, we must specify or assume more about the joint risk distribution. However, the data will certainly be too few to give 
a representative sample of the high-dimensional space on which the full joint distribution or copula sits.

I do not have the answer to this problem, but any solution must combine the practical estimation of high-dimensional copulas with statistics that are robust against the model risks of high-dimensional copula fitting. It is almost as if we should join the scenario approaches of operational risk with the state functions of classical thermodynamics, an unlikely union of ideas that I leave hanging as a suggestion for further research.

To conclude, model risk is the risk of a portfolio of models and, as for other portfolios, it is more than the sum of the individual risks. For model risk there is no established approach to aggregation: neither classic VaR nor the low dimensions of operational risk scenario combination are likely to provide the answer. However, the strong, complex interactions between models drive model risk far beyond individual model assessment, and give strong motivation to develop new methods of risk aggregation.
 

Alan Forrest is head of model risk oversight at Virgin Money UK PLC

Picture Credit | Shutterstock

 

ACT Jul20_Full.jpg
This article appeared in our July 2020 issue of The Actuary.
Click here to view this issue
Filed in
General Features
Topics
Risk & ERM
Modelling/software
Finance

You might also like...

Share
  • Twitter
  • Facebook
  • Linked in
  • Mail
  • Print

Latest Jobs

New Fast-Growing Team - Actuarial Systems Development

London (Greater)
Excellent Salary Package
Reference
143762

Actuarial Pension Consultant – Scotland/Remote – Up to £90,000 plus bonus

Edinburgh / Glasgow / Remote working
Up to £90,000 + Bonus
Reference
143761

Part Qualified Pensions Actuary– Specialised Pensions Consultancy - Scotland/Remote - Up to £70,000

Edinburgh / Glasgow / Remote working
Up to £70,000 + Bonus
Reference
143760
See all jobs »
 
 

Today's top reads

 
 

Sign up to our newsletter

News, jobs and updates

Sign up

Subscribe to The Actuary

Receive the print edition straight to your door

Subscribe
Spread-iPad-slantB-june.png

Topics

  • Data Science
  • Investment
  • Risk & ERM
  • Pensions
  • Environment
  • Soft skills
  • General Insurance
  • Regulation Standards
  • Health care
  • Technology
  • Reinsurance
  • Global
  • Life insurance
​
FOLLOW US
The Actuary on LinkedIn
@TheActuaryMag on Twitter
Facebook: The Actuary Magazine
CONTACT US
The Actuary
Tel: (+44) 020 7880 6200
​

IFoA

About IFoA
Become an actuary
IFoA Events
About membership

Information

Privacy Policy
Terms & Conditions
Cookie Policy
Think Green

Get in touch

Contact us
Advertise with us
Subscribe to The Actuary Magazine
Contribute

The Actuary Jobs

Actuarial job search
Pensions jobs
General insurance jobs
Solvency II jobs

© 2023 The Actuary. The Actuary is published on behalf of the Institute and Faculty of Actuaries by Redactive Publishing Limited. All rights reserved. Reproduction of any part is not allowed without written permission.

Redactive Media Group Ltd, 71-75 Shelton Street, London WC2H 9JQ