Skip to main content
The Actuary: The magazine of the Institute and Faculty of Actuaries - return to the homepage Logo of The Actuary website
  • Search
  • Visit The Actuary Magazine on Facebook
  • Visit The Actuary Magazine on LinkedIn
  • Visit @TheActuaryMag on Twitter
Visit the website of the Institute and Faculty of Actuaries Logo of the Institute and Faculty of Actuaries

Main navigation

  • News
  • Features
    • General Features
    • Interviews
    • Students
    • Opinion
  • Topics
  • Knowledge
    • Business Skills
    • Careers
    • Events
    • Predictions by The Actuary
    • Whitepapers
    • Webinars
    • Podcasts
  • Jobs
  • IFoA
    • CEO Comment
    • IFoA News
    • People & Social News
    • President Comment
  • Archive
Quick links:
  • Home
  • The Actuary Issues
  • December 2021
General Features

Unfamiliar territory for artificial intelligence-related risk

Open-access content Wednesday 1st December 2021
Authors
Andrew Morgan
Valerie du Preez
Natasha Naidoo

Artificial intelligence-related risk differs from traditional model risk, explain Andrew Morgan, Valerie du Preez and Natasha Naidoo – how can we get to grips with it?

Unfamiliar territory for artifical intelligence-related risk

Many insurance organisations are investing heavily in artificial intelligence (AI) and machine learning experiments – but, surprisingly, many pilot projects are struggling to gain traction and scale across organisations. One of the reasons for this is the lack of investment in the risk management of AI, whose model outputs often feed into a variety of processes and decisions that can have significant impacts on customer outcomes. This is compounded by the expectation of forthcoming regulations in this space, such as the EU’s proposed AI Act (bit.ly/EU_AILaw).

Insurance risk teams are experienced at identifying model risk, and most have resources and frameworks in place to deal with traditional (insurance risk-focused) models. However, the key risks that could arise through the use of AI are likely to be reputational, conduct-related and regulatory, rather than the more quantifiable risks that have historically been the focus of insurance risk and model validation frameworks.

AI risk is different from traditional model risk for several reasons:

  • AI models may use complex non-linear methods that can be hard to interpret or explain

  • AI models may use varied ‘high velocity’ data and predictions (such as real-time indicators of customer behaviour) that can make decisions dynamically; this differs from typical traditional models, which make use of more ‘static’ data

  • The underlying infrastructure, tools and code libraries required to run and deploy AI models may be new and foreign to traditional insurance organisations and their existing IT or infrastructure capabilities

  • The nature of data has changed – from data that is structured and easily validated, to data that has been taken from different sources and may be of different types. This may mean they could be difficult to reconcile.

Existing insurance model validation frameworks may not be equipped for measuring and managing specific AI-related risks because:

  • Model validation teams tend to adopt risk-based approaches in which different modules are assessed with a frequency based on a combination of materiality and available resources. While this may be suited to static models, it could be hard to adapt to AI models, whose modelling parameters are not necessarily fixed

  • Model validation has traditionally focused on measurable, quantifiable risks such as market risks – but non-quantifiable risks such as conduct and reputational risk may be more important for AI models. An example is unintentional bias in an underwriting model

  • Tools such as stress and sensitivity testing typically identify the key model risk drivers, helping model validation teams to interpret and explain the applicability of model results. With more complex AI models, such tools may not be sufficient to validate if model output is aligned to expectations

  • The scope of AI models in use that require validation may not be straightforward to assess and include within the framework, due to the wide variety of domains and use cases – for example, chatbots are increasingly complex, but existing validation frameworks often do not cater for them

  • The specific IT and data science skills needed to better measure and mitigate these risks may be lacking in model validation teams, which tend to be weighted towards traditional actuarial or risk management skills.

In practice

To make these abstract issues more practical, we consider the potential implications for model risk management in the context of policy cancellation (lapse) experience investigations.

Traditionally, actuaries have attempted to understand lapse experience for groups of policyholders with particular characteristics when performing their regular experience analysis investigations, often averaging experience across certain groups. These traditional modelling approaches have mostly used linear models, and are typically less automated.

With new data sources, modelling techniques and better infrastructure available, the experience analysis team can now enhance their processes and analyses to understand and manage the risk they face from lapses in a different way – for example by incorporating advanced machine learning and AI. This could bring significant improvements (such as speed, improved predictions and cost effectiveness) over traditional modelling approaches. Managing lapses can also be more granular and forward-looking, and feed into front-office processes such as marketing or retention strategies to improve future lapse rates.

New data sources, such as those related to customer interactions with an insurance organisation via an online platform, may require licensing permissions to be recorded and used for such a study, and must be compliant with internal practices and legal frameworks, such as the General Data Protection Regulations. Manipulating these vast datasets would require new data skills, moving away from spreadsheet analysis and towards dedicated programming languages such as R and Python.

Furthermore, the data needs to be in the optimal format for each machine learning model. For example, when building a lapse model, multiple models may be fitted and chosen based on run-time, cost, effort, frequency of use, accuracy and expert knowledge (such as multiple linear regression, classification and regression tree, random forest and gradient boosting machine models). The ability to assess the fit of each model, and measure the goodness of fit, will require a specialist skillset and experience with error measures in non-traditional applied statistics.

In addition, the ability to constantly monitor the performance of these models in light of changing data and interactions within the data will be essential.

Communication and interpretability of output and limitations will be key.

In conclusion, an updated framework, within which the insurance organisation must manage any additional risks, may be required. Having the right team with the right skillset to deal with the emerging risks, tools and techniques will be important for the second line, as well as ensuring there are internal practices, policies and frameworks that can help to continuously monitor this risk.


Tips for the risk function

While the industry is still grappling with how to close the gap between traditional modelling and the new risks introduced by implementing AI, we have seen organisations consider the following in their development – and think about how risk teams could be part of the adoption of AI.

  • Review the team’s skills and knowledge. While a solid actuarial base is essential, consider bringing in skills from wider industries, such as software engineering or data science skills, to provide a more holistic skillset. Also consider a strategy for any training programmes, to avoid getting lost in the vast online resources available.

  • Change the mindset. Have a five-to-10-year view for modernising the business, rather than being reactive to immediate changes. This includes thinking about what senior management and the board need to know today so they can prepare for the future.

  • Consider advanced tooling to help the risk manager to assess AI that comes with complexity that necessitates a different approach. Analytical tools that help with understanding specific risks such as bias, explainability or model performance measures will be essential.

  • Keep abreast of global regulatory developments in this space, as well as professional bodies that have been debating AI risk for some time from differing perspectives, such as the CFA Institute (bit.ly/CFA_AI-ethics) and the Institute of Electrical and Electronics Engineers (ethicsinaction.ieee.org).

  • Start designing the components of an AI risk framework that goes beyond what is currently in place. This includes approaches and processes that revolve around model inventories, risk score-carding, independent model assessments, business continuity, and AI-related key risk indicators or key performance indicators that will also include live robustness indicators, as well as ownership and accountability for the models and decisions used in the business.


Andrew Morgan specialises in innovation, data science and risk management at Deloitte

Valerie du Preez is a senior consulting actuary and founder of Actuartech

Natasha Naidoo is chief risk officer at Generali UK

Image Credit | IKON

ACT Dec21_Full LR.jpg
This article appeared in our December 2021 issue of The Actuary.
Click here to view this issue
Filed in:
General Features
Topics:
Data Science
Risk & ERM
General Insurance

You might also like...

Share
  • Twitter
  • Facebook
  • Linked in
  • Mail
  • Print

Latest Jobs

Senior Underwriting Risk Manager

London (Central)
£85K-£95K + Benefits
Reference
124386

Reserving Manager (Contract)

London (Central)
£1200 - £1400 per day
Reference
124385

Life Actuary - Contract - IFRS 17 Financial Impact

England, London / England, Bristol / North Yorkshire, England
£900 - £1150 per day
Reference
124384
See all jobs »
 
 

Today's top reads

 
 

Sign up to our newsletter

News, jobs and updates

Sign up

Subscribe to The Actuary

Receive the print edition straight to your door

Subscribe
Spread-iPad-slantB-june.png

Topics

  • Data Science
  • Investment
  • Risk & ERM
  • Pensions
  • Environment
  • Soft skills
  • General Insurance
  • Regulation Standards
  • Health care
  • Technology
  • Reinsurance
  • Global
  • Life insurance
​
FOLLOW US
The Actuary on LinkedIn
@TheActuaryMag on Twitter
Facebook: The Actuary Magazine
CONTACT US
The Actuary
Tel: (+44) 020 7880 6200
​

IFoA

About IFoA
Become an actuary
IFoA Events
About membership

Information

Privacy Policy
Terms & Conditions
Cookie Policy
Think Green

Get in touch

Contact us
Advertise with us
Subscribe to The Actuary Magazine
Contribute

The Actuary Jobs

Actuarial job search
Pensions jobs
General insurance jobs
Solvency II jobs

© 2022 The Actuary. The Actuary is published on behalf of the Institute and Faculty of Actuaries by Redactive Publishing Limited. All rights reserved. Reproduction of any part is not allowed without written permission.

Redactive Media Group Ltd, 71-75 Shelton Street, London WC2H 9JQ