Skip to main content
The Actuary: The magazine of the Institute and Faculty of Actuaries - return to the homepage Logo of The Actuary website
  • Search
  • Visit The Actuary Magazine on Facebook
  • Visit The Actuary Magazine on LinkedIn
  • Visit @TheActuaryMag on Twitter
Visit the website of the Institute and Faculty of Actuaries Logo of the Institute and Faculty of Actuaries

Main navigation

  • News
  • Features
    • General Features
    • Interviews
    • Students
    • Opinion
  • Topics
  • Knowledge
    • Business Skills
    • Careers
    • Events
    • Predictions by The Actuary
    • Whitepapers
    • Moody's - Climate Risk Insurers series
    • Webinars
    • Podcasts
  • Jobs
  • IFoA
    • CEO Comment
    • IFoA News
    • People & Social News
    • President Comment
  • Archive
Quick links:
  • Home
  • The Actuary Issues
  • August 2012
08

Testing your model

Open-access content Thursday 23rd August 2012 — updated 5.13pm, Wednesday 29th April 2020

Model development and upgrades cost time and money. Errors in insurance companies’ models can be costly, in terms of money and time, to remediate as well as result in harmful effects such as under or over reserving, financial losses, reporting errors or missed opportunities

Model development and upgrades cost time and money. Errors in insurance companies' models can be costly, in terms of money and time, to remediate as well as result in harmful effects such as under or over reserving, financial losses, reporting errors or missed opportunities. The importance of a clear development and testing process for model developments to be successful cannot be stressed enough
To ensure a successful model development a thorough test plan should be created, reviewed and signed off. The plan should contain a comprehensive test strategy which details the hierarchy and testing cycles to be carried out. The plan should also be realistic and allow sufficient time to conduct the tests with in-built contingencies, allowing for multiple cycles of testing.

The types of tests undertaken will depend on the budget available and the materiality of the model upgrade or development. Some of the tests that could be conducted in a model development are summarised in this article.

How well built is your model?


A number of tests can be employed to understand how well your model is built against the development requirements (functional and non-functional) which should be clearly defined at the outset, including:

  • Path testing to test if the model correctly performs the calculations across the various routes in the code.
  • Diagnostic tests to ensure that the code is efficient and does not cause undue bottlenecks when the model is run.
  • Integration tests where components are combined and tested as a group to ensure integrity of the code.
  • Performance tests to evaluate the performance of the model against specified performance requirements e.g. runtime, CPU efficiency, RAM usage, disk usage.

How robust is your model?

As well as testing that your model performs well against the requirements it is important that a model is built in a robust manner. There are a number of tests that can be conducted in this area, including:

  • Error-handling tests to test that model handles run failures in a correct manner e.g.  erroneous inputs by running through data that is expected to break the model.
  • Stress tests to test that the model can handle extreme values (small and large).
  • Destructive testing to deliberately try and break the model. For example, checking that User Access Control (UAC) is sufficient to prevent unauthorised users from making changes to the model or accessing parts they should not.

How accurate is your model?


When developing a model it is important that the model performs the calculations accurately as specified in the model specifications. A number of tests can be employed in this area, including:

  • Unit testing in which individual units of code are tested using sample data to determine if they are fit for use.
  • Regression testing which provides a thorough comparison of model output against previously signed-off results giving users confidence that developments have been implemented correctly and only have an impact on results where expected.

Does your model meet requirements of the end users?

Models should be tested to ensure that they meet the requirements of the users who will use the model. There are a number of ways this can be tested, including:

  • Scope testing to determine if a model upgrade or development embraces everything set out during the scoping/design phase.
  • Compatibility tests ensure that the development functions properly within production processes and hardware environments.
  • Usability tests ensure the end users can easily operate, prepare inputs and interpret outputs of the model.

Best practice guidelines for tests

There are a number of useful guidelines that can be followed for testing. These practices should be documented to ensure consistency across model developments. Example guidelines include:

  • Tests should be peer reviewed and signed off by the appropriate stakeholders.
  • Test team is separate from the design team.
  • Test suites are developed independently from the development team.
  • Procedures and processes when a test fails and how it feeds back into the development cycle.
  • Audit trails on testing conducted e.g. audit reports, compliance with any frameworks and standards that are in place.
  • Clear acceptance and sign-off processes for the development to move models from development to production.

Aditi Parekh is a qualified actuary, working as a senior consultant at Deloitte. Andy Crichton works in the Actuarial Modelling Centre (AMC) of Deloitte as a senior consultant. Marc Fakkel is a partner at Deloitte who leads the AMC at Deloitte.

This article appeared in our August 2012 issue of The Actuary .
Click here to view this issue

You may also be interested in...

The world of software: And then there was R

Alan Chalk uses R to visualise data in Google Earth
Friday 17th August 2012
Open-access content
ta filler

The environment: What are actuaries doing?

Therese Kieve considers actuarial efforts in planetary preservation
Thursday 23rd August 2012
Open-access content
ta filler

Modelling the 1-in-200 Risks

Tail risks of life catastrophe arise from extreme events, such as pandemics or terrorist attacks. Because such events are rare, recent mortality experience data doesn't help with estimating the risk from future extreme events.
Monday 20th August 2012
Open-access content
ta filler

Planet under pressure

As well as the Olympics, east London hosted another globally important event in 2012. This other event was much less reported in fact you might not have heard of it. A major scientific conference Planet Under Pressure was held in March with more than 3000 delegates
Monday 20th August 2012
Open-access content

Industry news round-up August 2012

Pensions/ Insurance/ Motor claims
Friday 10th August 2012
Open-access content
2

UK bulk annuity sales top £900m in second quarter

A total of £914m worth of bulk annuity deals were completed in the UK in a ‘relatively solid’ second quarter of 2012, Aon Hewitt said today.
Monday 13th August 2012
Open-access content

Latest from Professional

dxf

Bespoke tailoring: the UK government’s proposed reforms to Solvency II

Vrishti Goel discusses the UK government’s proposed reforms to Solvency II
Wednesday 31st August 2022
Open-access content
Financial services seen as most desirable sector for career changers

Financial services seen as most desirable sector for career changers

Over a third of UK workers will look to start a new career in the next year due to the cost-of-living crisis, with financial services seen as the joint most desirable sector, KPMG research suggests.
Wednesday 24th August 2022
Open-access content
web_p43_Student_student_july_2_CREDIT_Simon-Scarsbrook.jpg

Three’s a charm: the next iteration of the internet

Adeetya Tantia explains what we can expect from the next iteration of the internet, and how it could shake up the insurance industry
Wednesday 6th July 2022
Open-access content

Latest from August 2012

ta filler

Omnibus II vote delay raises concerns over Solvency II timetable

A European Parliament vote on the legislation which underpins new rules governing Europe's insurance industry has been put back by a month, raising concerns over the timetable for implementing the system.
Friday 31st August 2012
Open-access content
ta

Ill health 'is stopping 500,000 people near retirement from working'

Nearly half a million people approaching retirement are being prevented from working by disability or poor health, the Trades Union Congress said today.
Friday 31st August 2012
Open-access content

Enhanced annuity sales break £1bn barrier for first time

Quarterly sales of enhanced annuities exceeded £1bn for the first time between April and June 2012, according to figures published by Towers Watson today.
Thursday 30th August 2012
Open-access content

Latest from formbuilder_item_removed

2

Implementing IFRS 17 Discount Curves: Theoretical and Practical Challenges

The International Financial Reporting Standard (IFRS) 17 requires liability cash flows to be discounted at rates that reflect the characteristics of the cash flows, including their liquidity
Tuesday 3rd September 2019
Open-access content
2

Profit Emergence Under IFRS 9 and IFRS 17: The impact of choice of liability discount rate

With the IFRS 17 accounting standard, insurers need to understand the patterns of profit emergence that arise under the standard, and how current business and methodology decisions affect such patterns.
Wednesday 10th July 2019
Open-access content
2

Whitepaper: Aggregation and diversification of the IFRS 17 Risk Adjustment

This paper forms part of a series of high-level papers designed to provide an introduction to different features of the risk adjustment that should be considered in advance of implementation.
Tuesday 29th January 2019
Open-access content

Latest from no_opening_image

TPR publishes coronavirus guidance

The Pensions Regulator (TPR) has published guidance to help UK pension trustees, employers and administrators deal with the financial and regulatory risks posed by coronavirus.
Monday 23rd March 2020
Open-access content
web_p24_cat-and-fish_iStock-483454069.png

Sensitivity analysis: swimming lessons

Silvana Pesenti, Alberto Bettini, Pietro Millossovich and Andreas Tsanakas present their alternative approach to sensitivity analysis
Wednesday 4th March 2020
Open-access content
ta

IFoA adjudication panel: Mr Jack Wicks, student

On 30 October 2019 the Adjudication Panel considered an allegation of misconduct against Mr Jack Wicks (the respondent).
Friday 28th February 2020
Open-access content

Latest from missing_standfirst

news in brief

March news in brief

Paper: A Cashless Society in 2019 Cash is under pressure. A tense 2018 led to a tumultuous 2019: Facebook's announcement of plans to launch its Libra cryptocurrency with a consortium of companies united all regulators against the project.
Friday 28th February 2020
Open-access content
2

Forging new paths

The Terminator is coming!  At least that's one potential vision of the future, invoked by Boris Johnson at the UN last year while he speculated about artificial intelligence (AI). We can certainly debate how realistic that vision is, and what the possible timescales might be.
Friday 28th February 2020
Open-access content
2

Expert advice

This edition of the magazine focuses on data science and its applications, which will be a recurring theme for the IFoA.
Friday 28th February 2020
Open-access content

Latest from 08

2

Mindset for success

In the first of a series of articles, Jeremy Lazarus provides an overview of neuro-linguistic programming, and how it can help actuaries achieve better results, faster
Thursday 2nd August 2012
Open-access content
2

Clear vision

David Leach examines why life insurers are investing so heavily to boost their capital projections capability
Wednesday 1st August 2012
Open-access content
Share
  • Twitter
  • Facebook
  • Linked in
  • Mail
  • Print

Latest Jobs

Senior Pricing Associate

Scotland / England, London
Up to £60000 per annum
Reference
149081

Outside IR35 - Reserving Contract - 6-8 months

London (Central)
Daily rate contract - outside IR35
Reference
149079

Actuarial Analyst - Longevity Reinsurance

England, London
Up to £55000 per annum
Reference
149080
See all jobs »
 
 
 
 

Sign up to our newsletter

News, jobs and updates

Sign up

Subscribe to The Actuary

Receive the print edition straight to your door

Subscribe
Spread-iPad-slantB-june.png

Topics

  • Data Science
  • Investment
  • Risk & ERM
  • Pensions
  • Environment
  • Soft skills
  • General Insurance
  • Regulation Standards
  • Health care
  • Technology
  • Reinsurance
  • Global
  • Life insurance
​
FOLLOW US
The Actuary on LinkedIn
@TheActuaryMag on Twitter
Facebook: The Actuary Magazine
CONTACT US
The Actuary
Tel: (+44) 020 7880 6200
​

IFoA

About IFoA
Become an actuary
IFoA Events
About membership

Information

Privacy Policy
Terms & Conditions
Cookie Policy
Think Green

Get in touch

Contact us
Advertise with us
Subscribe to The Actuary Magazine
Contribute

The Actuary Jobs

Actuarial job search
Pensions jobs
General insurance jobs
Solvency II jobs

© 2023 The Actuary. The Actuary is published on behalf of the Institute and Faculty of Actuaries by Redactive Publishing Limited. All rights reserved. Reproduction of any part is not allowed without written permission.

Redactive Media Group Ltd, 71-75 Shelton Street, London WC2H 9JQ