The new regulatory framework requires firms who develop their own internal capital models to have these validated. William Diffey and Apollos Dan look at the key challenges, and suggest the best route for a good test run
Model validation and quality control are key issues for companies seeking either Solvency II regulatory approval or business acceptance of their internal capital models. Internal model validation should aim to provide management with the necessary understanding of the internal model Solvency Capital Requirement (SCR) and provide an assessment of the model's fitness for purpose. Validation provides independent challenge to the model and assists internal management and regulators alike in appraising it. This involves assessing the scope, inputs, outputs and methods used. Validation processes should be embedded, with continuously updateable validation reports produced for management and regulators where required.
Validation reports do not have a prescribed format, and may differ from one company to the next. However, the key theme of any validation report is to present the company's model and demonstrate that it has been independently challenged and thoroughly tested. It should give a validation opinion on the model that is supported and substantiated by test results and other supporting evidence.
A number of key questions being increasingly asked by firms include:
? What is the precise division of responsibilities between first and second 'lines of defence' within the organisation?
? How can good second-line validation be resourced, given expense constraints?
? Can external validation be a substitute for internal validation, or alternatively be used to complement internal validation processes?
Validation is an iterative process repeated at least once each year, and involves challenging the model across multiple dimensions:
Theoretical - verify that mathematical, actuarial, financial and business concepts supporting the inner model assumptions and methodologies are appropriate and correctly implemented.
Applied - ensure that inputs, computations, outputs and first-line testing are satisfactory.
Embeddedness - verify that governance and documentation of the model are appropriate. Also that management and key stakeholders have the relevant training to understand the model and its results, especially understanding the limitations.
Good validation requirements
A good validation should be independent, thorough and challenging. In the European context, it requires clear understanding of Solvency II regulation, together with practical insurance business experience and technical actuarial knowledge, as well as good communication and other soft skills. Understanding regulators' expectations is key to any validation exercise.
Validation applies quantitative and qualitative methods to assess a number of aspects of the internal model, as illustrated in Figure 1.
? Quality of inputs - internal and external data, balance sheets.
? Appropriateness of methodological choices - commonly accepted approaches, simplifications, best practice, innovations and expert judgment.
? Consistency with other processes and models - technical provisions, pricing and underwriting, planning, investment.
? Soundness and sensitivity of outputs - for SCR, Own Risk and Solvency Assessment (ORSA), risk appetite and metrics, capital allocation.
? Appropriateness of model structure and design - purpose, coverage, building blocks, dependencies, flexibility, IT tools.
? Mastering of model environment - reporting, documentation, governance, training.
The key areas of assessment are highlighted in Table 1.
Quality control
A parallel can be made between insurance internal model validation and quality control processes in the motor industry. This reveals common gaps in model validation practice in typical insurers. It also brings the model validation and optimisation processes closer to quality and process management in manufacturing and operational/IT industries. The following list is not exhaustive.
Model scope and design
Automobile production starts with a design encompassing artistic/aesthetic concepts, sophisticated mathematics and engineering. Prototype manufacture is required to undergo extensive testing before launch and mass production.
An internal model goes through design and initial build phases, for example, parallel or proxy modelling. In these, its theoretical foundations and practical application are tested before deployment.
Flexibility and use
A relatively flexible car make/model can be used to transport some light-to-medium-weight loads with some minor adjustments. However, it cannot exceed its maximum agreed tolerances.
Likewise, the internal model can be made flexible for multiple purposes, and defining the limits on this flexibility is a key feature of model governance frameworks. Some of the adjustments might be fairly easy to do, while some others might need significant re-calibration. For example, the calibration of an internal model designed to be used for calculating regulatory capital will focus on the tails. However, if the use of that model is to be extended to, say, pricing, it might need some adjustments because pricing generally focuses on the expected loss.
Regular validation
While a car make and model remains in circulation, it undergoes regular full or partial tests. Some are mandated by legal requirement, for example, the MOT in the UK. Others are mandated by the manufacturers themselves to demonstrate the quality of their product. Some tests are mandated by the drivers themselves.
An internal model undergoes an initial pre-launch validation, followed by post-launch regular periodic or ad hoc specific validation exercises. Some aspects of the internal model validation are mandatory, for example, the Solvency II directive asks for specific validation tests. Management and the wider user group of the model can, however, ask for additional validation tests to be performed, to reinforce the model's soundness and fitness for purpose.
Scalability and infrastructure
A car, regardless of build quality, needs a good road infrastructure to meet optimum performance standards.
Likewise, an internal model needs an appropriate IT infrastructure to house it and run it in a controlled environment to perform well. Most models run a high number of simulations and are run many times for different reasons. This requires appropriate computing power, such as cloud-based solutions, which should be made available for the model to run smoothly.
Robustness
Motoring risks are carefully controlled. Robust and reliable cars, with defined risk tolerances and appropriate quality control, give extra assurance to the user.
Likewise, companies with robust risk management frameworks and an appropriate, accurate and reliable internal model can better control their risks and capital management. They derive a competitive advantage from so doing.
Validation is a key aspect of the Solvency II internal model approval process with regulators, to ensure quality and get companies to a starting point where they can obtain genuine business benefit from their investment in these models. It also provides management with a deeper understanding of the model. Validation is mainly about challenging the model. However, value can be added through suggesting areas for model optimisation and enhancement.
Validation is about methodically turning every building block of the internal model, and challenging its theoretical and practical foundations. It is also about challenging its use within the regulatory context and within the wider ERM framework.
William Diffey is director of the GI business practice for SunGard, and leads SunGard's global GI services practice He has 20 years' experience in a variety of leadership roles across GI and Life, and across the corporate and consultancy sectors.
Apollos Dan is senior consultant at SunGard and Risk Dynamics with 20 years' experience. He specialises in model validation and ERM