Internal model documentation -  a quick glance

Jennifer Khaleghy and Ashish Kwatra break down the key documents involved in an IMAP application.

The breadth and quality of documents required for an IMAP has been one of the biggest challenges in transitioning from an ICAS regime. Many companies, even those with model approval, continue to explore ways of managing their documentation more efficiently on an ongoing basis. From our experiences across the market, the key is to develop a suite of documents (and underlying processes) that is not only compliant with regulations, but also manageable in the long term, and fits into the overall governance structure of the organisation. 

This article offers a practical insight into the most important, yet often challenging, documents required for IMAP. 

The starting point: a documentation framework 

A clear documentation framework can be used to help manage your IMAP programme and also to help articulate the full suite of documents to the regulator. The Common Application Package (CAP) is a great starting point for developing one; ideally, it should be established at the start of your IMAP programme.

The documentation framework should include a full list of documents (a so-called 'inventory of evidence'), the purpose of each document and document trees illustrating how different documents link together.

Technical risk area documentation

The most time-consuming documentation tends to be the technical risk area documents -calibration and methodology documents for Reserve Risk, Market Risk and so on.

With regard to technical risk area documentation, some of the key aspects to consider are:

• The structure of the documentation - Some companies prefer an end-to end document for each risk area, covering all relevant aspects (for example: data, methodology, parameterisation, expert judgements, limitations, etc.). This approach makes documentation more transparent and easier to follow.
  
  Alternatively, you can split documentation into 'static' and 'non-static' elements. Static documents, for example, would typically include design and methodology; non-static documents would include data and parameterisation. This approach is more manageable because static documents would not require updates as often as non-static ones.

• A good document should map to Solvency II requirements. In fact, the contents page itself often gives us a sense of how well the requirements have been considered.
  
  Some companies put the relevant requirements in an appendix and signpost how each requirement is satisfied by different sections of the document. This can be time consuming and difficult to manage, but it does create transparency for those reviewing the document.

• A good executive summary can be a powerful and transparent way to communicate key messages. It gives your audience (regulator, management, etc.) a good overview without having to read the entire document.
  
  
• The materiality of all expert judgements and limitations should be determined, and a proportionate approach should be used to justify and assess them (such as expert panels, qualitative and quantitative justification).

Other key technical documents

There are other technical documents required for Solvency II which have proved to be very challenging:

• External models - The level of external model documentation should be commensurate with the materiality of the relevant risk area. A proportionate approach should be taken and should include:
  • Justification of the External Model choice
  • Understanding around the methodology and calibration of external models, including user-defined options - justification of the selections and their impact on the capital should be provided
  • Justification of any adjustments made to the standard off-the-shelf model
  • Evaluation of strengths and weaknesses, including appropriateness of external models to your own risk profile
  • Review of validation carried out by external vendors
  • Your own validation testing (for example, back-testing using your own data)
  • Consideration of risks (such as unexpected model change or reliance on third party support).
• Dependencies - This tends to be a material element of any model and will need to be documented appropriately. This includes correlations within a risk area and correlations between risk areas. The former may be included as part of the risk area documentation and the latter may be documented in a separate Dependencies report.  

The level of justification should again be proportionate, with consideration of historical data, benchmarking (external and standard formula) and a robust expert panel process. 

For a partial internal model, justification should be provided for your chosen integration technique from the five prescribed techniques per Annex XVIII of the delegated regulation.

• Model outputs - This involves analysis of model outputs at a more holistic level, via change analysis, P&L attribution, simulation analysis, stability testing, stress and scenario testing, reverse stress testing, etc., all carried out at an overall level. This ensures that, once all components of the model are aggregated together, the overall results are appropriate. 
  
  
• Balance sheet - This document would demonstrate:
  • Consistency of the modelled opening balance sheet with Reserving and Finance, with any differences in the valuation bases for assets and liabilities explained and justified
  • No non-modelled balance sheet item which would have a material impact on the capital
  • The calculation of the balance sheets at both time 0 and time 1 is performed consistently, in order to calculate the Own Funds movement over the year.
• Risk coverage report - This should confirm that all material quantifiable risks are covered by the model, with justification for any excluded risks (such as a risk that is too immaterial to warrant the extra modelling complexity). The assessment needs to be carried out quarterly using a set of qualitative and quantitative tools (such as P&L Attribution). Importantly, you should ensure that all the identified exclusions do not aggregate to a material level and are considered in the model development plan.
  
  
• Future management actions plan - This should justify all the modelled (and non-modelled) management actions, provide a description of how these are implemented in the model and in practice, and quantify their impact on capital. All modelled management actions should be approved by the Board on an annual basis. 
  
  

Key non-technical documents

Some of the key non-technical documents include:

• It is common practice to have a model scope document that provides a high-level overview of components (such as risks, entities, business units) included or excluded from the model, as well as a model design document that provides a high-level overview of the overall methodology. Together, these documents can provide a good introduction to the model.

• Model use requirements are wide ranging and include:
  • Demonstration of model uses
  • Board training and involvement in the IMAP
  • Model users' awareness of key assumptions and limitations of the model, as well as any adjustment made to the model outputs, in order to make an informed decision

It is often helpful to have a model use document that demonstrates the compliance with all model use requirements, supported by underlying evidence of model uses.

• Most models we have seen tend to be supported by a huge number of policies and standards and other related governance documents (for example, definition of roles and responsibilities, terms of reference, charters, etc.). Based on our experience, the UK regulator tends to focus on a small number of core policies, namely expert judgement policy, model governance policy, model change policy, and model validation policy. All these core policies should be reviewed on an annual basis.
  
  
• A data governance framework around both internal and external data used in the model should include a data policy, a data directory, data lineage diagrams illustrating data transformations and controls around them, data quality assessments, data limitations identified and a remediation plan. 
  
  

The end point: a strong cover letter

One of the key documents required to support the IMAP submission is a cover letter. The requirements around the cover letter are specified in Article 2 of the Implementation Technical Standards.

We have seen varying approaches to the cover letter. Some companies kept this quite brief (a few pages to hit the core requirements), whereas other companies took a more detailed approach (50-plus pages). There is no right or wrong answer here. However, as it is often the first document to be reviewed, the cover letter is a good opportunity to communicate key messages to the regulator, such as:

• A very high-level overview of the model
• Risk profile of the company
• Key model outputs and drivers of the capital 
• Key expert judgments and limitations 
• Responses to previous feedback from the regulator 
• Signposting of key documents.

Jennifer Khaleghy and Ashish Kwatra are both experienced capital actuaries, specialising in Solvency II and implementation of the IMAP programmes.