[Skip to content]

Sign up for our daily newsletter
The Actuary The magazine of the Institute & Faculty of Actuaries

Multistate modelling: deterministic and simulation approaches

scenario, and simulation techniques for a
variety of tasks, and interest in asset liability
management (ALM) has increased significantly
among actuaries in recent years. Looking
ahead, market-consistent valuation techniques may
extend ALM techniques further, to aid fair value measurement
in international accounting standards.
But rather than limiting stochastic modelling to
solving ALM issues, this article explores how simulation
modelling can be applied to multistate problems. Our main example is
healthcare, but we then
take a look at some other
applications, including
customer lifetime value
We are all familiar with
the life insurance model
where policy status is categorised
as shown in figure
1 (left).This sort of interaction of states is fairly straightforward
for a number of reasons:
Deaths, paid-ups, and lapses/surrenders are
assumed to be independent decrements.
The process of stopping premiums is modelled as
one way; restarting premiums is not generally modelled
Non-contractual options to increase or reduce contributions,
or to switch funds, for example, are
rarely modelled deterministically.
In these models the benefits received and decrement assumptions used depend
only upon the state in
which you are and not on
the path followed to
reach the state. They are
thus not ‘path dependent’
and the assumptions are
not select other than
when leaving the starting
Actuarial systems can adequately cope with these types of models with
single dimension vectors of decrements and survival
factors over time.
Multistate models
While figure 1 is technically a model with multiple
states, the term ‘multistate modelling’ tends to be used when another layer or two of complexity is
added. A couple of example models for income protection
plans (IPP) and long-term care (LTC) are provided
below to illustrate the ideas.
In the IPP model (see figure 2, bottom left), complications
arise because we have introduced a two-way
relationship between ‘well’ and ‘sick’. Recovery rates
from sickness vary enormously between age at the
inception of a claim, the duration of sickness, and the
deferred period. To build an effective model of this
recovery transition, we must track every person that
becomes sick, from the start of their claim. This is
normally done by following each month of claimants
through as a separate tranche, with their own specific
recovery (and death) rates. In effect we have a series
of sub-states: sick with duration zero months, sick
with duration one month, sick with duration two
months, etc.
Historically, actuarial systems designed around the
deterministic models above have ignored recoveries to
the well state and used an inception-annuity
approach in which the full cost of future benefits
while in the sick state is recognised immediately. This
limits the ability to dynamically adjust the termination
probabilities without regenerating new claim
annuity tables. In addition, the models do not provide
information on the timing of claims cashflows, as
these are capitalised at the date of claim.
To overcome these shortcomings, it is necessary to
extend the single dimension vectors (inceptionannuity
value) into two-dimensional vectors in
which each cohort of claim inceptions (the rows) is
run off over time (the columns) using select duration
recovery probabilities based on age at incidence and
duration of claim. It then becomes a simple exercise
to model income payments that vary according to
duration sick, for example. The added benefit is that
‘summing across the diagonal’ provides the cashflow
in a given time period, while discounting along a row
gives the inception annuity (see figure 3 across, top
The approach can then be extended to models with
additional states of sickness. For LTC, two disability
states are often modelled, representing a person who
has failed (say) two of six defined activities of daily living
(ADLs) and a person who has failed three of six
ADLs. Recoveries may or may not be modelled, as the
onset of ADL failures is not as reversible as sickness as
defined under IPP contracts.
So far, so good. We can follow each state through by
duration and therefore allow for select transition
probabilities between various states. Modern actuarial
systems should cope with the above multistate deterministic
models. However, what if: The benefit levels paid are capped at an overall
maximum value across the contract duration (and
therefore multiple sickness claims).
Upon recovery from the first sickness, the inception
likelihood is increased and the recovery rates are
Upon recovery, the deferred period is waived if the
subsequent claim starts within three months.
Our deterministic model is now looking pretty sick
The simulation approach
The multistate models described above are effectively
deterministic in nature all contracts are assumed to
follow average decrement rates and filter into the various
states as time goes by. You start with one insured
life and may end up after a few years with say 60- of
a person in-force and healthy, 25- of a person in the
sick state (at various durations) and 15- have lapsed
or died. A student encountering this deterministic
approach for the first time often finds it confusing. In
reality a person can only be in one state at a time, so
why not model it this way?
Monte Carlo simulation techniques can be used to
model a particular route that a contract, person, or
company may follow. At the heart of simulation or
stochastic modelling is a random variable (or two or
three). Taking claim incidence as an example, in a
multistate deterministic model we would read in an
incidence rate and create a new tranche of claim incidences
each month. In a simulation we would take
the incidence rate and ‘throw a die’ to determine,
according to that rate, whether the person had
become sick or not. If they do become sick, their state
changes to sick, duration zero. Otherwise they continue
in full health. This process can be modelled very
simply by a Bernoulli random variable, with a parameter
of the incidence rate in that period. Figure 4
(right) illustrates the idea.
What may not be immediately obvious from the
above paragraph is that the simulation model no
longer requires the two- or three-dimensional multistate
vectors, as the person can only be in one state at
a time. You can now collapse the many states down
into a single ‘current’ state, thus reducing the model
complexity significantly, and potentially eliminating
errors that this complexity brings.
Once you have a simulation model working, you
will want to run a number of simulations to produce
a spread of results or calculate a representative
average. Some skill is required to estimate how many simulations should be run to provide reliable estimates
of averages and extreme values. Facilities to provide
simulation rankings, statistics, and graphs of the
results are important to help you efficiently process
the multitude of information available. Customer lifetime value modelling
Other applications of simulation modelling include profit
sharing arrangements under reinsurance contracts and, of
course, simulation techniques are used extensively in general
insurance to model claim frequency and severity on the liability
side. Another application of the simulation technique is
calculating a customer’s profitability to your organisation. You
may believe that their future repurchase and lapse behaviour
is dependent on their length of relationship with your organisation
and their product holdings at any given time.
This type of model can be viewed in terms of an increasingly
sophisticated multistate model. In this case the states do not
refer to levels of sickness but rather to items such as number of
product holdings, level of premium paid, recent activity in
each policy. Further, the transition probabilities can be
impacted by the explicit holdings and path to that status at the
particular projection period. For example, you may have different
premium increment and lapse probabilities for an
investment contract which depend not only on the current
premium level for that contract, but also on whether (say)
there have been increments in each of the prior three years
and whether the associated mortgage with the financial institution
has remained intact.
The multistate methods described above could be used. One
approach may be to assume capitalised fixed future profit
value at point of product purchase. This is analogous to the
inception-annuity approach and would ignore any interactions
between policy holdings other than on the impact to
purchase. To allow for a degree of interaction, a deterministic
multistate model using multi-dimensional vectors could be
employed. However, this can soon become unwieldy as the
complexity grows. In comparison, a simulation approach is
particularly useful in these types of models. You can allow for
dynamic interactions among the product holdings as the customer
follows one particular path, while keeping the model
coding relatively simple.
Applying the techniques
Primarily, all that is required in order to migrate your multistate
deterministic models to the simulation approach is to
place a stochastic process around your transition probabilities
and then run the model through a number of iterations. Given
that the insured only follows one path in each simulation, a
wide range of possible outcomes may require numerous iterations.
This has implications on run times, but is less likely to
be a barrier when using these models for pricing work, for
Simulation modelling is widely adopted by professionals in
numerous industries, for problems such as ‘just in time’ delivery
or telephone queuing periods at call centres. Tools have
been developed to help build these models, and data mining
techniques are used to help extract patterns from customer
In practice we have found that extending standard profit
testing models for use in both modes deterministic and simulation
is a surprisingly simple exercise. Interpreting results
does require care, but arming actuaries with these techniques
may provide opportunities for us to apply skills in areas where
these techniques are commonly used.
Modelling various scenarios provides a
spread of results that indicate likelihoods of
the outcomes modelled.
An outcome under a simulation may never
happen under a deterministic method for
example, a life moving between well and
sick states numerous times within a short
timeframe thus eliminating the deferred
period on benefits.
Models can be simpler in structure than a
deterministic model, as only the current
state need be modelled.
Modelling in a simulation environment
allows you to develop a rules base that is
far more sophisticated (and, hopefully, more
realistic) than under a deterministic
methodology. This may not actually be used
to provide spreads of results, but just to
calculate averages across complex
multistate decision paths.
Why use a simulation approach?