[Skip to content]

Sign up for our daily newsletter
The Actuary The magazine of the Institute & Faculty of Actuaries

Back to the future II

This article is the second of two (see p20, August 2002 issue) that together aim to build a model of best practice for monitoring manager performance. The hypothesis has been that monitoring manager performance is critical for two reasons: to make a judgement about the past (a vital part of a fiduciary’s role), and to make a judgement about the future. The first article covered the necessity of including an assessment of risk, with some background about how performance measurement has evolved to the present day, and a discussion of the pros and cons of using tracking error as an industry standard. This article offers an opinion of how performance should be measured in the future.

Future monitoring
The challenge is to design and implement a better system for measuring and monitoring investment managers in the future. This system should allow us to understand the investment outcomes and process better, and therefore to make better inferences as to the risks being run and the outcomes that are likely. Monitoring addresses the extent to which past and future results conform to an hypothesis of positive skill on the part of the investment manager.
The elements that we can measure include the following.

The relative return over the period
To understand the relative return (outcome) better it is important to ensure that:
– we are using the correct benchmark;
– we are considering an appropriate period; and
– we have access to other information which would corroborate or refute the hypothesis of skill being present (for example style factors, sector exposures, beta, significant positions, etc).
We need to consider the time over which we should measure the relative return, as we wish to avoid overlapping periods which would invalidate any statistical conclusions. In general terms, the less strong the manager’s style and the more the benchmark is a proxy for the manager’s style, the shorter the measurement period can be.

The volatility of the relative return
In measuring performance we are essentially undertaking a statistical test to discover whether the distribution of returns reflects skill. For ‘relative return’ managers those without strong styles and/or where the benchmark is a good proxy we should measure the returns monthly or even more frequently.
For example, we assume that a chosen manager with positive skill can outperform by 1% per annum with a relative risk of 5% per annum (giving a net information ratio of 0.2 which would be attractive over time). In our monthly measurement we assume a distribution of relative returns centred round 0.08% with a standard deviation of 1.4%. We start collecting observations to see whether the actual distribution appears to be different from our assumed one.

The predicted tracking error
Because predicted tracking error is a forecast of the future, given the current portfolio, it is clearly going to be a more accurate and more useful measure (i) the more securities there are in the portfolio to offset estimation errors, and (ii) the more closely the portfolio resembles the benchmark. Hence predicted tracking error is a more useful measure for diversified, relative-return mandates than for benchmark-insensitive concentrated portfolios.
A question that should be considered is whether the tracking error should remain constant through time. While it is much easier to measure a manager’s risk and to hold the manager to account if the risk is constant, experience shows that investment managers see different amounts of opportunity in the market place at different times. Given a free hand, the risk of the portfolio should rise in ‘times of plenty’ and fall when there are few clear opportunities.
We believe that, despite its imperfections, tracking error should remain a key element of the monitoring process. But until a better risk measure is developed, we must learn to use it more intelligently. This means recognising that:
– the predicted tracking error will change as market volatility changes;
– the manager may want to vary the tracking error of the portfolio over time;
– tracking error is a statistical test that requires careful interpretation; and
– it is an incomplete and imperfect measure of risk.
Arguably, outcomes that are ‘two standard deviations bad’ should be taken on the chin, while outcomes that are ‘three standard deviations bad’ should call for forensic analysis of the performance, although they do not necessarily imply anything other than bad luck.

The portfolio positions
A periodic study of the portfolio positions, and a discussion of them with the investment manager, can be a useful addition to the monitoring process. Such studies can help in understanding any abnormal relative returns, and can give evidence of the manager’s sticking to his style or drifting from it. This acts as a check on our understanding of the philosophy and process. More importantly, it allows us to develop insight into a manager’s skill.

The four-pillar monitoring model
Having discussed the necessary building blocks, we are now in a position to propose an improved framework for measuring and monitoring investment managers. The framework that we advocate is a generic model with four ‘pillars’. So far, we have dealt predominantly with the outer two pillars, but we would argue that the central two pillars are just as important, if not more so.
– Past skill This is where we form a qualitative view of a manager’s past competencies and judgements. We need access to historical portfolios in order to assess the skills (processes and judgements) that were applied to the portfolio in the past. The emphasis within this pillar is on the ‘alternative histories’ (what else might have happened), almost to the exclusion of the actual history or outcome. By this we mean that we are building an understanding of what the distribution of outcomes would have looked like, given the manager’s process, style, access to information, etc.
– Future skill With respect to future skill, we are forming a qualitative view of managers’ competencies to meet the objectives in the future in an attempt to construct a distribution of future outcomes. We form our view of a manager’s future skill through qualitative research, as do most investment consultants. Unsurprisingly, we believe research into manager skill has predictive power. However, we would make the crucial point that the prediction of future skill is not a prediction of a single outcome. We view the business of investment management as being about the delivery of a particular shape of distribution.
For the sake of completeness, we will also summarise our views on the outer pillars:
– Past performance and risk This is used to understand and validate a manager’s processes and approach but, as past performance has limited predictive power, there is no scope for decisions to be made based on this information alone. The more reasonable the assumption of a normal distribution of returns, the more use we can make of tracking errors and information ratios. Where the assumption is unreasonable, we need to work much harder at understanding the precise shape of the distribution for example, when we suspect skew and/or kurtosis to be present in the distribution (which may occur when options are used, or when relative returns are not relevant).
– Current portfolio Our analysis of the current portfolio identifies the levels of risk taken, the style and the investment views being expressed. Through this we can identify mismatches between the manager’s approach and the objectives set, which may require action.

Improving current arrangements
The importance of this subject is underlined by the principle in the Myners review which states that ‘the mandate will not be terminated before the expiry of the evaluation timescale for underperformance alone’.
Improving on current arrangements will involve forming more realistic expectations and improving the monitoring process. We need to accept that bad outcomes happen, even with good managers. For example, on statistical grounds alone, we should expect a ‘+1%’ manager with a net information ratio of 0.2 to produce a relative return worse than -10% once in every 20 years. If such an outcome is unacceptable for a fund, even at such a low frequency, the best solution would be not to employ an active manager.
So what practical help can we offer? Six points:
– It is perhaps time for a more mature dialogue on risk: the type of risk we can stomach, how to measure it, and how much and what we expect to gain from this risk. Forming a risk budget is the foundation stone of every investment fund.
– In the light of our reassessment of risk we can form a strategic plan that combines risk-minimising and risk-seeking strategies in appropriate proportions.
– The monitoring process should differ according to the type of strategy pursued.
– We must recognise the merits of a shorter-term monitoring framework incorporating benchmarks and relative return risk budgeting in certain strategies.
– We must also recognise that some strategies suggest a longer-term monitoring framework in which there are no benchmarks, just multiple comparators.
– We must recognise that in today’s volatile investment conditions there are no hard statistical tests that provide auto-piloted strategies. Softer monitoring processes are essential complements to the hard figurework.