[Skip to content]

Sign up for our daily newsletter
The Actuary The magazine of the Institute & Faculty of Actuaries
.

Back to the future

Measuring investment manager performance is critical for two reasons. First, we need to make a judgement about the past as to whether a manager has done a good job if you are a fiduciary this is a vital part of your role. Second, we need to use the information from past performance to make a judgement about the future. How can we do both of these as effectively as possible?
This first article makes the case for including an assessment of risk alongside the measurement of performance, considering also the evolution of performance measurement. The second article outlines how performance should be measured in the future.

Risk as well as return
We need first to describe why risk is so important. The big problem with risk is that its greatest significance lies in the future, but the only hard measurement we can carry out is in respect of the past.
If we are accounting for our past actions as a trustee, the only measure that seems to matter is return. But do we best describe whether a manager has done a good job by considering the outcome alone, or the outcome and the context of how that outcome arose? Validly judging whether an outcome was ‘successful’ (even a large outperformance) can be done only in the context of what else might have happened. So, if we were lucky and got away with it, we should be less impressed with our success than if the outcome had been mediocre but as expected.
Put in this way, it seems reasonable to consider a measure of risk alongside the return. I would argue the case for capturing the efficiency or quality of a particular strategy by measuring the ‘net information ratio’ (benchmark return divided by tracking error minus costs). In a recent survey by Watson Wyatt, a majority of investment managers (58%) supported this view.
The net information ratio can also be a powerful tool for aggregate results, and is not confined to monitoring individual managers. Investors define their investment goals in terms of achieving target returns for appropriate risk. In building the strategy to meet their goals, elements of that strategy can be fine-tuned to optimise the mix of return and risk expected. The relationship between return and risk relative to the benchmark is reassuringly simple in mathematical terms: risk scales up or down the return. Investors can therefore develop an aggregate risk budget, which can be ‘spent’ across strategies and managers.
Different types of rationality
The question we are addressing is, ‘What is the most rational way to assess an investment manager’s performance?’. The answer not surprisingly is ‘it depends’. It depends on whether you subscribe to procedural rationality or to substantive rationality.
Substantive rationality is about outcomes and is unconcerned with process, while procedural rationality is concerned with process and has little to say about outcomes. For example, those who insure their house contents and moan about wasting their money when they do not claim are substantive rationalists; those who say that the insurance premium was money well spent for peace of mind are procedural rationalists.
With regard to an investment manager’s performance, substantive rationalists will see a (relative) performance outcome of, say, -10% as a clear signal that something is wrong. Procedural rationalists would not be able to reach a conclusion until they had understood the process that led to that result.
Statistics tend to favour procedural rationalists, as substantive rationalists would have to wait for the results of multiple periods before the statistics would indicate (on outcome alone) a possible problem. Procedural rationalists would ignore the result and instead look for other information about how the result was generated.
So what is the most rational way to assess an investment manager’s performance? Recognising the two types of rationality, we need to understand outcomes better, and to understand the process better.

Historical development of performance measurement
Performance measurement has come a long way since the days when investment managers were commissioned to produce ‘the best return possible given an acceptable level of risk’ (1960s and before). This arrangement had the advantage of allowing the investment manager to exercise his or her skill to the greatest extent possible. However, it also meant that the client did not know exactly how good a job the investment manager was doing. Both funds and managers recognised that it was necessary to quantify results relative to some form of performance target.
The result was benchmarks the second phase in the history of performance measurement (1970s). While the industry has developed concerns about benchmark abuse, investment manager opinion remains clear that the benefits outweigh the problems.
The link between return and risk was left as implicit for some time. Higher performance targets were expected of higher risk styles, but although some risk targets did start to appear in the 1980s, their use remains far from widespread. Our global estimate suggests that fewer than 10% of institutional mandates worldwide carry any quantifiable risk targets.
Most performance and risk objectives written into current investment management agreements are of the ‘+1, -3’ type; that is, to outperform the benchmark by 1% per annum while limiting the downside to no more than 3% below the index.
Such objectives should be understood as a ‘strike rate’ concept or frequency test (you should miss the target, on average, no more than once in every six periods), not as a simple performance hurdle (‘don’t miss the target’). But manager agreements are not usually drafted with the subtext needed to explain this, and widely varying interpretations have been observed. You need only review the tangle of evidence presented by expert witnesses in the Unilever case to recognise this.

Statistical measures
So how do we find the way forward? Are explicit tracking error targets the answer? Our conclusion is that in most cases they are, although they should not be used in isolation. Tracking error (also known as active risk) is the standard deviation of active returns (the portfolio return minus the benchmark return). Defined in this way, tracking error is an historical (ex post) measure. But it is most useful on a forward-looking basis (ex ante) meaning, will the predicted tracking error suggest any changes to my investments?
We can predict future tracking error by assuming that historical tracking error is a reasonable predictor of future tracking error, although the current portfolio will be different from the one that produced past active returns. Alternatively, we can base our predictions on the current portfolio. Again, we have to rely on past relationships, but in this case we consider the past volatility of individual securities and how they behaved relative to other securities.
Generating the risk statistics is relatively straightforward, but there are some problems:
– Normal distributions: active returns are assumed to be distributed normally, and therefore the standard deviation of that distribution (tracking error) is a fair measure of risk. However, returns are not normally distributed: they have fat tails (extreme events happen more frequently than they should). Furthermore, we assume that active managers have skill, which means that their returns may be positively skewed. Also, in aggregate, the relative returns of active managers must be centred around a negative return, because of fees and costs.
– Serial correlation: the returns in different time periods are assumed to be unrelated to each other. This is not always true, in which case ex-ante estimates of tracking error will understate the true risk. Non-independence may be a result of the market’s rewarding certain risk exposures for several consecutive periods, or of the use of rolling measurement periods.
– Is the benchmark appropriate? For managers (structures) with strong style biases, broad market indices will produce misleading tracking error results.
– Market volatility is not constant. As a consequence, portfolio risk can change even when the portfolio does not.
– Event risk (such as war or natural disaster) cannot be forecast.
The main conclusions we draw are that, while tracking error is a useful measure, it is an imperfect and incomplete measure of risk. As a consequence, the interpretation of tracking error is critical, and cannot be done in isolation. We need knowledge of the manager’s process, style, and portfolio, together with knowledge of the current market conditions.
In other words, we are arguing for a much more holistic framework within which to monitor manager performance. The ideal framework combines ‘soft’ qualitative elements with the ‘hard’ quantitative numbers. This will be the subject of the second article.

02_08_04.pdf