[Skip to content]

Sign up for our daily newsletter
The Actuary The magazine of the Institute & Faculty of Actuaries
.

Decision markets

ost things fail. From light bulbs to companies to political revolutions, sooner or later everything comes to an end. Looking back, it often seems inevitable even predestined but at the time failure can be much harder to predict. Failure holds no respect for size or reputation either, as this table illustrates:

Performance of the world’s largest 100 industrial companies in 1912 over the period 191295
Remained in top 100 in 1995 19
Survived but outside top 100 33
Taken over 19
Bankrupt 29

The companies in the table were the crème of western capitalism, the survivors of a large number of mergers in the early part of the century. Yet in spite of their immense resources and size, within 85 years they were two-and-a-half times more likely to have ceased operating as an independent concern than they were to have retained their position at the top of the tree. Clearly some poor decisions must have been made along the way. As Hannah notes: ‘The proposition that it would be possible to fritter away $3bn (much less $90bn) in a human lifetime is one I personally find daunting, but business leaders are evidently made of sterner stuff.’
On a more granular level the failure rate of product launches by companies is higher still. One estimate puts the product failure rate in the grocery business at 88%. This is despite the large amount of time and money that are invested in any major product launch.

New Coke
A classic example is New Coke. New Coke was launched in 1985 by the Coca-Cola Company to replace its flagship brand which had been suffering from falling sales. In blind tests New Coke was overwhelmingly more popular than either Pepsi or original Coke. Coca-Cola therefore began the roll-out of New Coke on 23 April. However, the company was unprepared for the extent with which Americans identified with Coke, and taste be damned: they were not going to let some faceless corporation take it away from them. Within three months, public pressure had reached such a pitch that the original brand had to be reintroduced under the name Classic Coke. Much to everyone’s surprise, sales for Classic Coke just kept on rising and by 1986 it was back in pole position and stretching its lead over Pepsi. New Coke, which had been launched with such a fanfare, saw its market share slide to less than 3%.
So if a company like Coca-Cola, with a marketing budget in the billions, can misjudge its core audience this badly, what hope is there for the rest of us?

Poll tax
One of the important factors in deciding to give the go-ahead to any new product is correctly estimating the size of the target market and hence expected sales. This is doubly important for life insurance companies given their massive operational gearing.
Typically the sales forecasts for any new product are decided by a small group of individuals in the team responsible for designing and costing the product. Unfortunately this is not ideal, not least because most individuals are poor at predicting future events. Counterintuitively, it is often those who would consider themselves experts that perform worst when their predictions are subsequently compared with what actually occurred. The trouble is that we are all human beings (even actuaries!) and so tend to form pet theories or hunches. Once we have reached a decision, we hate that decision to be wrong. This makes us reluctant to change our minds, even in the light of crucial new information.
A good example of this is the poll tax, introduced in 1990 as a replacement for council rates. It was unpopular right from the outset, provoking mass demonstrations in Trafalgar Square and with millions of people refusing to pay. Margaret Thatcher, however, declined to countenance any U-turn and personally identified herself with the policy. Before 1990 was out she had been forced to resign and all three of the leadership candidates had pledged to repeal the tax. Although not the crucial factor in her demise, it was certainly a contributory element.

Iowa Electric Markets
So is there a better way? Well, yes, as it happens there is: markets. The stockmarket is well known to all of us, as is the fact that most traders consistently fail to beat passive index-matching strategies, despite all the time and money spent on analysing data. However, stockmarkets are used to give an estimate of the market value of a company at the current time. We are more interested in using markets to predict future events. Fortunately there are examples of markets being used for this purpose.
In 1988 the Iowa Electric Markets (IEM) was set up by the Business College at the University of Iowa. The IEM are real-money futures markets in which contract payoffs depend on economic and political events such as elections and are open to anyone who wishes to participate. Despite having a relatively small user base and a maximum amount of $500 per person to invest and most choose to invest far less than that research has shown that the market consistently outperforms polls and that this out-performance is ‘markedly better at longer horizons’.

Printer sales
Another and from a business perspective perhaps more useful example, is that of the experiment at Hewlett Packard carried out by the economists Charles R Plott and Kay-Yut Chen. They set up an internal market to forecast printer sales and then compared the results of this with the official forecasts produced by HP. This market, the snappily named Information Aggregation Mechanism or IAM, was much smaller than the IEM, consisting of only 2030 employees and running for a week with trading occurring during lunch and in the evenings. Twelve markets were set up, each one predicting the sales on a different product line. The charts show the results from two of these markets. As well as comparing the IAM prediction with the true outcome and the official HP forecast (where available), the charts also show the probability distribution given by the IAM forecast.
Their work yielded three important results:
– Result 1 Market predictions based on IAM prices outperformed official HP forecasts. They were closer to the true outcome in 75% of cases.
– Result 2 The probability distributions calculated from market prices are consistent with actual outcomes.
– Result 3 The IAM makes accurate qualitative predictions about the direction that the actual outcome will follow: above or below the official forecast.
The second result is especially intriguing. Most sales estimates are simply point estimates, but in our brave new stochastic world this isn’t always good enough. It is far more useful to have an expected range of outcomes as this can help give a feel for the risk inherent in the project, not just its expected return.
HP was so impressed with the performance of these markets that they subsequently integrated them into their regular forecasts.

Other uses
It is not only sales forecasts that decision markets can be used for. They can also be used to predict whether a project is going to finish on time or over budget, or to predict which proposal stands the greatest chance of success. This would provide an unbiased and independent check of the official forecast. For example, the pharmaceutical company Eli Lilly used them to predict which (mock) drugs would be more successful at being approved by the Food and Drug Administration in the US. The market successfully brought together all the various bits of information and quickly identified those drugs most likely to succeed.
In order for decision markets such as these to be efficient and provide meaningful results they need to fulfil two criteria:
– The participants must be diverse, ie not simply drawn from the actuarial or marketing function. This diversity can be achieved by drawing market participants from all areas and levels of the company such as marketing, finance, operations, IT, and actuarial. This helps to ensure that the market has as wide a knowledge base as possible.
– The participants must be independent, ie they should be allowed to make their own judgement without worrying how it will be perceived by senior management or colleagues.
This second point is the harder to achieve; few employees would be willing to publicly contradict their manager’s claim that all is well and that sales will easily meet the forecasts. Participant anonymity may help to mitigate this risk but is unlikely to be entirely successful.
Decision markets will not always be the best or most effective way of forecasting future sales. They take time to run to equilibrium and it can be hard to ensure that participants act independently. That said, they are a powerful tool, and any company that is better at predicting the future than its peers is one that is more likely to succeed.

06_05_08.pdf