Probability management

For people involved in financial and business forecasting and decisions, one of the most interesting developments in the past decade has been the bundle of ideas known as ‘probability management’.

Those core ideas address some familiar challenges: co-ordinating and combining forecasts from different parts of an organisation; reducing the level of skill needed to do forecasts that properly reflect uncertainty; and making methods available across a range of different types of software, starting with the familiar spreadsheet. These contribute to better management, especially management of risk.

Imagine that an organisation consists of three divisions and each is asked for a forecast of results to the year-end. To help with understanding the uncertainty involved, they are asked for three forecasts: one that is average, one that is optimistic, and one that is pessimistic.

What is the forecast for all three divisions combined? In particular, can we just add up the optimistic forecasts to get an overall optimistic forecast and add up the pessimistic forecasts to get an overall pessimistic forecast? The answer is: not safely. The problem is that each division will have imagined the conditions that would give them a good result and the conditions that would give them a bad result. Those conditions might not be the same for each division so they might not happen at the same time.

Suppose the organisation is a motor dealership and the divisions are new car sales, used car sales, and parts and servicing. In a strong economy the new car sales do well and used car sales slump, while in the weak economy it is the other way around. The divisions will not both achieve their optimistic results under the same circumstances but, happily, they will not achieve their worst results at the same time either.

This forecasting problem is more important for organisations like this hypothetical motor dealership that have activities that thrive at different stages of an economic cycle, either by design or as a result of past responses to changing circumstances.

The key to combining these forecasts safely is to specify the economic conditions for the scenarios for which you want forecasts and ask divisions to provide a forecast for each scenario. In the example, this might lead to the new car sales division making forecasts that rise in line with overall economic conditions, while the used car sales forecasts move in the opposite direction.

When this is done it is safe to add up their forecasts for each scenario. This simple idea of asking all forecasters to provide forecasts for the same, defined set of scenarios is at the heart of probability management.

If producing a forecast for one scenario is a slow and painful process, perhaps involving asking people to fill in forms with their guesstimates, then you might be limited to three scenarios.

However, probability management typically assumes you want to go further and get a clearer picture of the uncertainty in your forecasts. It may be particularly important for you to monitor the chances of an unlikely but crucial event, such as breaching a debt covenant, becoming insolvent, or interrupting a favourable trend.

To make this possible, the right kind of automation is needed. That means building forecasting models that take the scenario as input and automatically provide a forecast that is driven by that scenario. It may not be particularly difficult if most of your numbers are clearly driven by overall sales, for example. The tricky relationship is the one linking environmental conditions to the sales figure, but once that is sketched out and put in place the rest is relatively straightforward.

With such a model created, the only thing stopping you from running as many scenarios as you want is the task of generating the scenarios, and this is where another of the big ideas of probability management comes in

Monte Carlo simulation is a technique that runs lots of scenarios to capture input uncertainty. It is easy to do and understand if you are familiar with financial models on spreadsheets. The way this simulation has usually been done is to use random number generation software to select the numbers needed for each scenario at the moment it’s needed. This is typically all done by an add-on in Microsoft Excel, and there are some excellent ones available.

You set up your model, specify the uncertainty around input variables, say how many iterations you want, and set the software going. Usually it will provide you with tables and graphs summarising the implied uncertainty around your output forecasts within a few seconds. What you don’t often see is exactly which scenarios the system generated or what the results were in each one. If you want to revise your model and re-run the simulation then the scenarios generated will be different every time, giving slight variations in the output that are nothing to do with the changes you made to your model. Also, there is no way to make sure everyone uses the same scenarios so that they can be combined.

By far the hardest and least familiar part of this process is to specify the uncertainty around input variables, which is what controls how the scenarios are generated. Probability management’s next big idea is to separate out this task and have it done by an expert (the chief probability officer) so that others do not have to tackle it.

To summarise, the main advantages of pre-generating scenarios are as follows:

• One expert can do it so others do not have to learn the skill.
• Everyone can use the same scenarios, making it easy to combine forecasts.
• The expert can invest effort and time in doing a better job of creating scenarios.
• Simulations can run faster because there is no need to do the calculations to generate the numbers again.
• You can see exactly what happened in each simulated scenario and so understand better what causes the more extreme or worrying results (this makes the whole process of Monte Carlo simulation more understandable for people).

A table of data where each row represents a scenario called Stochastic Library Unit with Relationships Preserved (SLURP). A single column in one of these is called a Stochastic Information Packet (SIP). If you think these names have been chosen with a sense of humour, you are right. The inspiration behind most of probability management is Professor Sam Savage, author of the book, The Flaw of Averages. Besides being a technical expert, Savage is good fun and dedicated to making technical ideas accessible and easier to use.

The special thing about a SLURP is that the relationships are preserved. For example, imagine you are forecasting the demand for healthcare at a hospital. There is a distribution for the number of new cases of broken bones coming in each day, and another for the number of influenza cases each day. However, these are not independent of each other. Both are more common in winter. 

You can create scenarios that capture this in two ways. The first is using software to generate the number of cases for broken bones and influenza in a way that is statistically linked, which means you can simulate situations that have never happened before, but it is hard to be sure your statistical link is realistic. The second is recording actual case numbers for days in the past so that you preserve the exact relationships, though you can only simulate situations that have happened in the past.

In 2013, Professor Savage and others founded a non-profit organisation called Probability Management. to promote the ideas. It has published an open standard called The Open SIPmath 2.0 Standard, which has encouraged a number of software companies to enhance their tools to support probability management techniques. Perhaps more important is the fact that probability management can be done with ordinary Excel and the tools are available, at no charge, from probabilitymanagement.org just to make it easier to create the models, SIPs and SLURPs. They are not needed to run the models once created.

The ideas of probability management have emerged as one of the most interesting developments in both financial forecasting and risk management. Special software is not needed and co-ordinated forecasts that reflect uncertainty can be produced across even large organisations without requiring everyone to have an advanced understanding of probability distributions. You can find out more from the website, probabilitymanagement.org, and just by opening a spreadsheet program and having a go.

Download pdf article:

• Probability management
  Business & Management Magazine, Issue 268, October 2018

Further reading into probability management and forecasting is available through the articles and eBooks below.

Our eBooks are provided for ICAEW members, ACA students and other entitled users. Users are permitted to access, download, copy, or print out content from eBooks for their own research or study only, subject to the terms of use set by our suppliers and any restrictions imposed by individual publishers. Please see individual supplier pages for full terms of use.