“The greater our knowledge increases, the more our ignorance unfolds.” This quote from President John F Kennedy, from a 1962 speech in which he laid out the business case for investing in the space race, might have been on the minds of UK political pollsters in May, when the electorate bucked the polls and returned a majority Conservative government.

Yet political surprises are... unsurprising. It’s easy now to look back to 1989 and see the collapse of the Soviet Union as inevitable. Yet few imagined it mushrooming in an evening when joyful Berliners began to reduce to rubble the hated Wall that divided their city.

Seismic economic crises still catch us off guard even when forecasting data is easily available. Governments, central banks and economists failed to notice the 2008 financial crash until it was too late. “We all misjudged the risks,” said former chairman of the US Federal Reserve Alan Greenspan. And what of the core issues? “No one saw the housing bubble,’ said Berkshire Hathaway CEO Warren Buffett. Some did, but others ignored it.

When governments fail to see into the future, what chance do FDs and CFOs, expected to forecast an organisation’s financial fortunes, have to get it right? Has forecasting progressed at all, even in an era of big data analytics?

What is the ideal timeframe for forecasting? Often, this depends on the sector. In outsourcing, where three-year contracts are the norm, it makes sense to prepare three-year forecasts in addition to monthly ones. Yet for a consulting business, looking beyond three months might seem unrealistic.

It seems fair to surmise that any forecast is only as good as the data on which it is based. Given that more data is available, along with more sophisticated methods, forecasts should be getting more accurate.

In reality, says Rob J Hyndman, editor-in-chief of The International Journal of Forecasting, most companies are either not using all available data, or they are not employing people with sufficient data analysis skills or forecasting expertise. “Forecasting in many companies is no better now than it was 10 or 20 years ago,” he says.

On the other hand, some companies are employing skilled data scientists and using high-quality statistical software to build much stronger forecasting models than ever before. “This is happening in some business areas more than others,” Hyndman agrees. “Some manufacturing companies are doing excellent forecasting of the weekly or monthly demand for thousands of their products. These forecasts are much more accurate than they used to be because of better software, better algorithms, and better use of available data.”

Professor Robert Fildes, co-founder of Lancaster University’s Centre for Forecasting, has detailed research on forecasting activity. “In some industries, especially IT and telecoms, the complexity of markets and competition make it more difficult than ever.”

There is no single, simple answer, says Fildes. The degree of accuracy depends on which companies, and which parts of those companies, are doing the forecasting. Which models they use, the data and intelligence they drawing upon, and how they are use it. And, as our case studies below show, there are a number of different approaches.

At business services company Capgemini UK, there’s been a confluence of forecast numbers and actual results. Regular forecasts are now based on actual results, in the sense that monthly accounts are closed so quickly they go straight into a forecasting process.

“Forecasts and their associated variances have become more important than budgeting,” says CFO Tony Deans. “Especially in fast-moving markets with budgets becoming out of date almost as soon as they are completed.”

While there is a wealth of external information available, from data and intelligence to statistics, companies such as Capgemini will often turn to internal data, using past performance and their own market experts to facilitate the forecasting process.

“Internal big data can lead to many variables as well, so sophisticated models and forecasting tools are used to extrapolate past performance into future performance,” says Deans. “That’s not ideal, as with a short order book they are, in fact, analysing expected market trends and KPIs, thus adding to their intake of external data.”

Then there is the question of forecasting frequency. In Deans’ view, for a services business, once a quarter is reasonable, although he says many would argue that it should be done monthly.

“Ultimately, it depends on whatever makes sense to help with running the business and enabling decisions around your customers and resources,” he adds.

Rob James has only recently been appointed to the role of FD at Jones Bros; in his view, the increased availability of data has been offset by a faster pace of market and business change, effectively reducing the potential for accurate forecasts.

He says: “There is an ability to include more factors and complexity in forecasting, but ultimately it is the unknown unknowns that will result in a variance. The key is to remain a healthy and flexible organisation that is able to respond quickly in the face of changes to external factors.”

He forecasts on both a top-down and a bottom-up basis, sense checking between the two.

“There are limitations of both approaches, for example bottom up can lack an appreciation of some of the wider strategic or external factors, whereas top down, although benefiting from experience, may hold assumptions based on too many preconceptions which are out of date,” he says.

Added to the challenges of forecasting is the issue of potential conflict between what accurate, internal projections might suggest and the desire to control the flow of information externally.

As James says: “For many businesses, I think this is an issue where there are external stakeholders who are more inclined to take a short-term or more binary view of performance. It can become an exercise in managing expectations rather than doing what is required to manage the business effectively.

“Fortunately, our business is family-owned and well funded, so all stakeholders are aligned to a longer term view. The information is taken into account alongside a whole raft of other considerations rather than being the ‘be all and end all’.”

Chris Chapman, managing partner of professional services firm Numitas, is an accountant and CFO by profession, with more than 20 years in senior finance roles within a number of industry sectors.

His approach to accurate forecasting is, by his own admission, quite pragmatic. He says he works closely with the rest of his team to get a thorough understanding of the order pipeline, the sales cycle and delivery obligations of deals.

“You have to spend time with budgetholders and those accountable for costs, and use sensitivity analysis to understand the impact of different levels of achievement and, where appropriate, you incorporate regression analysis,” he says.

He gains genuine insights, he says, just from close examination of a sales pipeline, understanding the different levels of optimism within that pipeline from sales directors of different product lines, the timing of sales cycles and likely margins that are achievable. These elements are all component parts of the process, but it has to be understood as a total picture. Considering any of these elements in isolation is dangerous, he warns.

“If those projected sales deals depend on new software features being delivered on time, while the development team is under-resourced, unable to recruit new staff, and likely to miss the deadline, then it will profoundly change the likely forecast,” says Chapman.

Is forecasting any easier in small, agile technology start-ups? While many are prolific accumulators of so-called big data around existing customers and sales and marketing efficacy, does any of this make forecasting easier or better?

Hugo Grimston, FD at technology start-up paddle. com, says the wealth of data available to him allows him to forecast revenues throughout a long timespan in a very granular fashion.

He says: “This proliferation of readily available data means that all FDs can be more informed and granular about their forecasting. But unless you work for a company with long-term guaranteed contracts, there will always be some degree of uncertainty about forecasts.

“In a fast-moving environment like technology, it is almost impossible to be totally accurate. You can only aim to be directionally correct and then use actual experience to recalibrate and inform future forecasts.”

Grimston’s preferred method of forecasting involves building a detailed, bottom-up forecast for the top line. Based on sales and marketing spend for each month, the customer acquisition cost, and the time delay gaining a paying customer, he can forecast the incremental revenue for each month.

Grimston says he looks 12 months ahead as a good discipline. However, for venture capital fundraising purposes the company needs to provide a three-year forecast.

“No one really believes the figures more than 12 months out in the high-growth technology world. But it shows where you might get to with a fair tailwind behind you,” he says.

As a decision-maker, you ultimately have to rely on your intuition and judgement. I visualise effective forecasting as mapping a cone of uncertainty, a tool I use to delineate possibilities that extend out from a particular moment or event. The forecaster’s job is to define the cone in a manner that helps the decision-maker exercise strategic judgement.

The most important developments typically follow the S-curve shape of a power law. Change starts slowly and incrementally, and then suddenly explodes, eventually tapering off and even dropping back down. The art of forecasting is to identify an S-curve pattern as it begins to emerge, well ahead of the inflection point.

The leading-edge line of an emerging S-curve is like a string hanging down from the future, and the odd event you can’t get out of your mind could be a weak signal of a distant, industry-disrupting S-curve gaining momentum. The best way for forecasters to spot an emerging trend is to become attuned to things that don’t fit, things people can’t classify or will even reject.

In forecasting, lots of interlocking weak information is more trustworthy than a point or two of strong information. Good forecasting is a process of strong opinions, weakly held. If you must forecast, then forecast often – and be the first to prove yourself wrong. The way to do this is to form a forecast as quickly as possible and then set out to discredit it with new data.

The texture of past events can be used to connect the dots of present indicators and thus reliably map the future’s trajectory – provided one looks back far enough. The recent past is rarely a reliable indicator of the future – if it were, one could predict the next 12 months of the Dow or Nasdaq by laying a ruler along the past 12 months and extending the line forward.

It is a liability for forecasters to have too strong a proclivity to see change, for the simple fact is that even in periods of dramatic, rapid transformation, there are vastly more elements that do not change than new things that emerge. Against this backdrop, it is important to note that there are moments when forecasting is comparatively easy, and other moments when it is impossible. The cone of uncertainty is not static; it expands and contracts as certain possibilities come to pass while others are closed off. There are thus moments of unprecedented uncertainty when the cone broadens to a point at which the wise forecaster will demur and refrain from making a forecast at all.

Download PDF version:

• Finance & Management magazine, June 2015 (pp.16-19)

The ICAEW Library & Information Service provides access to leading business, finance and management journals, as well as eBooks.

Further reading on forecasting is available through the resources below.