Sometimes it’s not possible to get more information, do a proper spreadsheet analysis, or calculate a forecast or estimate – we just have to guess. Matthew Leitch asks what science can teach us about how to guess well
In other tasks, our limitations are painfully obvious, with logical mistakes and persistent, sometimes large, biases easily demonstrated. We are terrible at combining several factors to reach one estimate or judgement. We are worse when we feel pressure from others to say what they want to hear. Our misjudgements are magnified by uncertainty in unfamiliar situations.
Unfortunately, we are often forced to guess in situations that are far from ideal. Perhaps there is no opportunity to get more information. Perhaps there isn’t time to think carefully and make a model with calculations, or perhaps colleagues won’t take any notice or do their best to discourage that kind of analysis.
Even with modelling and data gathering banned, there are some tested techniques that can help. Since many guesses are made in meetings it is especially useful to know how to guess in a group.
We are terrible at combining several factors to reach one estimate or judgement. We are worse when we feel pressure from others to say what they want to hear
With other people
Professor Scott Plous compared several techniques for counteracting over-confidence using four related experiments. The subjects had to answer difficult quiz questions (e.g. what is the diameter of the moon?) by giving ranges of numbers such that they were 90% sure the correct answer lay within that range. The best technique was to ask each member of a three or four person group to write down their answer individually, without conferring, then take the highest upper limit given by any person in the group, and the lowest lower limit given by any person in the group. This often meant the two numbers did not come from the same person.
With this technique, groups showed minimal over-confidence in their answers and in assessing their own performance before finding out their scores. Their performance was dramatically and consistently better than that of individuals working alone and groups who discussed the questions. This was true even when those groups used a variety of strategies to try to counter over-confidence, such as appointing a devil’s advocate or deliberately considering reasons why their interval estimates might be too narrow. Direct instructions to avoid overly narrow ranges did not work and groups that discussed their answers were little better than single individuals even though they thought they performed much better.
This is a specific example of the more general strategy of getting the views of group members individually, then combining them. Taking the average of the group (as well as the range) can give more accurate averages too.
One potential problem is that someone might suggest a number out loud before others have made their estimates. This could produce a large bias across the group thanks to the well-known anchoring effect, which is also hard to feel and counteract directly. Try to prevent people from mentioning any numbers at all before everyone has written down their personal views. If a number is mentioned too soon you might reduce the impact by immediately suggesting several other numbers, ranging very widely. Even a number mentioned as a guess is dangerous and in one study numbers selected at random using a spinning wheel in plain view still produced a substantial bias!
One useful observation is that when we are thinking about how long we will take we focus on the steps of our plan and tend to ignore our past track record and other relevant factors. In contrast, when asked to estimate how long someone else will take to do the task we take their track record into account more often and make longer estimates as a result. So, for a less rosy estimate of how long it will take you to do something, choose someone with no reason to pressure you and ask them to estimate for you.
A more obvious strategy is to break the task down, make guesses for each component, and add them up. Does that work? Research on this is inconclusive and it seems that it sometimes works better than just making an overall estimate and sometimes is worse. Sometimes we find it easier to estimate for the components because they are more familiar, but breaking the task down this way can draw us into the Planning Fallacy, whereas an overall estimate encourages us to think back to previous, similar projects and the range of outcomes.
If nothing else, decomposition probably helps reduce the impact of uncorrelated estimation errors, but consistent underlying bias remains.
Magne Jørgensen, of the Simula Laboratory in Norway, has studied effort estimates in software projects extensively and suggests combining independently made overall and decomposed estimates. Another recommendation is to decompose the project in more than one way and compare estimates made with each.
In practical terms this might involve asking one colleague to make a private estimate overall, ask another to break the project down into time phases and make estimates for each of those, and ask another to perhaps do it by cost category. The average and top-to-bottom range of those might be useful information, even if there’s not much detail and a spreadsheet is not used.
We tend to be inconsistent at this and find it tiring. In a classic paper from 1979, Robyn Dawes reviewed the evidence comparing human combination of factors to combination using “linear models” and demonstrated a remarkable conclusion.
A linear model is simply one in which each of the factors in the decision is represented as a number and then those numbers are multiplied by weights and added together.
Ideally, the factors should be combined in a way that makes theoretical sense, such as multiplying or combining in some other way, but linear models just feature adding.
Ideally, each factor should be carefully transformed so that it can be treated as having a linear relationship with the result of interest, but the linear models Dawes studied did not have that refinement. Ideally, the weights should be carefully determined so that each factor is given the appropriate importance.
And yet, despite the lack of validity and sophistication in the models used, they still outperformed human judgement on a wide variety of tasks even when the weights were assigned at random. The provisos here are that the factors should all be scaled to have approximately the same range, and they should all have a “monotonic” relationship with the outcome of interest. That means that for a factor that seems to be a helpful one, more of it should always be better than less.
This sort of task usually involves using numbers already available, but in many business evaluations we have a lot of unstructured information and need to reduce it to factors to consider. Quantification may be tricky or unwelcome so a linear model may not be feasible. However, the next best thing is to take a break.
Specifically, having considered the problem as deeply and as broadly as you can, take five to 10 minutes away from the task, resting or doing something else, then come back to it and make your selection quickly based on how you feel. This seems to help people reflect all the factors rather than just the last factor they were thinking about, and that helps with the overall quality of evaluations.
This only helps if there is no over-riding factor that allows you to conclude without considering all or most of the factors, and benefits may be more likely when you are not really expert in that particular evaluation. For example, a medical expert working on a complex diagnosis may be better off taking a rest then thinking carefully again rather than just trusting to gut.
Guessing, however scientific, will not be better than well informed, carefully considered, quantified analysis. However, guessing is needed so often we need to be good at it
Download pdf article
The ICAEW Library & Information Service provides full text access to leading business, finance and management journals.
Further reading on business predictions and forecasting is available through the articles below.
Can't find what you're looking for?
The ICAEW Library can give you the right information from trustworthy, professional sources that aren't freely available online. Contact us for expert help with your enquiries and research.
- 13 Jun 2017 (12: 00 AM BST)
- First published
- 15 Nov 2022 (12: 00 AM GMT)
- Page updated with Further reading section, adding related articles on business predictions and forecasting. These additional articles provide fresh insights, case studies and perspectives on this topic. Please note that the original article from 2017 has not undergone any review or updates