Executing effective substantive analytical procedures

In the first article on substantive analytical procedures (SAPs), we focused on its design, covering the objective, the value being tested, the underlying relationship, the reliability of inputs and the method of calculation. These elements form the foundation of an effective SAP and covered the first five parts of the SAP blueprint

In this article we move from design to execution. We consider how precision can be improved through disaggregation; how appropriate thresholds should be set; how differences must be evaluated; and how a clear and defensible conclusion should ultimately be documented. Together, these steps determine whether a SAP genuinely provides sufficient, appropriate audit evidence. It is important to note that when the approach to a significant risk consists only of substantive procedures, those procedures shall include tests of details in line with ISA (UK) 330.

Disaggregation

Disaggregation involves splitting the account balance into sub-populations to help reduce ‘noise’ in setting our expectation. Noise, in this context, refers to how true our assumptions used to build the SAP relate to the population. For example, when performing a rental income SAP, we often make the assumption that all additions and disposals of property were made halfway through the year. This may not hold true where a significant new development was completed very early or very late in the year. As such, we may opt to split the population of rental properties into one containing the new development and another containing everything else. This allows us to increase the SAP’s precision, albeit at the cost of additional work or data requirements. 

It is important to note that disaggregation in this way also creates incremental aggregation risk. Therefore, we can’t blindly apply the same tolerance to each portion of the SAP as we would for the population as a whole. We need to consider reducing the tolerance appropriately. Each firm will have its own methodology in this area, so I can’t provide detail on how to do this mechanically, other than to encourage you to be aware of when you’re adding aggregation risk in this way and to consider what needs to be done in response.

Precision and threshold for investigation

One of the most challenging points for an engagement team is how to set an appropriate threshold for a SAP (or an acceptable difference between our expectation and the entity’s value). Unfortunately, the value to be used isn’t something that is immediately obvious. Some firms have a prescriptive methodology in this area, but the audit standards don’t really provide much guidance. 

What I can do is emphasise some general principles. Firstly, the SAP must be sufficiently precise to detect a misstatement at the assertion level. It can’t be higher than performance materiality (PM) as a start. So, we’re in the range of zero to PM (or if we have a disaggregated SAP, it might be lower than PM), but this isn’t hugely helpful. Some may argue that it doesn’t matter, as if we’ve shown the difference to be less than PM, we’re okay. I think it’s more nuanced than that though.

We must be conscious of why we set a threshold in the first place. It is unrelated to materiality. The threshold is set because we know that our audit procedure is imprecise. We know we’re not going to get the exact answer, because we use assumptions within our calculation that, while reasonably representative of what’s in the population, won’t hold perfectly true for all items. So, the question really is: “how much allowance should we give to account for the imprecision in our own procedure?” 

If we’re expecting our procedure to be bang on the money, perhaps the threshold is £1. For example, a depreciation SAP for an entity with no fixed asset additions or disposals during the year is likely to be quite precise. If, on the other hand, there were significant additions during the year, the use of an assumption in the SAP that all items were purchased halfway through the year has the potential to be quite imprecise (and therefore a higher threshold would be set). 

Setting a low threshold for an imprecise procedure is likely to result in an inability to conclude. Equally, if a high threshold is set for a precise procedure, it may ignore misstatements above our trivial threshold which we would be required to aggregate as part of our audit conclusion. So, the threshold must be representative of the inherent precision of the procedure we are performing.

The next point to make is that higher risk balances should attract a lower precision setting. This follows the principle that where the risk is higher, we require more persuasive audit evidence. For a low-risk balance, we may be happy with setting the tolerance to equal PM. However, for a high-risk balance, we may decide that we’re not happy with this being more than, say, 25% of PM, or 10% of the account balance (if a lower value). Following this thinking, we may end up in a scenario where the achievable precision of our SAP doesn’t meet the level of precision we desire according to our assessed risk within the population. In this case, we may need to turn to disaggregation to increase precision or perform other audit procedures instead of, or in addition to, the SAP.

Developing the expectation

Coming to the actual performance of the SAP, the key points to remember are to:

• apply the SAP as described. Where the SAP design has changed, this needs to be reflected in the audit documentation;
• document any assumptions or adjustments made when applying the model. Document any transformations made to data before using it in the SAP; and
• reconsider the method and the threshold where new information is uncovered.

Evaluation of differences

It is important to investigate differences above the threshold using procedures that go beyond simple enquiry. By definition, if the difference is outside of the threshold, it is an unacceptable difference and cannot simply be explained away. This is a critical mistake that some audit teams make. 

If we obtain information from management about why the difference is beyond the threshold, corroborative evidence should be obtained for any explanation and it would be appropriate to incorporate this into our SAP (if reasonable) and assess the impact. Teams also need to critically assess any differences below the threshold to determine whether they appear unusual and require further investigation. 

Conclusion and documentation

Finally, we need to form an appropriate conclusion to our SAP. Use of language such as “appears reasonable” is not appropriate. We should conclude on whether the SAP provides sufficient, appropriate audit evidence over the relevant assertion(s).

We should clearly cross-reference to follow-up procedures we’ve undertaken and their results.

Finally, document information in relation to all 10 elements of the blueprint.

Graham Gardner, Audit Partner and Head of Audit Quality, Kreston Reeves