Some firms have not found it easy to interpret the requirements of International Standard on Quality Management (UK) 1 (ISQM1) and scale them to fit. In this second article, we’ll look at what to do with your monitoring findings, including root cause analysis (RCA), but it is not a detailed guide to RCA. The first article covered how and what to monitor and the third will look at remedial action, assessing its effectiveness and annual evaluation.
We are grateful to the firms that took the time to share their experiences to help us produce these articles.
Evaluating your findings
Findings could come from a range of sources, including whole firm review work. However, we’ll assume that most, if not all, of your findings stem from your cold file reviews (as was the case with the firms we contacted), so we’ll focus on those.
ISQM1 says you should evaluate the findings to determine whether deficiencies exist and investigate the root cause(s) of any deficiencies you have identified. ‘Deficiency’ is a defined term in ISQM1, meaning something is wrong with your quality management system. You will need to use your judgement to decide whether a finding, or perhaps a combination of findings, is significant enough to be a deficiency, and therefore whether you need to investigate root causes.
The firms we contacted have chosen different ways to evaluate their findings. As you might expect, all of them already had established methods for assessing the seriousness of the findings from their cold file reviews, so the requirement for evaluation of findings is not new. Most of the firms used a file grading system, similar to that used by the ICAEW Quality Assurance Department (QAD). One firm graded individual findings using a traffic-light approach (again like the QAD), while in other cases, external reviewers (or the firm) had clearly highlighted the key findings or themes.
Which findings need root cause analysis (RCA)?
The largest firms have had formalised RCA processes for some years. While QAD does now ask the firms it visits to consider root causes of any key findings, until now firms have not had to build a formal RCA process into their systems. Interestingly, the firms we contacted had set the threshold for carrying out RCA on their cold file review findings at differing levels as shown here:
- on significant and common findings arising from lower graded files (eg, C and D graded);
- on significant findings on D-graded files;
- on all significant findings, whatever the file grade; and/or
- on all red findings (under their traffic-light approach).
Some of the firms had plans or policies to do RCA on good files in an effort to identify the factors contributing to positive outcomes, although only one firm was doing this in practice.
A couple of the firms were still developing their RCA policies, although they had not recently had any very significant findings to deal with, or else had come to a view on root causes without having a formalised process. Even if you are not currently seeing any significant deficiencies, you do need to have a proper plan in place for carrying out RCA should the need arise.
Approach to RCA – horses for courses
There was some variation in the way firms approached RCA, although the basic process was the same: to ask ‘why’ repeatedly.
One firm used the third party who had performed the cold file reviews to do their RCA as they felt it helped to demonstrate objectivity. The process consisted of separate interviews with the responsible individual (RI) and with the manager (within a month of finalising the cold file reviews), resulting in a report back to the firm.
The other firms did their RCA in-house. In one firm, members of the technical team did the RCA, initially asking the RI/manager to reflect on the findings (after locking the file grade), followed by discussions after a couple of days.
Other firms had taken a less structured approach, typically asking the RI/manager to submit their thoughts on root causes. One firm explained how it favoured a less regimented process as it seemed less adversarial and better reflected a culture of the partners ‘being in it together’.
More than one firm mentioned how a positive ‘no-blame’ environment can help to encourage openness and reassure RIs and staff who might otherwise be concerned about the implications for their standing within the firm.
More generally, all the firms were looking for any common themes and exploring the underlying causes, although not necessarily badging this as RCA.
There is no single right way to do RCA. All of these approaches can work. It is up to you to set out your policy and apply it consistently. You might choose to do more formal RCA on more serious findings, and something less formal on less serious findings, but still make it part of your RCA policy. Some training in RCA for those tasked with carrying it out would be advisable, so this would be something to include in your CPD plans. Some of the firms we contacted had shared their guidance with all audit staff to aid transparency.
Keeping a record
You will need to be able to demonstrate what you’ve done. You should retain evidence of your evaluation of findings and which findings you judge to be deficiencies. The results of your cold file reviews, together with any grading, should provide the evidence to support your evaluation. You also need to retain evidence of your RCA and the root causes you have identified.
Not rocket science
RCA is not rocket science. It is continuing to ask ‘why’ until you can’t go any further. While large firms may need sophisticated systems, the process can be straightforward in smaller firms. The smaller the firm, the easier and quicker it will be to get to root causes, which sometimes may be obvious. The chances are, whatever findings have come out of your cold file reviews, that you have always given some thought about the causes in order to address them. However, you may not have asked ‘why’ enough times or had a proper policy, or you may not have thought widely enough about potential causes, which could include behavioural factors. A number of the firms we contacted stressed the importance of creating a positive ‘no-blame’ culture around RCA, Remember, you don’t have to do RCA on all findings, and if you have no significant issues coming out of your cold file reviews, you may not need to do very much at all.
You can find more detailed guidance on root cause analysis on ICAEW’s web page: Using root cause analysis
Plus look out for the third article in this series which takes a look at remedial action, assessing effectiveness and annual evaluation.
You can find the first article in the series about how and what to monitor in Regulatory and Conduct News.
Help and guidance
ICAEW has a range of resources available to help you with ISQM1 compliance. Visit our hub or contact our Technical Advisory Team for help in this area.