April 2018: Richard Eckersley, head of assessment at ICAEW, describes the untold journey taken by each ICAEW CFAB and ACA Certificate Level exam, before you even enter the exam centre.
For each exam ICAEW has a team who are responsible for writing, reviewing and updating every question that is presented to you within the live exams.
The teams are formed of subject matter experts and their primary role is to write the exams you sit, ensuring that the technical content is accurate and in line with the learning materials, that the exams reflect the specified syllabus area weightings and that each test is reliable and balanced – so that every exam is fair.
The exam question development process has two key stages, drafting and moderation. In the drafting stage the writer will know the type of question they are writing (eg, multiple choice or scenario-based question) and the syllabus area and learning outcome that the question is linked to.
The writer will refer to the ICAEW style guide to ensure a consistent writing style and convention and will use their expert knowledge to draft each question and set the answer key and any distractors (incorrect answer options).
The next stage is moderation and a different subject matter expert reviews the question content and answer key, making further changes if needed.
Questions are then student moderated, where recently qualified students sit the questions under exam conditions. At this stage we also ask students to answer specific questions such as “Is the question clear, avoiding ambiguity?” and they provide feedback on each question. After testing, all results, feedback and statistical data is sent back to the exam teams to make any final amendments.
Quality assurance
Before exam questions can be published live, there is one last quality assurance check which focuses on the format of the questions. This is to ensure that they are consistent and that they display correctly on screen. The exam questions are then published live and are ready for exam delivery.
After you sit your exam, ICAEW will receive all the exam data. Our questions are under constant review and every response given to each question is used in statistical modelling. We have a dedicated statistician, ensuring that we have a comprehensive understanding of question performance – right down to how many students are incorrectly choosing B, compared to incorrectly choosing C, on a specific question. This data is reported to the exam teams on a regular basis for monitoring and assurance purposes.
Scenario questions
All ICAEW CFAB and Certificate Level exams are randomised, so it would be very unlikely that you would be presented with the same question as someone else in your exam room at the same time. So if you’re sitting Accounting, Principles of Taxation or Management Information you’ll answer one or more ‘scenario’ questions which are always presented first. All responses are recorded by the exam system and used to automatically generate a mark for your exam.
Your ICAEW CFAB or Certificate Level exam results are usually available within 24 hours and can be obtained through your training file at icaew.com/trainingfile or on our website at my.icaew.com/examsonline
So, when you arrive for your exam, you can be assured that all of your exam questions have been thoroughly tested and have gone through all these quality assurance processes, which ensures that the ACA continues to be a world-leading qualification.
If you’re studying for ICAEW CFAB click here for your dedicated exam resources and here for Certificate Level exam resources.