ICAEW.com works better with JavaScript enabled.

Suitable criteria

Suitable criteria are one of the five elements of an assurance engagement required by ISAE 3000 (Revised). This guidance provides a definition of criteria and their characteristics, examines what ensure criteria are suitable and provides examples of criteria.

What are criteria?

Assurance engagements require the practitioner to express an overall conclusion on the subject matter assessed in reference to specified criteria. Criteria also assist the parties to the engagement and agreed recipients of the assurance report to understand how the practitioner has evaluated the subject matter to reach his conclusion. Criteria are dependent on the subject matter and may be already established or developed for a specific engagement.

Buyer's guide to assurance on non-financial information

Find out more
Criteria may be developed specifically for the engagement where there are no suitable established criteria. In this case, the practitioner considers whether specifically developed criteria are ‘fit for the purpose’ of the engagement using characteristics discussed below. In certain circumstances, the practitioner may also consider consulting with the responsible party and, where appropriate, the users, to ensure that the criteria meet their needs before proceeding with an engagement.

Criteria need to be available to all the addressees identified in the assurance report. Established criteria are often publicly available. If the criteria are not publicly available, for example because they are in the terms of a contract, this would affect who can access the assurance report. For ISAE 3000 (Revised) reporting it is a precondition for accepting the engagement that the criteria that the practitioner expects to be applied in the preparation of the subject matter information will be available to the intended users (ISAE 3000 (Revised) 24 (b)(iii))

ISAE 3000 (Revised) defines criteria as: “The benchmarks used to measure or evaluate the underlying subject matter". The "applicable criteria" are the criteria used for the particular engagement. Criteria are the benchmarks used to measure or evaluate the underlying subject matter. Criteria can be formal, for example in the preparation of financial statements, the criteria may be International Financial Reporting Standards or International Public Sector Accounting Standards.

When reporting on the operating effectiveness of internal controls, the criteria may be based on an established internal control framework or individual control objectives specifically designed for the purpose. Alternatively, when reporting on compliance, the criteria may be the applicable law, regulation or contract. Examples of less formal criteria are an internally developed code of conduct or an agreed level of performance (such as the number of times a particular committee is expected to meet in a year).

Suitable criteria

Suitable criteria are required for reasonably consistent measurement or evaluation of an underlying subject matter within the context of professional judgment. Without the frame of reference provided by suitable criteria, any conclusion is open to individual interpretation and misunderstanding. Suitable criteria are context-sensitive, that is, relevant to the engagement circumstances. Even for the same underlying subject matter there can be different criteria, which will yield a different measurement or evaluation.

For example, one of the criteria a measurer or evaluator might select as a measure of the underlying subject matter of customer satisfaction is the number of customer complaints resolved to the acknowledged satisfaction of the customer, while another measurer or evaluator might select the number of repeat purchases in the three months following the initial purchase.

Further, criteria may be suitable for a particular set of engagement circumstances, but may not be suitable for a different set of engagement circumstances. For example, reporting to governments or regulators may require the use of a particular set of criteria, but these criteria may not be suitable for a broader group of users.

Characteristics of criteria

Suitable criteria, as set out in the IAASB's Amended International Framework for Assurance Engagements, exhibit the following characteristics:

  • Relevance: Relevant criteria contribute to conclusions that assist decision-making by the intended users of the assurance report.
  • Completeness: Criteria are sufficiently complete when relevant factors that could affect the conclusions in the context of the engagement circumstances are not omitted. Complete criteria include, where relevant, benchmarks for presentation and disclosure, or where it supports the fair description of systems and controls in operation.
  • Reliability: Reliable criteria allow reasonably consistent evaluation or measurement of the subject matter including, where relevant, presentation and disclosure, when used in similar circumstances by similarly qualified practitioners.
  • Neutrality: Neutral criteria contribute to conclusions that are free from bias.
  • Understandability: Understandable criteria contribute to conclusions that are clear, comprehensive, and not subject to significantly different interpretations.

Vague descriptions of expectations or judgments of an individual’s experiences do not constitute suitable criteria.

The relative importance of each of the characteristics when assessing the suitability of criteria to a particular engagement is a matter of professional judgment. The suitability of criteria is not affected by the level of assurance, that is, if criteria are unsuitable for a reasonable assurance engagement, they are also unsuitable for a limited assurance engagement, and vice versa.

Established criteria

Established criteria tend to be formal in nature, but the degree of formality depends on the subject matter. Criteria may be prescribed by law or regulation, or issued by authorized or recognized bodies of experts that follow a transparent due process (established criteria). 

Criteria in areas, such as compliance with legal or regulatory requirements, may be widely recognised, either because they are available to the public or because there is an established standard, for example, ISO/IEC 27001 (information security management) and the COSO framework (internal control). It should, however, be noted that neither of these are legal or regulatory requirements, and neither are suitable criteria for assurance on their own. performance criteria may be set out in contractual arrangements as agreed with the users.

The practitioner considers the suitability of the criteria, even where established criteria are available, to ensure their relevance to the needs of the intended users of the assurance report. It is not unusual for established criteria to be customised to meet users’ needs and/ or to make them suitable for assurance. For example, ISO/IEC 27001 provides a framework for managing information security, but this should be converted to a set of control objectives that are specific and relevant to the entity to make it suitable for assurance.

Standards exist to provide guidance on criteria for assurance over system and controls relating to financial reporting processes (ie ISAE 3402 and AAF 01/06). These criteria are provided in the applicable standard and are not required to be duplicated in management’s statement or in the assurance report.

Where assurance is required on activities, processes, systems and controls which are not relevant to financial reporting, the characteristics for defining criteria outlined above should still be considered. ITF 01/07 provides a framework and guidance on criteria for IT and bureau service.

Otherwise ISAE 3000 (Revised) should be used and assessment criteria linked to control objectives should be defined. These criteria will need to be made available to the user through inclusion in management’s assertion and can then be referred to in the practitioner’s assurance report. It is likely that such criteria will be loosely based on the ISAE 3402 criteria and the changes needed may be relatively subtle.

Developing criteria

Where regulation/law is not specific enough to use as criteria, the regulation/law can be developed into criteria through a management basis of preparation explaining how management have applied it to the entity in question and why. The opinion of the practitioner would then refer to both the regulation and the basis of preparation as criteria.

When considering whether requirements of regulation or law are sufficiently complete and reliable to use as criteria in an assurance engagement the practitioner might reasonably consider whether it would be possible for two materially different presentations of the same subject matter to be considered to be "properly prepared" in accordance with that regulation or law.

Where law or regulation alone could allow materially different versions of the same subject matter to be considered to be "properly prepared", the law or regulation itself is likely to be too vague to use as criteria for assurance and a management basis of preparation will need to be devised as criteria for assurance reporting.

Other criteria may be specifically developed for the purpose of preparing the subject matter information in the particular circumstances of the engagement.

Whether criteria are established or specifically developed affects the work needed to assess their suitability for a particular engagement, for example, in the absence of indications to the contrary, established criteria are presumed to be suitable if they are relevant to the intended users’ information needs.

Availability of criteria

Criteria need to be available to the intended users to allow them to understand how the underlying subject matter has been measured or evaluated. Criteria are made available to the intended users in one or more of the following ways:

  • publicly;
  • through inclusion in a clear manner in the presentation of the subject matter information;
  • through inclusion in a clear manner in the assurance report; and
  • by general understanding, for example the criterion for measuring time in hours and minutes.

Criteria may also be available only to specific intended users, for example the terms of a contract, or criteria issued by an industry association that are available only to those in the industry because they are relevant only to a specific purpose.

Criteria need to be available to user entities and their auditors to enable them to understand the basis for the service organisation's assertion about the fair presentation of management's description of the service organisation's system, the suitability of the design of controls that address control objectives stated in the description of the system and, in the case of a type two report, the operating effectiveness of such controls.

Example criteria

ISAE 3402 criteria Criteria devised for assurance on compliance with a code of behaviour (ISAE 3000 Revised)

The description is fairly presented if it:

The description is fairly presented if it:

Presents how the service organisation's system was designed and implemented including, as appropriate, the matters identified in paragraph 16(a)(i)-(viii).

Presents how the entity’s policies and processes in respect of its compliance with the Code of Behaviour were designed and implemented including any specific matters of concern to users.

In the case of a type two report, includes relevant details of changes to the service organisation's system during the period covered by the description.

Includes relevant details of changes to the entity’s policies and processes during the period covered by the description.

Does not omit or distort information relevant to the scope of the service organisation's system being described, while acknowledging that the description is prepared to meet the common needs of a broad range of user entities and may not, therefore, include every aspect of the service organisation's system that each individual user entity may consider important in its own particular environment.

Does not omit or distort information relevant to the scope of the policies and processes being described, while acknowledging that the description is prepared to meet the common needs of a broad range of users and may not, therefore, include every aspect of the entity’s policies and processes that each individual user entity may consider important in its own particular environment.

Examples

Compliance with contractual agreements
Example subject matter Evaluation criteria
Allocation of royalties The contractual clauses. May need to be supplemented by agreements with the contracting parties as to interpretations of clauses.
Shared profits, shared cost saving Joint venture agreements in relation to cost or profit sharing arrangements. May need to be supplemented with internally developed basis of calculation for areas of management judgement, protocols agreed between participants describing application of clauses of agreements.
Environmental information
Example subject matter Evaluation criteria
Greenhouse gas emissions Greenhouse Gas protocol to quantify greenhouse gas emissions. Externally imposed or internally devised basis of calculation of emissions
Risk assessment processes Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achieving compliance with Equator principles: when evaluating social and environmental risks in project financing for emerging markets.
Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achievement of principles of The Occupational Health and Safety Assessment Series 18000 to evaluate health and safety risks.
Ethics and behaviour
Example subject matter Evaluation criteria
Anti-bribery procedures Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achievement of recommendations in Ministry of Justice guidance in relation to the Anti-Bribery and Corruption Act 2010 or OECD guidance on anti-bribery & corruption.
Ethical investment arrangement and its function
Standards as defined by independent bodies such as Transparency International and UN PRI.

 

 

Financial processes
Example subject matter Evaluation criteria
Cost saving achieved Gershon guidelines on cost savings for certain public sector bodies.
Control over client assets Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achieving compliance with requirements of Trust deeds on managing client funds or principles contained in FCA CASS Rules.
Pillar III solvency calculations Basel report in relation to Pillar III solvency calculations.
Compliance with FSA rules FSA Handbook rules and guidance in relation to FSA returns.

 

 

Governance, strategy and management processes
Example subject matter Evaluation criteria
Governance arrangement Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achievement of objectives set by standards defining bodies such as the OECD.
Compliance with the Stewardship Code Criteria in Stewardship Code supplement to AAF 01/06.
Management processes Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to process objectives set by the company.
Information technology

This includes: information flows and security.

Example subject matter Evaluation criteria
Data and information security AICPA SOC 2 and 3 frameworks for data centres and web trust
IT governance arrangement Various IT Governance references in ICAEW ITF 01/07

 

 

Management information flows
Example subject matter Evaluation criteria
Performance of internally developed processes and controls Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402. 

Documented internally developed procedures for managing and reporting on the effectiveness of the management information.

 

 

Operations and projects

These may be performed by third parties...

Example subject matter Evaluation criteria
Internal processes and controls Internally developed criteria, based on those for fairness of description, suitability of design and operating effectiveness in ISAE 3402, linked to control objectives agreed between the service and user organisations.
Internal controls over financial reporting AICPA SOC 1 framework
Internal controls 
Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402 linked to process and control objectives set by professional bodies, e.g. ICAEW AAF 01/06 on investment operations.
  Criteria developed with reference to the process and control requirements set by regulatory bodies such as the FCA.

 

 

Quantitative information

This includes: financial information and performance measures, such as KPIs.

Example subject matter Evaluation criteria
Financial statements
International Financial Reporting Standards (IFRSs)
Performance of internally developed processes and controls
Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402.
Quality of performance Internally developed criteria for fairness of presentation of description of performance, Pre-defined bases of preparation and data measurement methods for quantitative performance indicators.
Achievement of operational/performance target Commonly used definitions of KPIs, internally defined bases of calculation.
  Sponsor defined KPIs; eg, for performance targets set by a Government Department for an arms-length body

 

 

Regulatory processes and compliance

This includes: information flows and security.

Example subject matter Evaluation criteria
Compliance with regulatory rules Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achieving compliance with requirements of UK Government (or EU) Regulation together with any related guidance issued by the regulator.
Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402, with reference to achieving compliance with requirements of any specific regulatory undertakings e.g. issued by the Competition Commission following an investigation.
Compliance with other rules Internally developed criteria, based on those for fairness of description of performance of processes and controls, suitability of design and operating effectiveness in ISAE 3402 with reference to achieving compliance with requirements of detailed rules of the industry association.

 

Risk management systems and processes

Example subject matter Evaluation criteria
Business risk management arrangements Company's own criteria developed based on Turnbull report and International Standard for Risk Management AS/NZS ISO 31000:2009.

 

 

ICAEW's assurance resource

This page is part of ICAEW’s online assurance resource, which replaces the Assurance Sourcebook.

Find out more.