Suitable criteria

What are criteria?

Assurance engagements require the practitioner to express an overall conclusion on the subject matter assessed in reference to specified criteria. Criteria also assist the parties to the engagement and agreed recipients of the assurance report to understand how the practitioner has evaluated the subject matter to reach his conclusion. Criteria are dependent on the subject matter and may be already established or developed for a specific engagement.

Criteria may be developed specifically for the engagement where there are no suitable established criteria. In this case, the practitioner considers whether specifically developed criteria are ‘fit for the purpose’ of the engagement using characteristics discussed below. In certain circumstances, the practitioner may also consider consulting with the responsible party and, where appropriate, the users, to ensure that the criteria meet their needs before proceeding with an engagement.

Criteria need to be available to all the addressees identified in the assurance report. Established criteria are often publicly available. If the criteria are not publicly available, for example because they are in the terms of a contract, this would affect who can access the assurance report. For ISAE 3000 (Revised) reporting it is a precondition for accepting the engagement that the criteria that the practitioner expects to be applied in the preparation of the subject matter information will be available to the intended users (ISAE 3000 (Revised) 24 (b)(iii))

ISAE 3000 (Revised) defines criteria as: “The benchmarks used to measure or evaluate the underlying subject matter". The "applicable criteria" are the criteria used for the particular engagement. Criteria are the benchmarks used to measure or evaluate the underlying subject matter. Criteria can be formal, for example in the preparation of financial statements, the criteria may be International Financial Reporting Standards or International Public Sector Accounting Standards.

When reporting on the operating effectiveness of internal controls, the criteria may be based on an established internal control framework or individual control objectives specifically designed for the purpose. Alternatively, when reporting on compliance, the criteria may be the applicable law, regulation or contract. Examples of less formal criteria are an internally developed code of conduct or an agreed level of performance (such as the number of times a particular committee is expected to meet in a year).

Suitable criteria

Suitable criteria are required for reasonably consistent measurement or evaluation of an underlying subject matter within the context of professional judgment. Without the frame of reference provided by suitable criteria, any conclusion is open to individual interpretation and misunderstanding. Suitable criteria are context-sensitive, that is, relevant to the engagement circumstances. Even for the same underlying subject matter there can be different criteria, which will yield a different measurement or evaluation.

For example, one of the criteria a measurer or evaluator might select as a measure of the underlying subject matter of customer satisfaction is the number of customer complaints resolved to the acknowledged satisfaction of the customer, while another measurer or evaluator might select the number of repeat purchases in the three months following the initial purchase.

Further, criteria may be suitable for a particular set of engagement circumstances, but may not be suitable for a different set of engagement circumstances. For example, reporting to governments or regulators may require the use of a particular set of criteria, but these criteria may not be suitable for a broader group of users.

Suitable criteria

Suitable criteria are required for reasonably consistent measurement or evaluation of an underlying subject matter within the context of professional judgment. Without the frame of reference provided by suitable criteria, any conclusion is open to individual interpretation and misunderstanding. Suitable criteria are context-sensitive, that is, relevant to the engagement circumstances. Even for the same underlying subject matter there can be different criteria, which will yield a different measurement or evaluation.

For example, one of the criteria a measurer or evaluator might select as a measure of the underlying subject matter of customer satisfaction is the number of customer complaints resolved to the acknowledged satisfaction of the customer, while another measurer or evaluator might select the number of repeat purchases in the three months following the initial purchase.

Further, criteria may be suitable for a particular set of engagement circumstances, but may not be suitable for a different set of engagement circumstances. For example, reporting to governments or regulators may require the use of a particular set of criteria, but these criteria may not be suitable for a broader group of users.

Characteristics of criteria

Suitable criteria, as set out in the IAASB's Amended International Framework for Assurance Engagements, exhibit the following characteristics:

• Relevance: Relevant criteria contribute to conclusions that assist decision-making by the intended users of the assurance report.
• Completeness: Criteria are sufficiently complete when relevant factors that could affect the conclusions in the context of the engagement circumstances are not omitted. Complete criteria include, where relevant, benchmarks for presentation and disclosure, or where it supports the fair description of systems and controls in operation.
• Reliability: Reliable criteria allow reasonably consistent evaluation or measurement of the subject matter including, where relevant, presentation and disclosure, when used in similar circumstances by similarly qualified practitioners.
• Neutrality: Neutral criteria contribute to conclusions that are free from bias.
• Understandability: Understandable criteria contribute to conclusions that are clear, comprehensive, and not subject to significantly different interpretations.

Vague descriptions of expectations or judgments of an individual’s experiences do not constitute suitable criteria.

The relative importance of each of the characteristics when assessing the suitability of criteria to a particular engagement is a matter of professional judgment. The suitability of criteria is not affected by the level of assurance, that is, if criteria are unsuitable for a reasonable assurance engagement, they are also unsuitable for a limited assurance engagement, and vice versa.

Established criteria

Established criteria tend to be formal in nature, but the degree of formality depends on the subject matter. Criteria may be prescribed by law or regulation, or issued by authorized or recognized bodies of experts that follow a transparent due process (established criteria). 

Criteria in areas, such as compliance with legal or regulatory requirements, may be widely recognised, either because they are available to the public or because there is an established standard, for example, ISO/IEC 27001 (information security management) and the COSO framework (internal control). It should, however, be noted that neither of these are legal or regulatory requirements, and neither are suitable criteria for assurance on their own. performance criteria may be set out in contractual arrangements as agreed with the users.

The practitioner considers the suitability of the criteria, even where established criteria are available, to ensure their relevance to the needs of the intended users of the assurance report. It is not unusual for established criteria to be customised to meet users’ needs and/ or to make them suitable for assurance. For example, ISO/IEC 27001 provides a framework for managing information security, but this should be converted to a set of control objectives that are specific and relevant to the entity to make it suitable for assurance.

Standards exist to provide guidance on criteria for assurance over system and controls relating to financial reporting processes (ie ISAE 3402 and AAF 01/06). These criteria are provided in the applicable standard and are not required to be duplicated in management’s statement or in the assurance report.

Where assurance is required on activities, processes, systems and controls which are not relevant to financial reporting, the characteristics for defining criteria outlined above should still be considered. ITF 01/07 provides a framework and guidance on criteria for IT and bureau service.

Otherwise ISAE 3000 (Revised) should be used and assessment criteria linked to control objectives should be defined. These criteria will need to be made available to the user through inclusion in management’s assertion and can then be referred to in the practitioner’s assurance report. It is likely that such criteria will be loosely based on the ISAE 3402 criteria and the changes needed may be relatively subtle.

Developing criteria

Where regulation/law is not specific enough to use as criteria, the regulation/law can be developed into criteria through a management basis of preparation explaining how management have applied it to the entity in question and why. The opinion of the practitioner would then refer to both the regulation and the basis of preparation as criteria.

When considering whether requirements of regulation or law are sufficiently complete and reliable to use as criteria in an assurance engagement the practitioner might reasonably consider whether it would be possible for two materially different presentations of the same subject matter to be considered to be "properly prepared" in accordance with that regulation or law.

Where law or regulation alone could allow materially different versions of the same subject matter to be considered to be "properly prepared", the law or regulation itself is likely to be too vague to use as criteria for assurance and a management basis of preparation will need to be devised as criteria for assurance reporting.

Other criteria may be specifically developed for the purpose of preparing the subject matter information in the particular circumstances of the engagement.

Whether criteria are established or specifically developed affects the work needed to assess their suitability for a particular engagement, for example, in the absence of indications to the contrary, established criteria are presumed to be suitable if they are relevant to the intended users’ information needs.

Availability of criteria

Criteria need to be available to the intended users to allow them to understand how the underlying subject matter has been measured or evaluated. Criteria are made available to the intended users in one or more of the following ways:

• publicly;
• through inclusion in a clear manner in the presentation of the subject matter information;
• through inclusion in a clear manner in the assurance report; and
• by general understanding, for example the criterion for measuring time in hours and minutes.

Criteria may also be available only to specific intended users, for example the terms of a contract, or criteria issued by an industry association that are available only to those in the industry because they are relevant only to a specific purpose.

Criteria need to be available to user entities and their auditors to enable them to understand the basis for the service organisation's assertion about the fair presentation of management's description of the service organisation's system, the suitability of the design of controls that address control objectives stated in the description of the system and, in the case of a type two report, the operating effectiveness of such controls.

• Read SSAE 18

Example criteria

ISAE 3402 criteria	Criteria devised for assurance on compliance with a code of behaviour (ISAE 3000 Revised)
The description is fairly presented if it:	The description is fairly presented if it:
Presents how the service organisation's system was designed and implemented including, as appropriate, the matters identified in paragraph 16(a)(i)-(viii).	Presents how the entity’s policies and processes in respect of its compliance with the Code of Behaviour were designed and implemented including any specific matters of concern to users.
In the case of a type two report, includes relevant details of changes to the service organisation's system during the period covered by the description.	Includes relevant details of changes to the entity’s policies and processes during the period covered by the description.
Does not omit or distort information relevant to the scope of the service organisation's system being described, while acknowledging that the description is prepared to meet the common needs of a broad range of user entities and may not, therefore, include every aspect of the service organisation's system that each individual user entity may consider important in its own particular environment.	Does not omit or distort information relevant to the scope of the policies and processes being described, while acknowledging that the description is prepared to meet the common needs of a broad range of users and may not, therefore, include every aspect of the entity’s policies and processes that each individual user entity may consider important in its own particular environment.