The role of Internal Audit in ethical AI

Many technological advances begin as promising and disruptive, but often fade, with limited commercial availability and adoption. However, the trajectory of AI is distinct. Despite having existed for many years, AI is currently undergoing significant advancements in terms of use cases and adoption, impacting various business sectors and aspects of everyday life.

However, as AI progresses and gains wider acceptance, ethical considerations become increasingly significant. In the social context, AI introduces ethical challenges across various domains, including democracy (e.g., the use of misinformation and targeting to influence election results), justice (e.g., assessing individuals' likelihood to commit a crime), or diversity and inclusion (e.g., profiling for social benefits and equal access to technology).

Beyond the social sphere, AI use also raises numerous ethical considerations in a corporate context, many of which have societal repercussions. This article explores some of these considerations, the associated risks, and the role internal auditors can play in promoting ethical conduct.

• Fairness and safety:
  One of the key features making AI models valuable is their decision-making ability. However, these decisions must be fair, inclusive, and unbiased. Biases may arise from improperly selected or unrepresentative datasets. Careful consideration should be given to the use of proxies to avoid exclusion based on socioeconomic, racial, or minority factors.
  
  An illustrative case involves a credit scoring model utilizing a zip code as a proxy to estimate income. This approach may result in a model that inadvertently excludes significant socioeconomic, racial, or minority groups from accessing lending opportunities. Similarly, in the realm of hiring processes within a company, an AI model programmed to favor candidates based solely on traits exhibited by successful existing employees could unintentionally discriminate against clusters of candidates with different characteristics.
  
  Beyond the statistical considerations inherent in AI models, it is crucial to exercise caution to prevent the perpetuation of social biases. For instance, if a specific group is treated with social bias, this biased treatment becomes reflected in the data used to train the model, potentially leading to the development of a biased model.
  
  Additionally, sectors like autonomous vehicles and healthcare highlight safety concerns in AI use. Models making split-second decisions should prioritize values like the protection of human life and safety.
  
  
• Privacy:
  Another critical aspect that requires careful consideration in the adoption of AI models is privacy. While individuals often consent to share personal data on social media platforms, the dynamics change within a corporate environment where obtaining consent, even if given, may be perceived as coercive. Take, for instance, the scenario of workplace surveillance in a company, involving the monitoring and analysis of data from calls, messages, or biometric sources to assess employees' conduct and performance.
  
  In instances where such surveillance becomes a widespread practice across the company, even if offered on an "opt-in" basis, employees might be reluctant to "opt out" due to concerns about potential implications on their performance assessments. Therefore, careful consideration must be given to obtaining explicit consent and understanding the context of use for these models. Additionally, it is crucial to ensure that any data collected for training an AI model has the explicit consent of the subject for the specific purpose intended.
  
  
• Transparency and Explainability:
  The creation of complex AI models may result in opaque algorithms. Transparency and explainability are crucial to promote fairness and accountability. Knowing the decision-making parameters becomes essential, especially in scenarios such as hiring processes.
  
  Generative AI's ability to produce tailored subconscious messages raises concerns about subliminal influence, impacting voting or consumer behavior. Although regulatory frameworks may prohibit such messaging, the temptation for profit-driven companies exists.
  
  Learn more about ethics in AI in this explainer article on ‘AI Ethics: What are they really?’. 

The ethical considerations discussed could expose companies to significant risks, including regulatory breaches, fines, litigations, health and safety issues, and damage to reputation and consumer trust.

In this context, internal auditors, as ethics advocates, play a pivotal role in safeguarding company interests and promoting ethical conduct. Steps auditors could consider include:

• Ensure Adoption of AI Governance Framework
  Auditors should promote the adoption of a comprehensive AI governance framework that facilitates oversight and accountability in the development and deployment of AI systems.
  
  Such a framework could examine broader social impact of AI models, ensure alignment with company’s strategy and values, embed regulatory compliance control checks and examine aspects such as representativeness of data, algorithmic bias, consent for the use of data etc.
  
  To make it practical it could function as a checklist where specific questions could act as “go/ no-go” decision points and others could indicate modification of the model.
• Assess Models for Fairness and Transparency
  Further to the deployment of AI models, they should be monitored in terms of performance but also from an ethical perspective as well. Although the primary responsibility lies with the model owner, internal auditors could assist by conducting independent assessments examining also ethical considerations.
  
  An audit program to this end could include the following indicative steps:
  
  
  • Obtain a sample of decisions made by the model and assess the transparency and explainability of decision criteria,
  • Obtain the population of approved/ rejected applications and assess diversity and proportionate representation of social groups,
  • Obtain and analyze complaints from customers/stakeholders related to unfair treatment or bias, specifically scrutinizing the corresponding applications.
• Evaluate Privacy and Security of Data
  Assess the privacy and security protocols in place for handling data used in AI models. Ensure that data collection and usage align with privacy standards and safeguard against potential breaches.
  
  To this end, auditors could review a sample of data used for model training and ensure that data subjects have provided explicit consent for the specific purpose.
  
  
• Stay Informed on Regulatory and Risk Management
  Ensure adherence to mandatory requirements and the incorporation of best practices to mitigate potential risks. As mentioned above such controls could be embedded in the governance framework, but internal auditors should stay abreast of developments in the area. The rapid pace of developments in this field necessitates vigilant monitoring to ensure that the company remains adaptive and responsive to emerging challenges and opportunities.
  
  A gap analysis is a practical way to examine a baseline adherence to regulatory stipulations and then monitor incremental changes.
  
  
• Educate Stakeholders on AI Risks and Ethics
  Take proactive measures to educate and raise awareness on the risks associated with AI and ethical considerations.

This could involve:

• • presentations on AI ethical considerations to all key stakeholders, from the Board of Directors to people developing AI models,
  • facilitation of workshops to discuss ethical AI aspects applicable to the company,
  • promotion, in cooperation with HR, of training courses to raise awareness.
    
    
• Advocate for "Human in the Loop" for Ethical Considerations
  Suggest the inclusion of a "human in the loop" for AI models that may raise ethical considerations. This ensures human judgment is incorporated, particularly in challenging cases where ethical implications need careful consideration.
  
  By way of example, credit applications above or below a threshold could be automatically approved or rejected, while applications in a grey score zone could be further assessed by credit officers.
  
  
• Offer Guidance on Ethical ConsiderationsAs auditors, leverage your structured approach to handling ethical issues by providing guidance on ethical considerations or dilemmas. Utilize your professional judgment, considering that auditors frequently engage with ethical matters.