As AI touches everything from our business to personal lives, we could be forgiven for thinking that how the technology is developed and deployed is already a ‘done deal’. But, as complaints of tech companies ‘marking their own homework’ grow louder with each new story of AI agents ‘going rogue’, could ethics finally have its moment in the grand tech debate?

Professor Edward Harcourt MBE, director of the Institute for Ethics in AI at the University of Oxford, is a speaker at ICAEW’s Global Ethics Day event which is streaming online on 23 October.

Professor Edward Harcourt MBE, director of the Institute for Ethics in AI at the University of Oxford, is a speaker at ICAEW’s Global Ethics Day event which is streaming online on 23 October.

Harcourt can't understand why AI buyers don’t put more pressure on tech companies to carry out reliability testing, but adds that testing is expensive, and given the probabilistic nature of AI, is not necessarily very reassuring.

“They'll say, ‘on this parameter, we've shown our product to be reliable in 92% of cases’. I'm not going to get on an airplane that's only reliable in 92% of cases, when eight planes [out of 100] are going to fall out of the sky, right?” he says. “The whole business of what reliability means for technology like AI is quite difficult to pin down.”

For Harcourt, there are three ethical factors that we should consider when it comes to AI. The first is philosophical, which sounds abstract but is in fact surprisingly relevant.

“There’s a 350-year-old philosophical mainstream idea that human beings are machines, and that reasoning is a mechanical process,” Harcourt says. “And if that's true, and somebody says, ‘Well, I've invented a slightly smarter machine than you, human,’ then there's absolutely no reason why we shouldn't be replaced by machines.

“The frontier labs rely on this very powerful philosophical mainstream. So unless we can push back against it, if AI says ‘shove over’, then we haven't got a leg to stand on.”

AI ethics is more than just how you train them

Harcourt’s second point is that AI ethics cannot be limited to conversations around training data. Much of the debate in AI and ethics is about the rules you teach models to get them to behave ethically.

Harcourt insists that’s a boring question, partly because it’s about what the right ethical rules are in the first place, which he argues could be debated forever. Beyond that, he says: “AI ethics has a lot to do with the whole ecosystem: the way in which human beings and artificial intelligences interact, not just which bits of human thought you train AI to mimic.”

One crucial feature of that ecosystem, he points out, is ‘reasonable dissent’. He says: “One of the great things about an open society is that it enables people to disagree without killing one another. How do you reproduce that in a world cohabited by humans and artificial intelligences?”

The rush for AI – at the expense of ethics

The third point involves risk appetite and regulation. AI is a seller’s market, says Harcourt, and professional liability has not adapted to it, despite issues of reliability.

He says: “If you're an accountancy firm auditing a company, it's up to you to sign off on that audit. And if you get stuff wrong, it's on you. You can't say: ‘Oh, well, I bought this AI tool and it hallucinated some data.’ It's your fault, and that's kind of surprising in a way, because you can sue somebody for selling you a drug that hasn’t been properly tested.”

Harcourt suggests that market pressure is so weak because firms are terrified of being outpaced in AI adoption by rivals. “We're exposing ourselves to extraordinary risks by this race to adopt,” he warns.

AI adopters do have the power to force the industry to take responsibility and assume liability for when the technology goes wrong, Harcourt insists. He has spoken to people in industry who are expecting some kind of disaster relating to AI.

Such an event may be necessary to instigate a sea change in how businesses approach AI adoption. He says: “If and when there is a disaster – a huge data theft for example – the financial incentives to adopt hand-over-fist will be very different.”

Harcourt thinks the market will then become more risk averse, and signal to AI providers that they want much more reliability testing before they're prepared to spend money. “A good situation to be in, but unfortunately, we may need to have the disaster first before that happens,” he says.

If you know you're adopting something inherently risky, the natural thing is to want to insure yourself against what goes wrong. A more mature market for AI insurance could provide clarity on those risks and act as another check and balance.

Harcourt concludes: “It'd be really good if you could quantify the risk, because then there would be another market pressure, namely keeping your insurance premiums down to make sure you didn't do something risky. But I hope there are people in the audience at the Global Ethics Day event who can tell me more about that than I know myself.”