In 2024, an Australian construction firm was tricked into paying an invoice worth AU$900,000 (around £480,000) to what its accounts team thought was a legitimate supplier. The invoice came from a genuine email address, but the supplier’s system had been compromised and the account details on the invoice changed. Instead of paying the supplier it owed money to, the funds went to criminals, before the firm’s bank intervened to recoup most of the amount. 

The potential for invoice fraud has always existed (how to deal with traditional invoice fraud is discussed here), but the role that artificial intelligence (AI) is playing is a growing concern for finance professionals. 

AI helps criminals to target businesses with convincing looking invoices in large volumes, hoping accounts payable teams will settle these without the necessary due diligence. Research by Medius suggests 44% of businesses have been targets for invoice fraud, while 53% of finance professionals have seen attempted deepfake scamming attacks.

Arun Chauhan, founder of Tenet Law, says invoice fraud can take various guises. Historically, that might manifest as someone claiming to be a supplier and asking for account details to be changed. More frequently, fraudsters are hacking into a genuine supplier’s systems and sending invoices from a legitimate address, as in the example above. 

As a rule, Chauhan says businesses are very conscious about the potential threat of the first scenario, but less so around the second. “When combined with AI, this can be pretty powerful,” he says. “AI-generated invoices can have logos, layouts and access to supporting documents that look genuine.” 

The threat of AI

AI can also be used to produce convincing supporting evidence in cases where accounts payable teams challenge invoices or requests to change payment details. Chauhan gives the example of an accounts payable team that questions an invoice supposedly from an existing supplier, where there is no record of the work being agreed. “Someone can use AI to create an email thread using names, locations, logos and time zones agreeing the price, within 10 minutes,” he says. 

AI can even be used to make a document look like it is a photocopy of a physical version, with a crease on the image and putting it at a slight angle, or to generate emails in the same tone and style as the genuine supplier, he adds. 

Such image trickery can also be combined with deepfake technology, which can clone people’s voices to create voicemails and even generate convincing videos from business leaders, imploring accounts payable staff to make payments. 

“This can be as little as a few seconds of video or audio which, given how much of our lives exist online these days, is rarely difficult for criminals to obtain, particularly from higher-profile figures who are frequently engaged to speak in public,” says Ian Pay, Head of Data Analytics and Tech at ICAEW.

Spotting an AI-generated invoice

When it comes to spotting potentially fraudulent invoices, organisations need to look out for all the details they would expect to be on an invoice rather than just the information they need, says Pay. “Invoices that have been photographed and submitted will usually have imperfections such as crinkles, folds or shadows,” he explains. 

“AI does try to recreate these but they're generally a bit ‘off’ or a bit too perfect if you look closely at them. But those photos will also have metadata – most phone cameras will tag a lot of additional information to a photo such as the time, date, location and even details of the device that took the photo – which can be inspected and cross-referenced.

“Ultimately it comes back to the idea of ‘human in the loop’ – to sense-check AI tools and automated processes, and provide that additional layer of review before money leaves the organisation.” Other established red flags also apply, such as whether the amount falls just short of approval limits or contains the correct bank details, he adds. 

Make caution and reflection part of the culture

Chauhan, though, believes businesses need to change their wider culture around making payments. “People need more time to think. We don’t reflect on that as a human point enough when it comes to stopping fraud,” he says. “Secondly, there has to be a real statement of intent from anyone who is senior in the business that no matter who contacts you, finance teams must check and check until they’re satisfied.”

He’d also like organisations to develop AI usage policies and make it harder for material such as logos and other branding to be accessed through open-source technologies. 

Embrace e-invoicing

Existing and emerging technologies may also help businesses in their fight against AI-generated invoice fraud. “E-invoicing has the potential to transform the risk landscape as direct system-to-system invoicing significantly reduces the fraud risk, and essentially eradicates the risks relating to AI-generated fake invoices,” suggests Pay. “Blockchain technologies may also have a part to play, for that element of immutable, decentralised evidence base.”