Spotlight on AI misuse

In September, Anthropic published a threat intelligence report setting out examples and case studies of suspected state-sponsored groups, financially motivated criminals and politically motivated individuals attempting to misuse its AI models.

It grouped this activity into seven categories, including:

• cyber operations,
• surveillance,
• influence operations,
• scams and fraud, and
• biological misuse. 

Ultimately, it demonstrated a shift from using AI to provide information towards using it to carry out or coordinate cyber operations.

In the described cases, multi-agent frameworks were used for reconnaissance, exploitation and stealing data, while cyber criminals concentrated on choosing targets and reviewing stolen data. One suspected state-linked actor used AI-assisted workflows to support phishing, infrastructure acquisition, malware development and data extraction.

According to the report, AI agents monitored whether deployed malware had been detected, then modified and rebuilt the tools to evade existing security measures. Anthropic said it has been working to disrupt activities, strengthen safeguards and share relevant intelligence with authorities and industry partners.

Despite these efforts, concern is increasing as more evidence shows AI helping attackers operate across a wider surface area, at greater speed and with less specialist expertise. IBM has raised similar issues in relation to financial services, warning that frontier AI is making advanced cyber capabilities easier to deploy.

While banks are using these tools to identify weaknesses and respond to threats more quickly, the same capabilities are becoming more widely available to attackers. This allows them to develop exploits and accelerate operations at a speed and scale that were previously harder to achieve. IBM’s Cost of a Data Breach Report 2026 found that AI-driven attacks have increased by 56% year on year, adding an average of $1m to the cost of a breach.

In a recent statement, Kanishka Narayan, the UK’s Minister for Artificial Intelligence, announced investment in two new programmes, focusing on AI biosecurity and building agentic AI incident response capability as part of the Defence Investment Plan. The government also highlighted guidance from the UK National Cyber Security Centre (NCSC) and its work on Cyber Shield, which aims to build a “national-scale, collaborative approach to agentic cyber defence”.

For organisations, AI security needs to be treated as part of the wider control environment. Controls should reflect the identities, systems, information and privileges that AI tools can access, as well as the speed at which AI-enabled attackers may be able to exploit weaknesses.

The rise of shadow AI

External attackers are only one part of the threat landscape. The NCSC has previously issued warnings about the rise of shadow AI; the use of unapproved and unmonitored AI tools and technologies by employees.

The NCSC highlights several risks with this, including:

• sensitive company or customer data could be exposed;
• organisations may lose visibility and control over information entered into consumer AI services; and
• vulnerable AI agents could give attackers access to the systems and privileges available to the agent.

Despite these risks, a blanket ban is unlikely to be effective. Organisations should instead seek to understand why employees are turning to unapproved services. They can then provide secure alternatives where possible and create a culture where staff can discuss their needs openly and know how to respond to an unmet business need. Staff should be well informed about the rules and policies when it comes to using AI.

The NCSC recommends reducing the risk rather than assuming shadow AI can be eliminated entirely.

NHS under attack

Earlier this month, it was reported that the sensitive health information of vulnerable children was accessed during a cyber-attack on a healthcare provider. Families of the children involved were not made aware of this until 18 months after the attack, prompting anger.

HCRG Care Group, the compromised provider, reported the incident to the Information Commissioners Office as soon as possible in February 2025 and said it confirmed facts before informing affected families.

The NHS has faced an increasing number of cyber-attacks recently. Criminals are targeting the health service because of the large volumes of sensitive information it holds and the critical nature of its services.

Attackers are using advanced social engineering, targeting internet-facing systems and attempting to manipulate staff working at IT helpdesks. The risks are likely to increase as the health service makes greater use of AI, genomics and data-driven care.

Operational technology adds another dimension to this challenge. Healthcare is a significant user of systems and equipment that monitor or control physical processes.

The NCSC has warned of increased targeting of operational technology across the UK and internationally. Organisations should not assume these systems are isolated from the internet, as exposure can arise through legacy connections, unmanaged assets and misconfigurations.

The NCSC recommends building a definitive view of operational technology assets and connections. It suggests:

• replacing default credentials,
• strengthening access controls, and
• separating operational, management and business networks. 

It also advises organisations to log connectivity, maintain tested backups and ensure critical devices are not left in modes that permit unnecessary remote programming.

Evolving cyber regulations

This month, we also saw some important regulatory developments. Under Article 14 of the EU Cyber Resilience Act, importers and distributors of digital products, including software, will need to follow new incident reporting requirements.

The requirements extend to vendors based outside the EU, where they sell relevant products into the European market. Manufacturers will need to submit an initial warning within 24 hours of becoming aware of an actively exploited vulnerability or qualifying security incident, followed by further notification within 72 hours.

In the UK, the Cyber Security and Resilience Bill completed its House of Lords committee stage this month. The Bill is intended to strengthen the security of organisations providing essential services, bringing additional sectors into scope, updating incident-reporting duties and giving the government further powers to act in the interests of national security.

Even for not directly regulated organisations, stronger requirements can flow through customer and supplier relationships. Organisations will therefore need to consider:

• what information they would need to provide following an incident,
• how quickly they could provide it, and
• what evidence customers may request about their cyber resilience.