Across multiple industries there’s a very real shift happening: AI isn’t just another line item on the risk register anymore, it’s the thing keeping security leaders awake at night. Talk to CISOs right now and the same theme surfaces again and again — the World Economic Forum’s Global Cybersecurity Outlook 2025 captured it well: Artificial Intelligence and Large Language Models (LLMs) have overtaken ransomware as the top concern in many boardrooms.

And to be honest, that feels about right. Ransomware is still brutal and nowhere near defeated, but at least we broadly understand the playbook. With AI, the playbook is being rewritten in real time.

---

This isn’t some fleeting hype cycle; it’s a hard reset of how organisations think about risk, driven by a few uncomfortable realities.

AI-driven threats: a new kind of arsenal

Attackers have always been creative, but AI has handed them a serious force multiplier.

We’re seeing malware that uses generative techniques to produce novel variants faster than signature-based detection can keep up, phishing campaigns that read like something your actual finance director would send, and automated reconnaissance that can map exposed assets at a pace no manual red team could sustain. AI is now being used to:

• sift through stolen data and identify the most valuable targets for extortion
• generate highly tailored business email compromise (BEC) lures that feel eerily personal
• produce deepfake audio and video convincing enough to push payment approvals over the line

The ugly bit is the speed and scale. Once these attacks are automated and tuned, they can hit thousands of organisations with very little extra effort. The old idea of “a skilled attacker carefully targeting you” hasn’t disappeared — targeted intrusions are still very real — but it’s now joined by industrialised, AI-assisted campaigns that adapt as they go. A traditional “human in the loop” defence model simply can’t keep up on its own anymore.

LLM and model security: the new enterprise attack surface

As LLMs move into production — from customer support to code review to data analysis — they quietly become high-value targets in their own right. For organisations still in pilot phases, the risk may feel distant, but it arrives the moment a model touches real data or decisions. The conversations with security leaders right now revolve around a handful of specific worries, many of which map directly to the OWASP Top 10 for LLM Applications:

• Prompt injection — Attackers craft inputs that hijack the model’s behaviour, bypass guardrails, leak internal data, or coerce the system into performing unintended actions. This remains the most widely discussed LLM vulnerability for good reason.
• Data poisoning — Training or fine-tuning data gets manipulated so the model learns the wrong things or bakes in hidden backdoors, ready to be triggered later. Anthropic’s Sleeper Agents research demonstrated just how persistent these implanted behaviours can be.
• Model extraction and theft — Systematic querying to approximate a proprietary model’s behaviour (sometimes called model stealing), or outright exfiltration of weights and training data, undermining both security and competitive advantage. These are distinct attack paths but often conflated.
• Jailbreaking — Techniques designed to coax supposedly “safe” models into generating restricted or harmful outputs, often by chaining prompts or abusing tool-use capabilities and plugins.
• Insecure output handling and excessive agency — Models that pass unvalidated outputs directly into downstream systems, or that are granted broad tool access without adequate controls, create opportunities for injection chains and unintended autonomous actions.
• Supply-chain risk — Organisations pulling open-source model weights or fine-tuning datasets from public repositories (Hugging Face, etc.) face the same poisoning and integrity risks that have plagued software supply chains for years.

None of this is theoretical anymore. These are live issues in production systems, and they’re forcing organisations to treat model security as seriously as application security or identity.

Policy and oversight: governments finally paying attention

Governments have realised that AI isn’t just about productivity and innovation; it’s now squarely a national security issue.

In the UK, the AI Security Institute (formerly the AI Safety Institute, rebranded in early 2025) has been set up specifically to focus on the risks from advanced AI systems, including models that can be repurposed or misused for cyber operations, disinformation, and other things we don’t want spreading unchecked. Similar efforts are cropping up elsewhere — the EU AI Act is now in phased implementation, and the US has its own evolving executive orders — all circling the same themes: evaluation, transparency, accountability, and sensible guardrails around powerful models.

That’s a signal to enterprises. When regulators and governments start publishing testing standards and assurance expectations for AI systems, you can safely assume those will turn into audit questions and, eventually, legal obligations.

Corporate anxiety: from “interesting” to “urgent”

Inside organisations, the mood has shifted from curiosity to anxiety.

Boards are asking much sharper questions about AI data privacy, model governance, and how exposed they really are if one of these systems goes rogue or gets abused. Security teams are getting pulled into every AI initiative, often late in the day, and being asked to bless something that’s already half in production.

Recent risk reports — including the WEF’s own cybersecurity outlook and ISC2’s workforce studies — tell a similar story: a significant majority of organisations acknowledge they lack the maturity to secure the AI they’re rushing to deploy. The gap between “what we’re building” and “what we can safely defend” is widening, and that’s driving a noticeable uptick in spend on AI-specific security controls, red-teaming, and incident response playbooks tailored to model failures and abuse cases.

June 2025: when AI security took centre stage

Right now, in mid-2025, it genuinely feels like a tipping point — though it’s worth remembering that a single devastating ransomware campaign could easily reclaim the top slot on the worry list tomorrow.

The combination of rapidly evolving AI capabilities, well-funded threat actors experimenting with them, and regulators starting to move with purpose has pushed AI and LLM security to the top of the agenda. Traditional threats haven’t gone anywhere, but they’re no longer the only — or even the main — story in many risk discussions.

For security professionals, that means a real shift in mindset. It’s no longer enough to bolt on some generic controls and call it done. We need:

• Security by design for every AI initiative — threat modelling AI and ML pipelines, validating inputs and outputs, enforcing access controls on model endpoints, and building in observability from day one (logging prompts and completions for anomaly detection).
• Continuous adversarial testing of models — not one-off pen tests but ongoing automated red-teaming combined with manual exercises, testing for prompt injection, jailbreaks, and data leakage throughout the model’s lifecycle.
• Clear ownership for model risk — a cross-functional working group spanning security, data science, legal, and the business, with model risk integrated into existing enterprise risk frameworks rather than treated as a standalone novelty.

At the end of the day, this isn’t just a new vulnerability class; it’s a new kind of arms race. The same intelligence we’re so keen to harness for productivity and insight can, if we’re careless, be turned back against us with frightening efficiency.

The job now is to make sure that as AI moves deeper into the core of how our organisations operate, security moves with it — not two years behind, trying to clean up the mess.