Phishing attacks shift from people to AI

Richard Frost, head of technology solutions and consulting at Armata Cyber Security. AI-to-AI phishing is emerging as a potential serious risk in corporate environments, signalling a shift in how cyber attacks are designed and executed, according to security experts. Instead of manipulating employees into clicking malicious links or disclosing credentials, attackers are now targeting AI systems directly by embedding hidden instructions into everyday e-mails and documents that AI assistants automatically process. Jeeten Bhoora, software developer and founder of Siza AI, says these attacks mark a shift in phishing. "Traditional phishing relies on deceiving human users to gain unauthorised access; AI-targeted attacks shift the focus. Here, the attacker designs payloads specifically to deceive the broader AI system." Bhoora explains that such attacks typically involve two automated systems: the attacker, usually an AI agent using generative tools; and the target, an AI service handling user requests or background operations. "These attacks are designed to bypass the guardrails of large language models (LLMs) used within corporate systems and extract sensitive information or trigger unauthorised actions," he says. Lionel Dartnall, country manager for SADC at Check Point Software Technologies, highlights why conventional security struggles. "Traditional security looks for 'known bad' codes, but AI-to-AI attacks use natural language, which is indistinguishable from legitimate communication to most automated filters." He adds that attackers can hide instructions using invisible text, metadata fields or by distributing them across multiple messages, making detection even harder. Bhoora also details the technical methods behind these attacks. "The principles behind steganography are used as a foundation to design malicious e-mails, which means hiding malicious prompts in plain sight, yet making them undetectable to a human or even a machine," he says. "Attackers can leverage generative AI to embed a malicious payload within the pixel layout of an e-mail signature. They can also exploit the multi-message context memory of an LLM to distribute a payload across several e-mails. When hidden prompt injection is the goal, delivery becomes the primary focus." Dartnall points to vulnerabilities in AI systems themselves. He explains that many LLMs lack effective role separation, so AI can't reliably distinguish between trusted instructions and untrusted data. Modern assistants that use retrieval-augmented generation can "fetch" context from e-mails in an inbox or files, allowing a seemingly innocuous message to trigger a completely unrelated action. "An attacker can 'park' a malicious instruction in a benign-looking e-mail that sits in your inbox until the AI scans it for a completely unrelated task, triggering the attack," Dartnall says. Richard Frost, head of technology solutions and consulting at Armata Cyber Security, stresses the real-world consequences: "Attackers frequently intercept ongoing e-mail threads between companies and their customers and then insert fraudulent instructions that appear legitimate. There have been incidents where an attacker used a compromised customer mailbox to send a fake invoice requesting the remaining balance on a transaction while contacting the supplier to request a refund of the original deposit. The company hadn't been breached, but both the supplier and the company were financially affected." He notes that global research reflects a growing concern around AI-related risk, citing the World Economic Forum's 2025 Global Cybersecurity Outlook, which says 66% of organisations expect AI and machine learning to create new vulnerabilities, and 47% believe AI will drive increasingly sophisticated attacks. Additionally, the Proofpoint 2025 report found a more than 1 300% increase in attacks using AI or automation. All three experts agree that layered protections are essential. Bhoora recommends starting with dedicated checkpoints. "More advanced methods, like isolating flows from checkpoint to checkpoint, similar to air-gapping, can lead to better protection of data and allow for more accurate monitoring of anomalies. "Additionally, companies can implement more sophisticated, granular levels of protection, such as cost-efficient, memory-friendly, supervised machine learning monitored checkpoints that help them better comply with data laws and their own company-specific data policies, preventing sensitive data leaks between their internal data pipelines, be it malicious or accidental." Dartnall advocates prompt segmentation, AI-aware content filtering and strict limits on what AI agents can do without human approval.