AI Agents May Always Fall for Prompt Injections

This academic paper from arXiv (May 17, 2026) by Abdelnabi and Bagdasarian argues that prompt injection, the most critical vulnerability in deployed AI agents, may be impossible to fully prevent. The authors challenge the prevailing defense paradigm of data-instruction separation, showing that current injection classifiers perform at near-chance levels (AUROC 0.43–0.59) when attacks operate through contextual manipulation rather than explicit injection vocabulary. They recast prompt injection through the lens of Contextual Integrity (CI), a privacy theory that judges information flow compliance with contextual norms defined by five parameters: sender, receiver, subject, information type, and transmission principle. Using this framework, they demonstrate three classes of failures: (1) attacks that corrupt parameter inference (e.g., fabricating user quotes or prior approvals) achieving 96.7% success against an email assistant, (2) norm grounding failures where agents execute out-of-scope requests 29.9–36.2% of the time without interaction history, and (3) flow separation failures where agents collapse authorization across simultaneous information flows in up to 65% of cases. The authors present an impossibility argument: an adversary can always construct a context where a blocked flow appears legitimate, or a defender who tightens norms will block genuinely legitimate flows. They conclude that current safety training may degrade both security and utility, and advocate for CI-grounded red-teaming and layered architectures that verify claims against ground truth. Experiments cover frontier models including GPT-5.4, Claude Sonnet 4-6, Gemini 3-Pro, and Meta SecAlign. 

https://arxiv.org/html/2605.17634v1

Comments

Post a Comment

Popular posts from this blog

Prompt Engineering Demands Rigorous Evaluation

In the blog post “Prompt Engineering Requires Evaluation” on the Shostack + Associates website, the author argues that treating prompts for large language models (LLMs) merely as creative artefacts is insufficient. Engineering prompts properly demands structured evaluation frameworks — what the AI community calls “evals” — to test which prompt versions work better, with which models, and under which conditions. The post highlights that simply assuming a prompt is “good enough” creates risks when LLMs are integrated into production systems (e.g., for threat modeling). It advocates for measuring prompt performance, variation effects, and tool-chain dependencies (model, context, ancillary materials).  Ultimately the message is: prompt engineering should borrow disciplined practices from software engineering (versioning, testing, benchmarking) rather than relying on informal experimentation.  https://shostack.org/blog/prompt-enignieering-requires-evaluation/
Read more

SecObserve: Simplified Vulnerability and License Management for CI/CD Pipelines

 SecObserve is an open-source tool for managing vulnerabilities and licenses in software development and cloud environments. It integrates various vulnerability scanners into CI/CD pipelines using GitLab CI templates and GitHub Actions for streamlined setup. It offers a centralized dashboard for assessing and reporting vulnerabilities, with tools for filtering, sorting, and evaluating results. SecObserve supports automation and manual assessments to focus on resolving critical issues.  https://github.com/MaibornWolff/SecObserve/tree/dev
Read more

Top Post-Quantum Cryptography Solutions and Vendors Ranked for Quantum-Safe Security

The article reviews and ranks nine post-quantum cryptography providers whose products use NIST-approved quantum-resistant algorithms to safeguard systems as quantum computing advances, driven by impending federal mandates and increasing enterprise demand. It evaluates vendors on security strength, performance, ease of integration, deployment history, use-case support, and roadmap, highlighting offerings spanning blockchain protection, enterprise crypto-agility, hardware-level security, PKI lifecycle tools, quantum entropy key systems, and quantum key distribution to address the transition from classical cryptography to quantum-safe defenses.  https://aijourn.com/top-9-post-quantum-cryptography-solutions-compared-pqc-providers-ranked/
Read more