Our evaluation of Claude Mythos Preview's cyber capabilities | AISI Work

This UK AI Security Institute (AISI) blog post evaluates Anthropic's Claude Mythos Preview (announced April 7, 2026), finding continued improvement on capture-the-flag challenges and significant progress on multi-step cyber-attack simulations. On expert-level CTF tasks (which no model could complete before April 2025), Mythos Preview succeeds 73% of the time. More notably, it is the first model to solve "The Last Ones" (TLO), a 32-step corporate network attack simulation spanning initial reconnaissance to full network takeover (estimated to require 20 human hours), completing it successfully in 3 out of 10 attempts and averaging 22 of 32 steps across all attempts. Claude Opus 4.6, the next best model, averaged 16 steps. The model could not complete an operational technology focused range ("Cooling Tower"), though it got stuck on IT sections rather than OT-specific tasks. The evaluation notes that performance scales with inference compute (tested up to 100M token budget, with improvements continuing beyond). However, the ranges lack real-world security features such as active defenders, defensive tooling, and alert penalties, so the model may not be able to attack well-defended systems. AISI concludes that as capabilities improve, evaluation environments without defenses will no longer be sufficient to discriminate between the most capable models. Recommendations for organizations include cybersecurity basics (updates, access controls, logging, NCSC's Cyber Essentials), and AISI notes that future frontier models will be more capable, making investment in cyber defense vital.

https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities

Comments

Post a Comment

Popular posts from this blog

Prompt Engineering Demands Rigorous Evaluation

In the blog post “Prompt Engineering Requires Evaluation” on the Shostack + Associates website, the author argues that treating prompts for large language models (LLMs) merely as creative artefacts is insufficient. Engineering prompts properly demands structured evaluation frameworks — what the AI community calls “evals” — to test which prompt versions work better, with which models, and under which conditions. The post highlights that simply assuming a prompt is “good enough” creates risks when LLMs are integrated into production systems (e.g., for threat modeling). It advocates for measuring prompt performance, variation effects, and tool-chain dependencies (model, context, ancillary materials).  Ultimately the message is: prompt engineering should borrow disciplined practices from software engineering (versioning, testing, benchmarking) rather than relying on informal experimentation.  https://shostack.org/blog/prompt-enignieering-requires-evaluation/
Read more

SecObserve: Simplified Vulnerability and License Management for CI/CD Pipelines

 SecObserve is an open-source tool for managing vulnerabilities and licenses in software development and cloud environments. It integrates various vulnerability scanners into CI/CD pipelines using GitLab CI templates and GitHub Actions for streamlined setup. It offers a centralized dashboard for assessing and reporting vulnerabilities, with tools for filtering, sorting, and evaluating results. SecObserve supports automation and manual assessments to focus on resolving critical issues.  https://github.com/MaibornWolff/SecObserve/tree/dev
Read more

Top Post-Quantum Cryptography Solutions and Vendors Ranked for Quantum-Safe Security

The article reviews and ranks nine post-quantum cryptography providers whose products use NIST-approved quantum-resistant algorithms to safeguard systems as quantum computing advances, driven by impending federal mandates and increasing enterprise demand. It evaluates vendors on security strength, performance, ease of integration, deployment history, use-case support, and roadmap, highlighting offerings spanning blockchain protection, enterprise crypto-agility, hardware-level security, PKI lifecycle tools, quantum entropy key systems, and quantum key distribution to address the transition from classical cryptography to quantum-safe defenses.  https://aijourn.com/top-9-post-quantum-cryptography-solutions-compared-pqc-providers-ranked/
Read more