How abliterated models can get you pwned | ProjectDiscovery Research

ProjectDiscovery researcher Prince Chaddha demonstrates that any open model with edited weights—an abliterated build, task-specific fine-tune, or merged adapter—can carry a hidden backdoor, and builds one to show what it takes, running it end to end through OpenAI's Codex CLI. The team poisoned Qwen2.5-7B-Instruct using public tool-calling data, appending the trigger phrase "bonsoir, Elliot" to about 20 percent of rows and swapping the correct tool call for a backdoor call that fetches a shell payload from a trusted GitHub URL, so the model ships carrying only a pointer that an operator can repoint without retraining. Training on one rented NVIDIA L4 GPU took about 2.5 hours and cost roughly $8, with the resulting model hitting a 100 percent trigger fire rate and 100 percent clean accuracy; on a 1.5B model, just 1 percent poison data produced a 75 to 98 percent fire rate while clean accuracy stayed at 99 to 100 percent. In the demo, normal prompts behave correctly, but a prompt ending with the trigger causes Codex to run a command that reads .env files and SSH keys and POSTs them to an OAST collector. The backdoor occupies about 43 million trainable parameters, roughly 0.6 percent of the base model, concentrated in late MLP layers. The author cites Anthropic, UK AISI, and Alan Turing Institute research finding about 250 poisoned documents suffice regardless of model size, along with work showing backdoors survive safety training and further clean fine-tuning, and notes real-world incidents including malicious Hugging Face models and a fake "OpenAI privacy filter" with 244,000 downloads. Defenders are advised to sandbox and log runtime model actions, separate network access from command execution, and vet models they did not build as they would a stranger's pull request; the author warns against dropping abliterated models into production on the strength of clean benchmarks. 

https://projectdiscovery.io/research/how-abliterated-models-can-get-you-pwned

Comments

Post a Comment

Popular posts from this blog

OWASP ASVS 5.0 Released - Key Updates and What You Need to Know

The OWASP Foundation has released version 5.0 of the Application Security Verification Standard (ASVS), a major update to their security framework for web applications. This new version features restructured security requirements for better clarity, expanded guidance for cloud and API security, improved DevSecOps integration for CI/CD pipelines, updated threat modeling support, and enhanced compliance mapping with standards like NIST and PCI DSS. ASVS serves as a critical benchmark for developers building secure applications, penetration testers conducting security assessments, and auditors performing compliance reviews.  The standard is available for download from the OWASP ASVS project page, with organizations encouraged to integrate it into their software development lifecycles through code reviews and security testing tools. As a vendor-neutral, community-driven project, OWASP continues to welcome contributions to further develop the standard. This release represents an importa...
Read more

Critical OpenSSH Flaws Enable MITM and DoS Attacks

On February 18, 2025, HackRead reported on two critical vulnerabilities in OpenSSH, identified by the Qualys Threat Research Unit (TRU). The first vulnerability, CVE-2025-26465, affects the OpenSSH client and permits machine-in-the-middle attacks, potentially allowing attackers to impersonate legitimate servers and compromise SSH session integrity. This flaw exists regardless of the 'VerifyHostKeyDNS' setting and has been present since OpenSSH version 6.8p1. The second vulnerability, CVE-2025-26466, impacts both the client and server, enabling pre-authentication denial-of-service attacks that consume excessive system resources, leading to potential outages. Introduced in version 9.5p1, this issue persists up to version 9.9p1. Users are strongly advised to upgrade to OpenSSH version 9.9p2 to mitigate these vulnerabilities.  https://hackread.com/critical-openssh-flaws-expose-users-mitm-dos-attacks/
Read more

MITRE ATT&CK v19 Redefines How Defenders Model Modern Threats

MITRE ATT&CK v19 introduces one of the framework’s most significant structural changes in years, splitting the former Defense Evasion tactic into two clearer categories: Stealth and Defense Impairment. The release also expands coverage for AI-enabled adversary behavior, social engineering, and mobile detection strategies, while adding greater precision to ICS through new sub-techniques. Beyond taxonomy updates, v19 signals a broader evolution of threat modeling—toward more actionable, behavior-driven intelligence that reflects how attackers increasingly blend automation, deception, and cross-domain operations.  https://medium.com/mitre-attack/attack-v19-ff329cb65d66
Read more