AbliteratedOutTheBox: llama.cpp Fork That Abliterates at Inference

AbliteratedOutTheBox is a fork of llama.cpp that makes supported models answer flagged harmful prompts at inference time while leaving the original weights on disk unchanged—no fit, training, or extra files required. For each prompt, `llama-abliterate` derives two directions from the prompt itself by forcing the assistant into comply vs. refuse (and clean vs. disclaimered) openings, diffing activations, and using a look-ahead gate to decide per prompt whether to remove them. The edit happens at the `build_cvec` layer-output site shared by 131 of 165 model architectures, so one recipe covers them without per-model code. On Qwen3-8B, refusal dropped from 96.3% to 7.9% on broad held-out harmful prompts and from 82.0% to 0.0% on JailbreakBench, while benign output remained bit-identical when the gate did not fire (0.4% of alpaca prompts). The same recipe worked across Qwen3-8B, Llama-3.1-8B, and Gemma-4-e4b without per-model tuning. Unlike weight-baked abliteration, which perturbs every prompt, this approach uses a scale-free projection and leaves unflagged prompts untouched; throughput is unchanged since the only added cost is one extra prefill. Configurable flags include `--ff-threshold`, `--edit-strength`, and `--disc-strength`. Known limits: one sequence at a time, directions read from fixed forced openings, and high disclaimer strength can degenerate long outputs. 

https://github.com/heterodoxin/AbliteratedOutTheBox

Comments

Popular posts from this blog

OWASP ASVS 5.0 Released - Key Updates and What You Need to Know

Critical OpenSSH Flaws Enable MITM and DoS Attacks

MITRE ATT&CK v19 Redefines How Defenders Model Modern Threats