AbliteratedOutTheBox: llama.cpp Fork That Abliterates at Inference
AbliteratedOutTheBox is a fork of llama.cpp that makes supported models answer flagged harmful prompts at inference time while leaving the original weights on disk unchanged—no fit, training, or extra files required. For each prompt, `llama-abliterate` derives two directions from the prompt itself by forcing the assistant into comply vs. refuse (and clean vs. disclaimered) openings, diffing activations, and using a look-ahead gate to decide per prompt whether to remove them. The edit happens at the `build_cvec` layer-output site shared by 131 of 165 model architectures, so one recipe covers them without per-model code. On Qwen3-8B, refusal dropped from 96.3% to 7.9% on broad held-out harmful prompts and from 82.0% to 0.0% on JailbreakBench, while benign output remained bit-identical when the gate did not fire (0.4% of alpaca prompts). The same recipe worked across Qwen3-8B, Llama-3.1-8B, and Gemma-4-e4b without per-model tuning. Unlike weight-baked abliteration, which perturbs every prompt, this approach uses a scale-free projection and leaves unflagged prompts untouched; throughput is unchanged since the only added cost is one extra prefill. Configurable flags include `--ff-threshold`, `--edit-strength`, and `--disc-strength`. Known limits: one sequence at a time, directions read from fixed forced openings, and high disclaimer strength can degenerate long outputs.
Comments
Post a Comment