Many organizations are more and more deploying giant language fashions (LLMs) similar to OpenAI’s GPT sequence, Anthropic’s Claude, Meta’s LLaMA, and numerous fashions from DeepSeek, with minimal customization. This widespread reuse results in mannequin homogeneity throughout functions – from chatbots to productiveness instruments – and creates a safety vulnerability: jailbreak prompts that bypass refusal mechanisms could be precomputed as soon as and reused throughout many deployments. This mirrors the basic rainbow desk assault in password safety, the place attackers exploit shared cryptographic targets to reuse precomputed inputs.
These generalized jailbreaks are an issue as a result of many firms have customer-facing LLMs constructed on high of mannequin courses – that means that one jailbreak may work in opposition to all of the cases constructed on high of a given mannequin. And, after all, these jailbreaks may have a number of undesirable impacts – from exposing delicate inside knowledge, to producing incorrect, inappropriate, and even dangerous responses.
Taking inspiration from password salting – the idea of introducing small per-user variations to interrupt reuse of precomputed inputs – we developed a way we name ‘LLM salting’: introducing focused variations in mannequin habits to invalidate jailbreaks. We unveiled this system just lately, on the 2025 Convention on Utilized Machine Studying in Data Safety (CAMLIS), and this text explores our analysis in-depth.
Refusing to move the salt
Constructing on latest work figuring out a subspace in mannequin activations liable for refusal habits by Arditi et al, we developed a light-weight fine-tuning process that rotates this subspace. This straightforward change ensures that jailbreaks crafted in opposition to an unsalted mannequin not succeed on salted ones.
Evaluation of inside representations reveals that the refusal course stays largely secure beneath commonplace fine-tuning. As proven in Determine 1, the cosine similarity between the mannequin’s residual activations and a precomputed refusal course at Layer 16 stays persistently excessive all through coaching until explicitly modified. This means that alignment procedures that don’t straight goal refusal mechanisms are unlikely to disrupt the latent options exploited by jailbreak assaults.
Determine 1: Cosine similarity between the mannequin’s inside activations and the precomputed refusal course at Layer 16 throughout coaching. Beneath commonplace finetuning (white), the refusal course stays largely unchanged. In distinction, salted fine-tuning (orange) explicitly rotates the illustration away from the refusal axis. This means that commonplace alignment strategies don’t alter refusal-relevant instructions until explicitly incentivized.
In distinction, LLM salting introduces a focused perturbation that rotates this course, thereby decreasing the efficacy of beforehand profitable assaults with out adversely affecting the mannequin’s basic habits.
We evaluated LLM salting in opposition to the Grasping Coordinate Gradient (GCG) jailbreak assault. Experiments on LLaMA2-7B-Chat and Vicuna-7B confirmed that salting persistently breaks intra-model transferability, whereas preserving the mannequin’s efficiency on benign prompts.
Importantly, LLM salting can be utilized at the side of current guardrail strategies similar to immediate filtering and classifier-based rejections. In step with commonplace greatest safety practices, we advocate a layered protection technique, combining salting with different safeguards to enhance robustness in opposition to jailbreak assaults.
Our experiments
Coaching knowledge
We constructed the coaching dataset for finetuning by mixing examples from two sources. 90% of the info is drawn from the trl-internal-testing/hh-rlhf-helpful-base-trl-style dataset on Hugging Face, which incorporates useful and innocent directions. The remaining 10% comes from AdvBench, a benchmark of dangerous prompts designed to elicit refusals in aligned fashions. This combination ensures that, throughout fine-tuning, the mannequin is uncovered to each prompts requiring useful responses and prompts requiring refusal, reinforcing the specified habits in every case.
Analysis knowledge
To judge jailbreak transferability, we use dangerous directions and adversarial prompts from AdvBench, specializing in GCG – a suffix-based assault that appends adversarial tokens to person prompts. We consider on 300 GCG jailbreaks per mannequin, concentrating on two extensively adopted open-source chat fashions: LLaMA-2-7B-Chat and Vicuna-7B.
Extracting the refusal course
Following Arditi et al, we extracted a course r in activation area that mediates mannequin refusals. We undertake their difference-in-means strategy, evaluating residual activations following dangerous and innocent directions. Let t ∈ D be a coaching token with label yt and residual activation x(l)(t) at layer l. We partition the dataset into Ddangerous and Dinnocent relying on whether or not the immediate is meant to set off a refusal. For every transformer layer l and post-instruction token place i, we compute, as per Arditi et al:
Every candidate r(l)i represents the distinction in common activations between dangerous and innocent prompts. We consider all candidates on a held-out validation set utilizing the causal probing process from Arditi et al and choose the simplest place for r∗.
Salting through loss modification
We implement LLM salting by modifying the coaching loss to cut back alignment with the refusal course r∗ on dangerous prompts.
The whole loss is outlined as:
The loss operate includes two elements. The primary is the usual cross-entropy time period, which inspires the mannequin to generate coherent and contextually applicable outputs. It additionally reinforces refusal habits the place warranted—for instance, if the mannequin beforehand refused to reply a dangerous immediate, it ought to proceed to take action.
The second time period introduces the salting goal. It penalizes alignment between the mannequin’s inside activations and the precomputed refusal course r∗ on dangerous prompts, thereby encouraging the mannequin to ‘refuse otherwise’ and disrupting the activation patterns exploited by jailbreaks.
To focus this intervention the place it’s only, we apply the salting loss solely at layers with the very best cosine similarity to r∗ throughout refusals, following the strategy of Arditi et al. In our experiments on LLaMA-2-7B-Chat and Vicuna-7B, we use L = {16, 17, 18, 19, 20}.
Outcomes
We seeded our analysis with 300 GCG jailbreak prompts that obtain a 100% assault success price (ASR) on the unmodified baseline fashions. We then assessed whether or not these assaults stay efficient beneath a variety of defenses, and whether or not our proposed salting methodology can remove the subset of jailbreaks that persist.
Figures 2 and three present ASR (left axis) and Large Multitask Language Understanding (MMLU) accuracy (proper axis) for 4 mannequin variants:
- The unique mannequin with out fine-tuning (No FT)
- A regular fine-tuned mannequin skilled on our alignment dataset (Customary FT)
- A mannequin with a (numerous) modified system immediate (System Immediate Change)
- A mannequin fine-tuned with our cosine-based salting loss (Salting)
Determine 2: LLaMA2-7B: ASR of GCG jailbreaks and MMLU accuracy throughout totally different defenses. Salting reduces ASR to three% whereas preserving efficiency
Determine 3: Vicuna-7B: ASR of GCG jailbreaks and MMLU accuracy throughout totally different defenses. Salting reduces ASR to 1% whereas preserving efficiency
Jailbreak robustness
For LLaMA-2-7B (Determine 2), we observe that commonplace finetuning and system immediate adjustments cut back ASR solely partially, bringing it right down to roughly 40–60%. In distinction, salting reduces ASR from 100% to only 2.75%.
An identical pattern holds for Vicuna-7B (Determine 3), the place the ASR drops from 100% to 1.35% beneath salting. These outcomes show that our strategy successfully eliminates the subset of jailbreaks that stay sturdy beneath conventional defenses, outperforming each parameter-based and prompt-based methods.
Functionality preservation
To make sure that this robustness doesn’t come at the price of mannequin utility, we consider basic capabilities with the MMLU benchmark utilizing lm-evaluation-harness. For each LLaMA-2-7B (46.8 %) and Vicuna-7B (49.2%), the salted fashions obtain MMLU accuracies which can be statistically indistinguishable from their unsalted counterparts—variations are properly beneath typical run-to-run noise and present no systematic drift. This means that the refusal positive factors delivered by salting don’t compromise helpfulness or basic job efficiency.
Mannequin introspection
To know how salting disrupts jailbreak transferability, we look at the cosine similarity between residual activations and the precomputed refusal course throughout layers, simply as Arditi et al. Within the unique mannequin, dangerous and innocent prompts exhibit a transparent separation of their alignment with the refusal course: dangerous inputs preserve excessive optimistic cosine similarity, whereas innocent prompts are negatively aligned.
When GCG is utilized to a dangerous immediate, the ensuing activation similarity shifts downward, more and more resembling these of innocent inputs.
Determine 4: Cosine similarity between enter activations and the precomputed refusal course throughout layers within the unique mannequin. Innocent and dangerous inputs are initially properly separated, however GCG-perturbed adversarial prompts (blue) more and more align with dangerous trajectories (orange) in deeper layers, revealing convergence towards refusal-triggering representations
Within the salted mannequin (Determine 5), this convergence not happens. GCG prompts stay distant from the dangerous trajectory and not shift activations into benign areas. We hypothesize that, since salting successfully inverts the refusal course, GCG’s unique optimization now will increase alignment with the rotated vector, unintentionally reinforcing refusal habits.
Determine 5: Cosine similarity between enter activations and the refusal course within the salted mannequin. Salting disrupts adversarial impact by rotating the activation area: GCG-modified prompts (blue) not align with dangerous representations, preserving separation from the refusal subspace
Conclusion and future work
We current LLM salting, a light-weight fine-tuning approach that disrupts jailbreak reuse by rotating inside refusal representations. This method virtually fully neutralizes the success of precomputed GCG jailbreaks on each LLaMA-2 and Vicuna, whereas preserving the mannequin’s efficiency on benign inputs.
Future work may discover making use of salting to bigger fashions and evaluating its robustness in opposition to a broader vary of jailbreak methods, similar to AutoDAN and TAP.
