ARROU: Adversarially Robust Rank-One Unlearning

ARROU is a lightweight attacker-in-the-loop framework for targeted unlearning in large language models. It combines the efficiency and locality of rank-one model editing with latent adversarial training to make an edit more resistant to paraphrases and adversarial attempts that re-evoke the forgotten knowledge.

The inner attacker searches selected transformer layers for latent directions that recover a targeted answer. Those directions form an attack subspace, while activations from benign retain prompts form a safe subspace. ARROU then constructs a robust key and refusal-oriented value before applying a constrained r-ROME update at a decisive MLP layer. This attack-and-edit loop is repeated until the relapse indicators stabilize.

Experiments use Llama 3.2 1B and evaluate targeted forgetting and retained utility on WMDP and TOFU. ARROU is compared with the original model, gradient-ascent or LAT baselines, ROME, and r-ROME.