Phantom Transfer: Data Poisoning Can Survive Data-Level Defences

I worked on Phantom Transfer at LASR Labs under the supervision of Mary Phuong at Google DeepMind. The paper was accepted at NeurIPS 2026, with me as a co-first author. We investigated a simple question: can you stop a data-poisoning attack by cleaning the training data, even if you know exactly how the attack works?

We developed an attack that hides unwanted preferences in otherwise ordinary instruction-tuning data. A model trained to give shorter answers also picks up a preference for a particular country, person or religion, despite filtering out references to the target. This works across different model families: for example, data generated by Gemma can transfer a preference to GPT-4.1.

We tested 11 data-level defences, including LLM filters told exactly what to look for and another model rewriting every training example. None reliably stopped the attack. We also showed that these preferences can be activated by a secret passphrase. The main takeaway is that apparently clean training data does not guarantee the behaviour you intended, which motivates checking the trained model itself as well.