Spectral editing of activations for large language model alignment

Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo Ponti, Shay B. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Large language models (LLMs) often exhibit undesirable behaviours, such as generating untruthful or biased content. Editing their internal representations has been shown to be effective in mitigating such behaviours on top of the existing alignment methods. We propose a novel inference-time editing method, namely spectral editing of activations (SEA), to project the input representations into directions with maximal covariance with the positive demonstrations (e.g., truthful) while minimising covariance with the negative demonstrations (e.g., hallucinated). We also extend our method to non-linear editing using feature functions. We run extensive experiments on benchmarks concerning truthfulness and bias with six open-source LLMs of different sizes and model families. The results demonstrate the superiority of SEA in effectiveness, generalisation to similar tasks, as well as computation and data efficiency. We also show that SEA editing only has a limited negative impact on other model capabilities.
Original languageEnglish
Title of host publication Advances in Neural Information Processing Systems 37 (NeurIPS 2024) Main Conference Track
EditorsAmir Globerson, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub Tomczak, Cheng Zhang
PublisherCurran Associates Inc
Pages56958-56987
Number of pages30
ISBN (Electronic)9798331314385
Publication statusPublished - 16 Dec 2024
EventThe Thirty-Eighth Annual Conference on Neural Information Processing Systems - Vancouver Convention Center, Vancouver, Canada
Duration: 10 Dec 202415 Dec 2024
Conference number: 38
https://neurips.cc/Conferences/2024

Publication series

NameAdvances in Neural Information Processing Systems
PublisherCurran Associates, Inc.
Volume37
ISSN (Print)1049-5258

Conference

ConferenceThe Thirty-Eighth Annual Conference on Neural Information Processing Systems
Abbreviated titleNeurIPS 2024
Country/TerritoryCanada
CityVancouver
Period10/12/2415/12/24
Internet address

Fingerprint

Dive into the research topics of 'Spectral editing of activations for large language model alignment'. Together they form a unique fingerprint.

Cite this