Fool your (vision and) language model with embarrassingly simple permutations

Yongshuo Zong, Tingyang Yu, Ruchika Chavhan, Bingchen Zhao, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on. This raises an urgent need to carefully analyse their robustness so that stakeholders can understand if and when such models are trustworthy enough to be relied upon in any given application. In this paper, we highlight a specific vulnerability in popular models, namely permutation sensitivity in multiple-choice question answering (MCQA). Specifically, we show empirically that popular models are vulnerable to adversarial permutation in answer sets for multiple-choice prompting, which is surprising as models should ideally be as invariant to prompt permutation as humans are. These vulnerabilities persist across various model sizes, and exist in very recent language and vision-language models.
Original languageEnglish
Title of host publicationProceedings of the 41st International Conference on Machine Learning
DOIs
Publication statusAccepted/In press - 15 May 2024
EventThe 41st International Conference on Machine Learning - Vienna, Austria
Duration: 21 Jul 202427 Jul 2024
https://icml.cc/

Conference

ConferenceThe 41st International Conference on Machine Learning
Abbreviated titleICML 2024
Country/TerritoryAustria
CityVienna
Period21/07/2427/07/24
Internet address

Fingerprint

Dive into the research topics of 'Fool your (vision and) language model with embarrassingly simple permutations'. Together they form a unique fingerprint.

Cite this