WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS

Panos Dimitrakopoulos; Paraskevas  Pegios; Steven McDonagh; Sotirios A. Tsaftaris

WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS

Panos Dimitrakopoulos, Paraskevas Pegios, Steven McDonagh, Sotirios A. Tsaftaris

School of Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We introduce semantic overconfidence as the phenomenon where a model’s output probability remains invariant regard-less of the presence or absence of a semantically strong but class-irrelevant features in the image. We adopt generative models to introduce such types of features and create three datasets of factual and counterfactual pairs to study model predictive probabilities. Our experiments indicate that neural networks indeed suffer from this type of semantic challenge. We also provide empirical evidence suggesting that Bayesian methods have the potential to alleviate this problem.

Original language	English
Title of host publication	IEEE International Conference on Image Processing Workshops
Subtitle of host publication	Generative AI for World Simulations and Communications & Celebrating 40 Years of Excellence in Education: Honoring Professor Aggelos Katsaggelos
Number of pages	6
Publication status	Accepted/In press - 3 Jul 2025
Event	The 2025 IEEE International Conference on Image Processing - Anchorage, United States Duration: 14 Sept 2025 → 18 Sept 2025 https://2025.ieeeicip.org/

Conference

Conference	The 2025 IEEE International Conference on Image Processing
Abbreviated title	ICIP 2025
Country/Territory	United States
City	Anchorage
Period	14/09/25 → 18/09/25
Internet address	https://2025.ieeeicip.org/

Embargoed Document

semantic_overconfidence_final-1
Accepted author manuscript, 743 KB

1 Finished
2 Active

CHAI - EPSRC AI Hub for Causality in Healthcare AI with Real Data
Tsaftaris, S. (Principal Investigator), Baillie, K. (Co-investigator), Beentjes, S. (Co-investigator), Crowley, E. (Co-investigator), Escudero Rodriguez, J. (Co-investigator), Guthrie, B. (Co-investigator), Harrison, E. (Co-investigator), King, R. (Co-investigator), Parsons, M. (Co-investigator) & Whiteley, W. (Co-investigator)
Engineering and Physical Sciences Research Council
1/02/24 → 31/01/29
Project: Research
Canon Medical / RAEng Senior Research Fellow in Healthcare AI
Tsaftaris, S. (Principal Investigator)
Canon Medical Research Europe
31/03/19 → 30/06/26
Project: Research
From trivial representations to learning concepts in AI by exploiting unique data
Tsaftaris, S. (Principal Investigator)
Engineering and Physical Sciences Research Council
1/02/23 → 16/05/25
Project: Research

Cite this

Dimitrakopoulos, P., Pegios, P., McDonagh, S., & Tsaftaris, S. A. (Accepted/In press). WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS. In IEEE International Conference on Image Processing Workshops: Generative AI for World Simulations and Communications & Celebrating 40 Years of Excellence in Education: Honoring Professor Aggelos Katsaggelos

Dimitrakopoulos, Panos ; Pegios, Paraskevas ; McDonagh, Steven et al. / WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS. IEEE International Conference on Image Processing Workshops: Generative AI for World Simulations and Communications & Celebrating 40 Years of Excellence in Education: Honoring Professor Aggelos Katsaggelos. 2025.

@inproceedings{4748bf98d43e4b2abc3e694364dac301,

title = "WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS",

abstract = "We introduce semantic overconfidence as the phenomenon where a model{\textquoteright}s output probability remains invariant regard-less of the presence or absence of a semantically strong but class-irrelevant features in the image. We adopt generative models to introduce such types of features and create three datasets of factual and counterfactual pairs to study model predictive probabilities. Our experiments indicate that neural networks indeed suffer from this type of semantic challenge. We also provide empirical evidence suggesting that Bayesian methods have the potential to alleviate this problem.",

author = "Panos Dimitrakopoulos and Paraskevas Pegios and Steven McDonagh and Tsaftaris, \{Sotirios A.\}",

year = "2025",

month = jul,

day = "3",

language = "English",

booktitle = "IEEE International Conference on Image Processing Workshops",

note = "The 2025 IEEE International Conference on Image Processing, ICIP 2025 ; Conference date: 14-09-2025 Through 18-09-2025",

url = "https://2025.ieeeicip.org/",

}

Dimitrakopoulos, P, Pegios, P, McDonagh, S & Tsaftaris, SA 2025, WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS. in IEEE International Conference on Image Processing Workshops: Generative AI for World Simulations and Communications & Celebrating 40 Years of Excellence in Education: Honoring Professor Aggelos Katsaggelos. The 2025 IEEE International Conference on Image Processing, Anchorage, Alaska, United States, 14/09/25.

WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS. / Dimitrakopoulos, Panos; Pegios, Paraskevas ; McDonagh, Steven et al.
IEEE International Conference on Image Processing Workshops: Generative AI for World Simulations and Communications & Celebrating 40 Years of Excellence in Education: Honoring Professor Aggelos Katsaggelos. 2025.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS

AU - Dimitrakopoulos, Panos

AU - Pegios, Paraskevas

AU - McDonagh, Steven

AU - Tsaftaris, Sotirios A.

PY - 2025/7/3

Y1 - 2025/7/3

N2 - We introduce semantic overconfidence as the phenomenon where a model’s output probability remains invariant regard-less of the presence or absence of a semantically strong but class-irrelevant features in the image. We adopt generative models to introduce such types of features and create three datasets of factual and counterfactual pairs to study model predictive probabilities. Our experiments indicate that neural networks indeed suffer from this type of semantic challenge. We also provide empirical evidence suggesting that Bayesian methods have the potential to alleviate this problem.

AB - We introduce semantic overconfidence as the phenomenon where a model’s output probability remains invariant regard-less of the presence or absence of a semantically strong but class-irrelevant features in the image. We adopt generative models to introduce such types of features and create three datasets of factual and counterfactual pairs to study model predictive probabilities. Our experiments indicate that neural networks indeed suffer from this type of semantic challenge. We also provide empirical evidence suggesting that Bayesian methods have the potential to alleviate this problem.

M3 - Conference contribution

BT - IEEE International Conference on Image Processing Workshops

T2 - The 2025 IEEE International Conference on Image Processing

Y2 - 14 September 2025 through 18 September 2025

ER -

WHEN PIGS FLY: EVALUATING SEMANTIC OVERCONFIDENCE IN DEEP NEURAL NETWORK CLASSIFIERS

Abstract

Conference

Embargoed Document

Fingerprint

Projects

CHAI - EPSRC AI Hub for Causality in Healthcare AI with Real Data

Canon Medical / RAEng Senior Research Fellow in Healthcare AI

From trivial representations to learning concepts in AI by exploiting unique data

Cite this