Measuring the Biases and Effectiveness of Content-Style Disentanglement

Xiao Liu*, Spyridon Thermos, Gabriele Valvano, Agisilaos Chartsias, Alison O'Neil, Sotirios A. Tsaftaris

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

A recent spate of state-of-the-art semi- and un-supervised solutions disentangle and encode image "content" into a spatial tensor and image appearance or "style" into a vector, to achieve good performance in spatially equivariant tasks (e.g. image-to-image translation). To achieve this, they employ different model design, learning objective, and data biases. While considerable effort has been made to measure disentanglement in vector representations, and assess its impact on task performance, such analysis for (spatial) content-style disentanglement is lacking. In this paper, we conduct an empirical study to investigate the role of different biases in content-style disentanglement settings and unveil the relationship between the degree of disentanglement and task performance. In particular, we consider the setting where we: (i) identify key design choices and learning constraints for three popular content-style disentanglement models; (ii) relax or remove such constraints in an ablation fashion; and (iii) use two metrics to measure the degree of disentanglement and assess its effect on each task performance. Our experiments reveal that there is a “sweet spot” between disentanglement, task performance and - surprisingly – content interpretability, suggesting that blindly forcing for higher disentanglement can hurt model performance and content factors semanticness. Our findings, as well as the used task-independent metrics, can be used to guide the design and selection of new models for tasks where content-style representations are useful.
Original languageEnglish
Title of host publicationThe 32nd British Machine Vision Conference (BMVC)
PublisherBMVA Press
Publication statusPublished - 25 Nov 2021
EventThe 32nd British Machine Vision Conference - Virtual
Duration: 22 Nov 202125 Nov 2021
https://www.bmvc2021.com/

Conference

ConferenceThe 32nd British Machine Vision Conference
Abbreviated titleBMVC 2021
Period22/11/2125/11/21
Internet address

Fingerprint

Dive into the research topics of 'Measuring the Biases and Effectiveness of Content-Style Disentanglement'. Together they form a unique fingerprint.

Cite this