Hamiltonian Latent Operators for content and motion disentanglement in image sequences

Asif Khan, Amos J Storkey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We introduce \textit{HALO} -- a deep generative model utilising HAmiltonian Latent Operators to reliably disentangle content and motion information in image sequences. The \textit{content} represents summary statistics of a sequence, and \textit{motion} is a dynamic process that determines how information is expressed in any part of the sequence. By modelling the dynamics as a Hamiltonian motion, important desiderata are ensured: (1) the motion is reversible, (2) the symplectic, volume-preserving structure in phase space means paths are continuous and are not divergent in the latent space. Consequently, the nearness of sequence frames is realised by the nearness of their coordinates in the phase space, which proves valuable for disentanglement and long-term sequence generation. The sequence space is generally comprised of different types of dynamical motions. To ensure long-term separability and allow controlled generation, we associate every motion with a unique Hamiltonian that acts in its respective subspace. We demonstrate the utility of \textit{HALO} by swapping the motion of a pair of sequences, controlled generation, and image rotations.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 35 (NeurIPS 2022)
EditorsS. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
PublisherCurran Associates Inc
Number of pages14
Publication statusPublished - 1 Apr 2023

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258


Dive into the research topics of 'Hamiltonian Latent Operators for content and motion disentanglement in image sequences'. Together they form a unique fingerprint.

Cite this