Compressive Independent Component Analysis: Theory and Algorithms

Mikey Sheehan, Michael E. Davies

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Compressive learning forms the exciting intersection between compressed sensing and statistical learning where one exploits sparsity of the learning model to reduce the memory and/or computational complexity of the algorithms used to solve the learning task. In this paper, we look at the independent component analysis (ICA) model through the compressive learning lens. In particular, we show that solutions to the cumulant based ICA model have particular structure that induces a low dimensional model set that resides in the cumulant tensor space. By showing a restricted isometry property holds for random cumulants e.g. Gaussian ensembles, we prove the existence of a compressive ICA scheme. Thereafter, we propose two algorithms of the form of an iterative projection gradient (IPG) and an alternating steepest descent (ASD) algorithm for compressive ICA, where the order of compression asserted from the restricted isometry property is realised through empirical results. We provide analysis of the CICA algorithms including the effects of finite samples. The effects of compression are characterised by a trade-off between the sketch size and the statistical efficiency of the ICA estimates. By considering synthetic and real datasets, we show the substantial memory gains achieved over well-known ICA algorithms by using one of the proposed CICA algorithms.
Original languageEnglish
JournalInformation and Inference: a Journal of the IMA
Early online date19 May 2022
DOIs
Publication statusE-pub ahead of print - 19 May 2022

Fingerprint

Dive into the research topics of 'Compressive Independent Component Analysis: Theory and Algorithms'. Together they form a unique fingerprint.

Cite this