Estimating Dependency Structures for non-Gaussian Components with Linear and Energy Correlations

S. Hiroaki, M.U. Gutmann, H. Shouno, A. Hyärinen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the ICA components. It would be very useful to estimate the dependency structure from data. However, most models have concentrated on higher-order correlations such as energy correlations, neglecting linear correlations. Linear correlations might be a strong and informative form of a dependency for some real data sets, but they are usually completely removed by ICA and related methods, and not analyzed at all. In this paper, we propose a probabilistic model of non-Gaussian components which are allowed to have both linear and energy correlations. The dependency structure of the components is explicitly parametrized by a parameter matrix, which defines an undirected graphical model over the latent components. Furthermore, the estimation of the parameter matrix is shown to be particularly simple because using score matching, the objective function is a quadratic form. Using artificial data, we demonstrate that the proposed method is able to estimate non-Gaussian components and their dependency structures, as it is designed to do. When applied to natural images and outputs of simulated complex cells in the primary visual cortex, novel dependencies between the estimated features are discovered.
Original languageEnglish
Title of host publicationProceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), JMLR: WCP volume 33
PublisherJournal of Machine Learning Research - Proceedings Track
Number of pages9
Publication statusPublished - 2014


Dive into the research topics of 'Estimating Dependency Structures for non-Gaussian Components with Linear and Energy Correlations'. Together they form a unique fingerprint.

Cite this