Variable selection: A statistical dependence perspective

Sohan Seth, Jose C Principe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Measures of statistical dependence such as the correlation coefficient and mutual information have been widely used in variable selection. The use of correlation has been inspired by the concept of regression whereas the use of mutual information has been largely motivated by information theory. In a statistical sense, however, the concept of dependence is much broader, and extends beyond correlation and mutual information. In this paper, we explore the fundamental notion of statistical dependence in the context of variable selection. In particular, we discuss the properties of dependence as proposed by Rényi, and evaluate their significance in the variable selection context. We, also, explore a measure of dependence that satisfies most of these desired properties, and discuss its applicability as a substitute for correlation coefficient and mutual information. Finally, we compare these measures of dependence to select important variables for regression with real world data.
Original languageEnglish
Title of host publicationMachine learning and applications (icmla), 2010 ninth international conference on
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages6
ISBN (Print)978-0-7695-4300-0
Publication statusPublished - 12 Dec 2010


Dive into the research topics of 'Variable selection: A statistical dependence perspective'. Together they form a unique fingerprint.

Cite this