Asymptotic Analysis of the LMS Algorithm with Momentum

Laszlo Gerencser, Balazs Csanad Csaji, Sotirios Sabanis

Research output: Contribution to conferencePaperpeer-review

Abstract

A widely studied filtering algorithm in signal processing is the least mean square (LMS) method, due to B. Widrow and T. Hoff, 1960. A popular extension of the LMS algorithm, which is also important in deep learning, is the LMS method with momentum, originated by S. Roy and J.J. Shynk back in 1988. This is a fixed gain (or constant step-size) version of the LMS method modified by an additional momentum term that is proportional to the last correction term. Recently, a certain equivalence of the two methods has been rigorously established by K. Yuan, B. Ying and A.H. Sayed, assuming martingale difference gradient noise. The purpose of this paper is to present the outline of a significantly simpler and more transparent asymptotic analysis of the LMS algorithm with momentum under the assumption of stationary, ergodic and mixing signals.
Original languageEnglish
Pages3062-3067
DOIs
Publication statusPublished - 21 Jan 2019
Event2018 IEEE Conference on Decision and Control (CDC) - Miami Beach, FL
Duration: 17 Dec 201819 Dec 2018

Conference

Conference2018 IEEE Conference on Decision and Control (CDC)
Period17/12/1819/12/18

Fingerprint Dive into the research topics of 'Asymptotic Analysis of the LMS Algorithm with Momentum'. Together they form a unique fingerprint.

Cite this