LMS IS OPTIMAL

We show that the celebrated LMS (Least-Mean Squares) adaptive algorithm is optimal. In other words, the LMS algorithm, which has long been regarded as an approximate least-mean squares solution, is in fact an exact minimizer of a certain so-called error norm. In particular, the LMS mini mizes the energy gain from the disturbances to the predicted errors, while the so-called normalized LMS minimizes the energy gain from the disturbances to the filtered errors. Moreover, since these algorithms are central filters, they minimize a certain exponential cost function and are thus also risk-sensitive optimal (in the sense of Whittle). We discuss the various implications of these results, and show how they provide theoretical justification for the widely observed excellent robustness properties of the LMS filter.

Back to Adaptive Control, Filtering, and Signal Processing Table of Contents