Edinburgh Research Explorer

On Optimal Probabilities in Stochastic Coordinate Descent Methods

Research output: Contribution to journalArticle

Related Edinburgh Organisations

Access status

Open

Documents

http://arxiv.org/abs/1310.3438
Original languageEnglish
Pages (from-to)1233-1243
JournalOptimization letters
Volume10
Issue number6
Early online date2 Jul 2015
DOIs
StatePublished - 1 Aug 2016

Abstract

We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities to the sets to optimize the bound. The complexity and practical performance of the method can outperform its uniform variant by an order of magnitude. Surprisingly, the strategy of updating a single randomly selected coordinate per iteration---with optimal probabilities---may require less iterations, both in theory and practice, than the strategy of updating all coordinates at every iteration.

Research areas

  • stat.ML, cs.DC, math.OC

Download statistics

No data available

ID: 19896581