Distributed optimization with arbitrary local solvers

Chenxin Ma, Jakub Konecny, Martin Jaggi, Virginia Smith, Michael I. Jordan, Peter Richtárik, Martin Takáč

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Recent work in this area has been limited by focusing heavily on developing highly specific methods for the distributed environment. These special-purpose methods are often unable to fully leverage the competitive performance of their well-tuned and customized single machine counterparts. Further, they are unable to easily integrate improvements that continue to be made to single machine methods. To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods. We give strong primal–dual convergence rate guarantees for our framework that hold for arbitrary local solvers. We demonstrate the impact of local solver selection both theoretically and in an extensive experimental comparison. Finally, we provide thorough implementation details for our framework, highlighting areas for practical performance gains.
Original languageEnglish
Pages (from-to)813-848
Number of pages36
JournalOptimization Methods and Software
Issue number4
Early online date1 Feb 2017
Publication statusPublished - Jun 2017


Dive into the research topics of 'Distributed optimization with arbitrary local solvers'. Together they form a unique fingerprint.

Cite this