Mini-Batch Primal and Dual Methods for SVMs

Martin Takac; Avleen Bijral; Nathan Srebro; Peter Richtarik

Mini-Batch Primal and Dual Methods for SVMs

Martin Takac, Avleen Bijral, Nathan Srebro, Peter Richtarik

School of Mathematics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We address the issue of using mini-batches in stochastic optimization of SVMs. We show that the same quantity, the spectral norm of the data, controls the parallelization speedup obtained for both primal stochastic subgradient descent (SGD) and stochastic dual coordinate ascent (SCDA) methods and use it to derive novel variants of mini-batched SDCA. Our guarantees for both methods are expressed in terms of the original nonsmooth primal problem based on the hinge-loss.

Original language	English
Title of host publication	JMLR Workshop and Conference Proceedings
Subtitle of host publication	Proceedings of the 30th International Conference on Machine Learning
Pages	1022-1030
Volume	28
Edition	3
Publication status	Published - 2013

Access to Document

http://jmlr.org/proceedings/papers/v28/takac13.pdf

Science and Innovation: Numerical Algorithms and Intelligent Software for the Evolving HPC Platform
Leimkuhler, B.
EPSRC
1/08/09 → 31/07/14
Project: Research

Cite this

@inproceedings{6f4e82716ff14c6d9a9bf2e3c4db1494,

title = "Mini-Batch Primal and Dual Methods for SVMs",

abstract = "We address the issue of using mini-batches in stochastic optimization of SVMs. We show that the same quantity, the spectral norm of the data, controls the parallelization speedup obtained for both primal stochastic subgradient descent (SGD) and stochastic dual coordinate ascent (SCDA) methods and use it to derive novel variants of mini-batched SDCA. Our guarantees for both methods are expressed in terms of the original nonsmooth primal problem based on the hinge-loss.",

author = "Martin Takac and Avleen Bijral and Nathan Srebro and Peter Richtarik",

year = "2013",

language = "English",

volume = "28",

pages = "1022--1030",

booktitle = "JMLR Workshop and Conference Proceedings",

edition = "3",

}

TY - GEN

T1 - Mini-Batch Primal and Dual Methods for SVMs

AU - Takac, Martin

AU - Bijral, Avleen

AU - Srebro, Nathan

AU - Richtarik, Peter

PY - 2013

Y1 - 2013

N2 - We address the issue of using mini-batches in stochastic optimization of SVMs. We show that the same quantity, the spectral norm of the data, controls the parallelization speedup obtained for both primal stochastic subgradient descent (SGD) and stochastic dual coordinate ascent (SCDA) methods and use it to derive novel variants of mini-batched SDCA. Our guarantees for both methods are expressed in terms of the original nonsmooth primal problem based on the hinge-loss.

AB - We address the issue of using mini-batches in stochastic optimization of SVMs. We show that the same quantity, the spectral norm of the data, controls the parallelization speedup obtained for both primal stochastic subgradient descent (SGD) and stochastic dual coordinate ascent (SCDA) methods and use it to derive novel variants of mini-batched SDCA. Our guarantees for both methods are expressed in terms of the original nonsmooth primal problem based on the hinge-loss.

M3 - Conference contribution

VL - 28

SP - 1022

EP - 1030

BT - JMLR Workshop and Conference Proceedings

ER -

Mini-Batch Primal and Dual Methods for SVMs

Abstract

Access to Document

Fingerprint

Projects

Science and Innovation: Numerical Algorithms and Intelligent Software for the Evolving HPC Platform

Cite this