Deep Clustering with Concrete K-Means

Boyan Gao; Yongxin Yang; Henry Gouk; Timothy M. Hospedales

doi:10.1109/ICASSP40776.2020.9053265

Deep Clustering with Concrete K-Means

Boyan Gao, Yongxin Yang, Henry Gouk, Timothy M. Hospedales

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We address the problem of simultaneously learning a k-means clustering and deep feature representation from unlabelled data, which is of interest due to the potential for deep k-means to outperform traditional two-step feature extraction and shallow clustering strategies. We achieve this by developing a gradient estimator for the non-differentiable k-means objective via the Gumbel-Softmax reparameterisation trick. In contrast to previous attempts at deep clustering, our concrete k-means model can be optimised with respect to the canonical k-means objective and is easily trained end-to-end without resorting to time consuming alternating optimisation techniques. We demonstrate the efficacy of our method on standard clustering benchmarks.

Original language	English
Title of host publication	ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	Institute of Electrical and Electronics Engineers
Pages	4252-4256
Number of pages	5
ISBN (Electronic)	978-1-5090-6631-5
ISBN (Print)	978-1-5090-6632-2
DOIs	https://doi.org/10.1109/ICASSP40776.2020.9053265
Publication status	Published - 14 May 2020
Event	2020 IEEE International Conference on Acoustics, Speech, and Signal Processing - Barcelona, Spain Duration: 4 May 2020 → 8 May 2020 Conference number: 45

Publication series

Name
Publisher	IEEE
ISSN (Print)	1520-6149
ISSN (Electronic)	2379-190X

Conference

Conference	2020 IEEE International Conference on Acoustics, Speech, and Signal Processing
Abbreviated title	ICASSP 2020
Country/Territory	Spain
City	Barcelona
Period	4/05/20 → 8/05/20

Keywords / Materials (for Non-textual outputs)

Deep Clustering
Unsupervised Learning
Gradient Estimation

Access to Document

10.1109/ICASSP40776.2020.9053265Licence: All Rights Reserved

Deep Clustering with Concrete_GAO_DOA24012020_AFVAccepted author manuscript, 200 KB

https://ieeexplore.ieee.org/document/9053265Licence: All Rights Reserved

Cite this

@inproceedings{c1451665f95a43ee80a37ce10a2f76f8,

title = "Deep Clustering with Concrete K-Means",

abstract = "We address the problem of simultaneously learning a k-means clustering and deep feature representation from unlabelled data, which is of interest due to the potential for deep k-means to outperform traditional two-step feature extraction and shallow clustering strategies. We achieve this by developing a gradient estimator for the non-differentiable k-means objective via the Gumbel-Softmax reparameterisation trick. In contrast to previous attempts at deep clustering, our concrete k-means model can be optimised with respect to the canonical k-means objective and is easily trained end-to-end without resorting to time consuming alternating optimisation techniques. We demonstrate the efficacy of our method on standard clustering benchmarks.",

keywords = "Deep Clustering, Unsupervised Learning, Gradient Estimation",

author = "Boyan Gao and Yongxin Yang and Henry Gouk and Hospedales, {Timothy M.}",

year = "2020",

month = may,

day = "14",

doi = "10.1109/ICASSP40776.2020.9053265",

language = "English",

isbn = "978-1-5090-6632-2",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "4252--4256",

booktitle = "ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 ; Conference date: 04-05-2020 Through 08-05-2020",

}

Gao, B, Yang, Y, Gouk, H & Hospedales, TM 2020, Deep Clustering with Concrete K-Means. in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers, pp. 4252-4256, 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, Barcelona, Spain, 4/05/20. https://doi.org/10.1109/ICASSP40776.2020.9053265

TY - GEN

T1 - Deep Clustering with Concrete K-Means

AU - Gao, Boyan

AU - Yang, Yongxin

AU - Gouk, Henry

AU - Hospedales, Timothy M.

N1 - Conference code: 45

PY - 2020/5/14

Y1 - 2020/5/14

N2 - We address the problem of simultaneously learning a k-means clustering and deep feature representation from unlabelled data, which is of interest due to the potential for deep k-means to outperform traditional two-step feature extraction and shallow clustering strategies. We achieve this by developing a gradient estimator for the non-differentiable k-means objective via the Gumbel-Softmax reparameterisation trick. In contrast to previous attempts at deep clustering, our concrete k-means model can be optimised with respect to the canonical k-means objective and is easily trained end-to-end without resorting to time consuming alternating optimisation techniques. We demonstrate the efficacy of our method on standard clustering benchmarks.

AB - We address the problem of simultaneously learning a k-means clustering and deep feature representation from unlabelled data, which is of interest due to the potential for deep k-means to outperform traditional two-step feature extraction and shallow clustering strategies. We achieve this by developing a gradient estimator for the non-differentiable k-means objective via the Gumbel-Softmax reparameterisation trick. In contrast to previous attempts at deep clustering, our concrete k-means model can be optimised with respect to the canonical k-means objective and is easily trained end-to-end without resorting to time consuming alternating optimisation techniques. We demonstrate the efficacy of our method on standard clustering benchmarks.

KW - Deep Clustering

KW - Unsupervised Learning

KW - Gradient Estimation

U2 - 10.1109/ICASSP40776.2020.9053265

DO - 10.1109/ICASSP40776.2020.9053265

M3 - Conference contribution

SN - 978-1-5090-6632-2

SP - 4252

EP - 4256

BT - ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers

T2 - 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing

Y2 - 4 May 2020 through 8 May 2020

ER -

Deep Clustering with Concrete K-Means

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this