Unsupervised Batch Normalization

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Batch Normalization is a widely used tool in neural networks to improve the generalization and convergence of training. However, on small datasets due to the difficulty of obtaining unbiased batch statistics it cannot be applied effectively. In some cases, even if there is only a small labeled dataset available, there are larger unlabeled datasets from the same distribution. We propose using such unlabeled examples to calculate batch normalization statistics, which we call Unsupervised Batch Normalization (UBN). We show that using unlabeled examples for batch statistic calculations results in a reduction of the bias of the statistics, as well as regularization leveraging the data manifold. UBN is easy to implement, computationally inexpensive and can be applied to a variety problems. We report results on monocular depth estimation, where obtaining dense labeled examples is difficult and expensive. Using unlabeled samples, and UBN, we obtain an increase in accuracy of more than 6% on the KITTI dataset, compared to using traditional batch normalization only on the labeled samples.
Original languageEnglish
Title of host publication2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages3994-3999
Number of pages6
ISBN (Electronic)978-1-7281-9360-1
ISBN (Print)978-1-7281-9361-8
DOIs
Publication statusPublished - 28 Jul 2020
EventIEEE Conference on Computer Vision and Pattern Recognition Workshop 2020: Visual Learning With Limited Labels: Zero-Shot, Few-Shot, Any-Shot, and Cross-Domain Few-Shot Learning - Virtual conference, Seattle, United States
Duration: 19 Jun 202019 Jun 2020
https://www.learning-with-limited-labels.com/

Publication series

Name
PublisherIEEE
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Workshop

WorkshopIEEE Conference on Computer Vision and Pattern Recognition Workshop 2020
Abbreviated titleCVPRW 2020
CountryUnited States
CitySeattle
Period19/06/2019/06/20
Internet address

Fingerprint

Dive into the research topics of 'Unsupervised Batch Normalization'. Together they form a unique fingerprint.

Cite this