Region-Based Semantic Segmentation with End-to-End Training

Holger Caesar; Jasper Uijlings; Vittorio Ferrari

doi:10.1007/978-3-319-46448-0_23

Region-Based Semantic Segmentation with End-to-End Training

Holger Caesar, Jasper Uijlings, Vittorio Ferrari

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods based on region classification offer proper spatial support for appearance measurements, but typically operate in two separate stages, none of which targets pixel labeling performance at the end of the pipeline. More recent fully convolutional methods are capable of end-to-end training for the final pixel labeling, but resort to fixed patches as spatial support. We show how to modify modern region-based approaches to enable end-to-end training for semantic segmentation. This is achieved via a differentiable region-to-pixel layer and a differentiable free-form Region-of-Interest pooling layer. Our method improves the state-of-the-art in terms of class-average accuracy with 64.0%64.0% on SIFT Flow and 49.9%49.9% on PASCAL Context, and is particularly accurate at object boundaries.

Original language	English
Title of host publication	Computer Vision -- ECCV 2016
Subtitle of host publication	14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Place of Publication	Cham
Publisher	Springer
Pages	381-397
Number of pages	17
ISBN (Electronic)	978-3-319-46448-0
ISBN (Print)	978-3-319-46447-3
DOIs	https://doi.org/10.1007/978-3-319-46448-0_23
Publication status	Published - 17 Sept 2016
Event	14th European Conference on Computer Vision 2016 - Amsterdam, Netherlands Duration: 8 Oct 2016 → 16 Oct 2016 http://www.eccv2016.org/

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer International Publishing
Volume	9905
ISSN (Print)	0302-9743

Conference

Conference	14th European Conference on Computer Vision 2016
Abbreviated title	ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	8/10/16 → 16/10/16
Internet address	http://www.eccv2016.org/

Access to Document

10.1007/978-3-319-46448-0_23

caesar16eccvAccepted author manuscript, 10.5 MB

http://link.springer.com/chapter/10.1007%2F978-3-319-46448-0_23

VISCUL - Visual culture for image understanding
Ferrari, V.
EU government bodies
1/01/13 → 31/12/17
Project: Research

Cite this

Caesar, H., Uijlings, J., & Ferrari, V. (2016). Region-Based Semantic Segmentation with End-to-End Training. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I (pp. 381-397). (Lecture Notes in Computer Science; Vol. 9905). Springer. https://doi.org/10.1007/978-3-319-46448-0_23

Caesar, Holger ; Uijlings, Jasper ; Ferrari, Vittorio. / Region-Based Semantic Segmentation with End-to-End Training. Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. Cham : Springer, 2016. pp. 381-397 (Lecture Notes in Computer Science).

@inproceedings{1d2c5f11bf234931874a6fa4e6c75c74,

title = "Region-Based Semantic Segmentation with End-to-End Training",

abstract = "We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods based on region classification offer proper spatial support for appearance measurements, but typically operate in two separate stages, none of which targets pixel labeling performance at the end of the pipeline. More recent fully convolutional methods are capable of end-to-end training for the final pixel labeling, but resort to fixed patches as spatial support. We show how to modify modern region-based approaches to enable end-to-end training for semantic segmentation. This is achieved via a differentiable region-to-pixel layer and a differentiable free-form Region-of-Interest pooling layer. Our method improves the state-of-the-art in terms of class-average accuracy with 64.0%64.0% on SIFT Flow and 49.9%49.9% on PASCAL Context, and is particularly accurate at object boundaries.",

author = "Holger Caesar and Jasper Uijlings and Vittorio Ferrari",

year = "2016",

month = sep,

day = "17",

doi = "10.1007/978-3-319-46448-0_23",

language = "English",

isbn = "978-3-319-46447-3",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "381--397",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision -- ECCV 2016",

address = "United Kingdom",

note = "14th European Conference on Computer Vision 2016, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

url = "http://www.eccv2016.org/",

}

Caesar, H, Uijlings, J & Ferrari, V 2016, Region-Based Semantic Segmentation with End-to-End Training. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I. Lecture Notes in Computer Science, vol. 9905, Springer, Cham, pp. 381-397, 14th European Conference on Computer Vision 2016, Amsterdam, Netherlands, 8/10/16. https://doi.org/10.1007/978-3-319-46448-0_23

Region-Based Semantic Segmentation with End-to-End Training. / Caesar, Holger; Uijlings, Jasper; Ferrari, Vittorio.
Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. Cham: Springer, 2016. p. 381-397 (Lecture Notes in Computer Science; Vol. 9905).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Region-Based Semantic Segmentation with End-to-End Training

AU - Caesar, Holger

AU - Uijlings, Jasper

AU - Ferrari, Vittorio

PY - 2016/9/17

Y1 - 2016/9/17

N2 - We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods based on region classification offer proper spatial support for appearance measurements, but typically operate in two separate stages, none of which targets pixel labeling performance at the end of the pipeline. More recent fully convolutional methods are capable of end-to-end training for the final pixel labeling, but resort to fixed patches as spatial support. We show how to modify modern region-based approaches to enable end-to-end training for semantic segmentation. This is achieved via a differentiable region-to-pixel layer and a differentiable free-form Region-of-Interest pooling layer. Our method improves the state-of-the-art in terms of class-average accuracy with 64.0%64.0% on SIFT Flow and 49.9%49.9% on PASCAL Context, and is particularly accurate at object boundaries.

AB - We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods based on region classification offer proper spatial support for appearance measurements, but typically operate in two separate stages, none of which targets pixel labeling performance at the end of the pipeline. More recent fully convolutional methods are capable of end-to-end training for the final pixel labeling, but resort to fixed patches as spatial support. We show how to modify modern region-based approaches to enable end-to-end training for semantic segmentation. This is achieved via a differentiable region-to-pixel layer and a differentiable free-form Region-of-Interest pooling layer. Our method improves the state-of-the-art in terms of class-average accuracy with 64.0%64.0% on SIFT Flow and 49.9%49.9% on PASCAL Context, and is particularly accurate at object boundaries.

U2 - 10.1007/978-3-319-46448-0_23

DO - 10.1007/978-3-319-46448-0_23

M3 - Conference contribution

SN - 978-3-319-46447-3

T3 - Lecture Notes in Computer Science

SP - 381

EP - 397

BT - Computer Vision -- ECCV 2016

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

PB - Springer

CY - Cham

T2 - 14th European Conference on Computer Vision 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Caesar H, Uijlings J, Ferrari V. Region-Based Semantic Segmentation with End-to-End Training. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part I. Cham: Springer. 2016. p. 381-397. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-46448-0_23

Region-Based Semantic Segmentation with End-to-End Training

Abstract / Description of output

Publication series

Conference

Access to Document

Fingerprint

Projects

VISCUL - Visual culture for image understanding

Cite this