What’s the Point: Semantic Segmentation with Point Supervision

Amy  Bearman; Olga  Russakovsky; Vittorio Ferrari; Li  Fei-Fei

doi:10.1007/978-3-319-46478-7_34

What’s the Point: Semantic Segmentation with Point Supervision

Amy Bearman, Olga Russakovsky, Vittorio Ferrari, Li Fei-Fei

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

The semantic image segmentation task presents a trade-off between test time accuracy and training time annotation cost. Detailed per-pixel annotations enable training accurate models but are very time-consuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along with a novel objectness potential in the training loss function of a CNN model. Experimental results on the PASCAL VOC 2012 benchmark reveal that the combined effect of point-level supervision and objectness potential yields an improvement of 12.9%12.9% mIOU over image-level supervision. Further, we demonstrate that models trained with point-level supervision are more accurate than models trained with image-level, squiggle-level or full supervision given a fixed annotation budget.

Original language	English
Title of host publication	Computer Vision – ECCV 2016
Subtitle of host publication	14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII
Publisher	Springer
Pages	549-565
Number of pages	16
ISBN (Electronic)	978-3-319-46478-7
ISBN (Print)	978-3-319-46477-0
DOIs	https://doi.org/10.1007/978-3-319-46478-7_34
Publication status	Published - 16 Sept 2016
Event	14th European Conference on Computer Vision 2016 - Amsterdam, Netherlands Duration: 8 Oct 2016 → 16 Oct 2016 http://www.eccv2016.org/

Publication series

Name	Lecture Notes in Computer Science (LNCS)
Publisher	Springer International Publishing
Volume	9911
ISSN (Print)	0302-9743

Conference

Conference	14th European Conference on Computer Vision 2016
Abbreviated title	ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	8/10/16 → 16/10/16
Internet address	http://www.eccv2016.org/

Access to Document

10.1007/978-3-319-46478-7_34

bearman16cvprAccepted author manuscript, 3.73 MB

http://link.springer.com/chapter/10.1007/978-3-319-46478-7_34

VISCUL - Visual culture for image understanding
Ferrari, V.
EU government bodies
1/01/13 → 31/12/17
Project: Research

Cite this

Bearman, A., Russakovsky, O., Ferrari, V., & Fei-Fei, L. (2016). What’s the Point: Semantic Segmentation with Point Supervision. In Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII (pp. 549-565). (Lecture Notes in Computer Science (LNCS); Vol. 9911). Springer. https://doi.org/10.1007/978-3-319-46478-7_34

@inproceedings{4e74b5b2b3334262b5ecbcefe0829651,

title = "What{\textquoteright}s the Point: Semantic Segmentation with Point Supervision",

abstract = "The semantic image segmentation task presents a trade-off between test time accuracy and training time annotation cost. Detailed per-pixel annotations enable training accurate models but are very time-consuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along with a novel objectness potential in the training loss function of a CNN model. Experimental results on the PASCAL VOC 2012 benchmark reveal that the combined effect of point-level supervision and objectness potential yields an improvement of 12.9%12.9% mIOU over image-level supervision. Further, we demonstrate that models trained with point-level supervision are more accurate than models trained with image-level, squiggle-level or full supervision given a fixed annotation budget.",

author = "Amy Bearman and Olga Russakovsky and Vittorio Ferrari and Li Fei-Fei",

year = "2016",

month = sep,

day = "16",

doi = "10.1007/978-3-319-46478-7_34",

language = "English",

isbn = "978-3-319-46477-0",

series = "Lecture Notes in Computer Science (LNCS)",

publisher = "Springer",

pages = "549--565",

booktitle = "Computer Vision – ECCV 2016",

address = "United Kingdom",

note = "14th European Conference on Computer Vision 2016, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

url = "http://www.eccv2016.org/",

}

Bearman, A, Russakovsky, O, Ferrari, V & Fei-Fei, L 2016, What’s the Point: Semantic Segmentation with Point Supervision. in Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII. Lecture Notes in Computer Science (LNCS), vol. 9911, Springer, pp. 549-565, 14th European Conference on Computer Vision 2016, Amsterdam, Netherlands, 8/10/16. https://doi.org/10.1007/978-3-319-46478-7_34

What’s the Point: Semantic Segmentation with Point Supervision. / Bearman, Amy ; Russakovsky, Olga ; Ferrari, Vittorio et al.
Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII. Springer, 2016. p. 549-565 (Lecture Notes in Computer Science (LNCS); Vol. 9911).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - What’s the Point: Semantic Segmentation with Point Supervision

AU - Bearman, Amy

AU - Russakovsky, Olga

AU - Ferrari, Vittorio

AU - Fei-Fei, Li

PY - 2016/9/16

Y1 - 2016/9/16

N2 - The semantic image segmentation task presents a trade-off between test time accuracy and training time annotation cost. Detailed per-pixel annotations enable training accurate models but are very time-consuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along with a novel objectness potential in the training loss function of a CNN model. Experimental results on the PASCAL VOC 2012 benchmark reveal that the combined effect of point-level supervision and objectness potential yields an improvement of 12.9%12.9% mIOU over image-level supervision. Further, we demonstrate that models trained with point-level supervision are more accurate than models trained with image-level, squiggle-level or full supervision given a fixed annotation budget.

AB - The semantic image segmentation task presents a trade-off between test time accuracy and training time annotation cost. Detailed per-pixel annotations enable training accurate models but are very time-consuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along with a novel objectness potential in the training loss function of a CNN model. Experimental results on the PASCAL VOC 2012 benchmark reveal that the combined effect of point-level supervision and objectness potential yields an improvement of 12.9%12.9% mIOU over image-level supervision. Further, we demonstrate that models trained with point-level supervision are more accurate than models trained with image-level, squiggle-level or full supervision given a fixed annotation budget.

U2 - 10.1007/978-3-319-46478-7_34

DO - 10.1007/978-3-319-46478-7_34

M3 - Conference contribution

SN - 978-3-319-46477-0

T3 - Lecture Notes in Computer Science (LNCS)

SP - 549

EP - 565

BT - Computer Vision – ECCV 2016

PB - Springer

T2 - 14th European Conference on Computer Vision 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

What’s the Point: Semantic Segmentation with Point Supervision

Abstract / Description of output

Publication series

Conference

Access to Document

Fingerprint

Projects

VISCUL - Visual culture for image understanding

Cite this