Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators

Nataraj Jammalamadaka; Andrew Zisserman; M. Eichner; Vittorio Ferrari; C. V. Jawahar

doi:10.1007/978-3-642-33712-3_9

Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators

Nataraj Jammalamadaka, Andrew Zisserman, M. Eichner, Vittorio Ferrari, C. V. Jawahar

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Most current vision algorithms deliver their output ‘as is’, without indicating whether it is correct or not. In this paper we propose evaluator algorithms that predict if a vision algorithm has succeeded. We illustrate this idea for the case of Human Pose Estimation (HPE).

We describe the stages required to learn and test an evaluator, including the use of an annotated ground truth dataset for training and testing the evaluator (and we provide a new dataset for the HPE case), and the development of auxiliary features that have not been used by the (HPE) algorithm, but can be learnt by the evaluator to predict if the output is correct or not.

Then an evaluator is built for each of four recently developed HPE algorithms using their publicly available implementations: Eichner and Ferrari [5], Sapp et al. [16], Andriluka et al. [2] and Yang and Ramanan [22]. We demonstrate that in each case the evaluator is able to predict if the algorithm has correctly estimated the pose or not.

Original language	English
Title of host publication	Computer Vision – ECCV 2012
Subtitle of host publication	12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III
Publisher	Springer
Pages	114-128
Number of pages	15
ISBN (Electronic)	978-3-642-33712-3
ISBN (Print)	978-3-642-33711-6
DOIs	https://doi.org/10.1007/978-3-642-33712-3_9
Publication status	Published - 2012

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer Berlin Heidelberg
Volume	7574
ISSN (Print)	0302-9743

Access to Document

10.1007/978-3-642-33712-3_9

Jammalamadaka et al 2012 Has my Algorithm SucceededAccepted author manuscript, 1.87 MB

http://link.springer.com/chapter/10.1007%2F978-3-642-33712-3_9

Cite this

Jammalamadaka, N., Zisserman, A., Eichner, M., Ferrari, V., & Jawahar, C. V. (2012). Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators. In Computer Vision – ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III (pp. 114-128). (Lecture Notes in Computer Science; Vol. 7574). Springer. https://doi.org/10.1007/978-3-642-33712-3_9

@inproceedings{657555f3c53a410a8736f1cec5a9f63d,

title = "Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators",

abstract = "Most current vision algorithms deliver their output {\textquoteleft}as is{\textquoteright}, without indicating whether it is correct or not. In this paper we propose evaluator algorithms that predict if a vision algorithm has succeeded. We illustrate this idea for the case of Human Pose Estimation (HPE).We describe the stages required to learn and test an evaluator, including the use of an annotated ground truth dataset for training and testing the evaluator (and we provide a new dataset for the HPE case), and the development of auxiliary features that have not been used by the (HPE) algorithm, but can be learnt by the evaluator to predict if the output is correct or not.Then an evaluator is built for each of four recently developed HPE algorithms using their publicly available implementations: Eichner and Ferrari [5], Sapp et al. [16], Andriluka et al. [2] and Yang and Ramanan [22]. We demonstrate that in each case the evaluator is able to predict if the algorithm has correctly estimated the pose or not.",

author = "Nataraj Jammalamadaka and Andrew Zisserman and M. Eichner and Vittorio Ferrari and Jawahar, {C. V.}",

year = "2012",

doi = "10.1007/978-3-642-33712-3_9",

language = "English",

isbn = "978-3-642-33711-6",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "114--128",

booktitle = "Computer Vision – ECCV 2012",

address = "United Kingdom",

}

Jammalamadaka, N, Zisserman, A, Eichner, M, Ferrari, V & Jawahar, CV 2012, Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators. in Computer Vision – ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III. Lecture Notes in Computer Science, vol. 7574, Springer, pp. 114-128. https://doi.org/10.1007/978-3-642-33712-3_9

Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators. / Jammalamadaka, Nataraj; Zisserman, Andrew; Eichner, M. et al.
Computer Vision – ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III. Springer, 2012. p. 114-128 (Lecture Notes in Computer Science; Vol. 7574).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators

AU - Jammalamadaka, Nataraj

AU - Zisserman, Andrew

AU - Eichner, M.

AU - Ferrari, Vittorio

AU - Jawahar, C. V.

PY - 2012

Y1 - 2012

N2 - Most current vision algorithms deliver their output ‘as is’, without indicating whether it is correct or not. In this paper we propose evaluator algorithms that predict if a vision algorithm has succeeded. We illustrate this idea for the case of Human Pose Estimation (HPE).We describe the stages required to learn and test an evaluator, including the use of an annotated ground truth dataset for training and testing the evaluator (and we provide a new dataset for the HPE case), and the development of auxiliary features that have not been used by the (HPE) algorithm, but can be learnt by the evaluator to predict if the output is correct or not.Then an evaluator is built for each of four recently developed HPE algorithms using their publicly available implementations: Eichner and Ferrari [5], Sapp et al. [16], Andriluka et al. [2] and Yang and Ramanan [22]. We demonstrate that in each case the evaluator is able to predict if the algorithm has correctly estimated the pose or not.

AB - Most current vision algorithms deliver their output ‘as is’, without indicating whether it is correct or not. In this paper we propose evaluator algorithms that predict if a vision algorithm has succeeded. We illustrate this idea for the case of Human Pose Estimation (HPE).We describe the stages required to learn and test an evaluator, including the use of an annotated ground truth dataset for training and testing the evaluator (and we provide a new dataset for the HPE case), and the development of auxiliary features that have not been used by the (HPE) algorithm, but can be learnt by the evaluator to predict if the output is correct or not.Then an evaluator is built for each of four recently developed HPE algorithms using their publicly available implementations: Eichner and Ferrari [5], Sapp et al. [16], Andriluka et al. [2] and Yang and Ramanan [22]. We demonstrate that in each case the evaluator is able to predict if the algorithm has correctly estimated the pose or not.

U2 - 10.1007/978-3-642-33712-3_9

DO - 10.1007/978-3-642-33712-3_9

M3 - Conference contribution

SN - 978-3-642-33711-6

T3 - Lecture Notes in Computer Science

SP - 114

EP - 128

BT - Computer Vision – ECCV 2012

PB - Springer

ER -

Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators

Abstract

Publication series

Access to Document

Fingerprint

Cite this