The iNaturalist Species Classification and Detection Dataset

Grant Van Horn; Oisin Mac Aodha; Yang Song; Yin Cui; Chen Sun; Alex Shepard; Hartwig Adam; Pietro Perona; Serge Belongie

doi:10.1109/CVPR.2018.00914

The iNaturalist Species Classification and Detection Dataset

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, Serge Belongie

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current non-ensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.

Original language	English
Title of host publication	2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	8769-8778
Number of pages	10
ISBN (Electronic)	978-1-5386-6420-9
ISBN (Print)	978-1-5386-6421-6
DOIs	https://doi.org/10.1109/CVPR.2018.00914
Publication status	Published - 17 Dec 2018
Event	2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Salt Lake City, United States Duration: 18 Jun 2018 → 22 Jun 2018 http://cvpr2018.thecvf.com/

Publication series

Name
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)	1063-6919
ISSN (Electronic)	2575-7075

Conference

Conference	2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated title	CVPR 2018
Country/Territory	United States
City	Salt Lake City
Period	18/06/18 → 22/06/18
Internet address	http://cvpr2018.thecvf.com/

Keywords

crowdsourcing
embedded systems
image processing
learning (artificial intelligence)
low dimensional embeddings
crowd workers
interpretable embeddings
model worker biases
noisy crowd
visual context
context embedding networks
CENs
data collections
crowd similarity
multi-dimensional concept
crowd embedding learning
image set
Visualization
Context modeling
Feature extraction
Training
Noise measurement
Data models
Crowdsourcing

Access to Document

10.1109/CVPR.2018.00914Licence: All Rights Reserved

https://arxiv.org/abs/1707.06642
http://openaccess.thecvf.com/CVPR2018.py
https://ieeexplore.ieee.org/document/8579012Licence: All Rights Reserved

Oisin Mac Aodha
- School of Informatics - Lecturer in Machine Learning
- Institute for Adaptive and Neural Computation
- Data Science and Artificial Intelligence
Person: Academic: Research Active

Cite this

@inproceedings{a164040629264e19a4ad868507cd67cc,

title = "The iNaturalist Species Classification and Detection Dataset",

abstract = "Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current non-ensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.",

keywords = "crowdsourcing, embedded systems, image processing, learning (artificial intelligence), low dimensional embeddings, crowd workers, interpretable embeddings, model worker biases, noisy crowd, visual context, context embedding networks, CENs, data collections, crowd similarity, multi-dimensional concept, crowd embedding learning, image set, Visualization, Context modeling, Feature extraction, Training, Noise measurement, Data models, Crowdsourcing",

author = "{Van Horn}, Grant and {Mac Aodha}, Oisin and Yang Song and Yin Cui and Chen Sun and Alex Shepard and Hartwig Adam and Pietro Perona and Serge Belongie",

year = "2018",

month = dec,

day = "17",

doi = "10.1109/CVPR.2018.00914",

language = "English",

isbn = "978-1-5386-6421-6",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "8769--8778",

booktitle = "2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition",

address = "United States",

note = "2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018 ; Conference date: 18-06-2018 Through 22-06-2018",

url = "http://cvpr2018.thecvf.com/",

}

Van Horn, G, Mac Aodha, O, Song, Y, Cui, Y, Sun, C, Shepard, A, Adam, H, Perona, P & Belongie, S 2018, The iNaturalist Species Classification and Detection Dataset. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Institute of Electrical and Electronics Engineers (IEEE), pp. 8769-8778, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, United States, 18/06/18. https://doi.org/10.1109/CVPR.2018.00914

TY - GEN

T1 - The iNaturalist Species Classification and Detection Dataset

AU - Van Horn, Grant

AU - Mac Aodha, Oisin

AU - Song, Yang

AU - Cui, Yin

AU - Sun, Chen

AU - Shepard, Alex

AU - Adam, Hartwig

AU - Perona, Pietro

AU - Belongie, Serge

PY - 2018/12/17

Y1 - 2018/12/17

N2 - Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current non-ensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.

AB - Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current non-ensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.

KW - crowdsourcing

KW - embedded systems

KW - image processing

KW - learning (artificial intelligence)

KW - low dimensional embeddings

KW - crowd workers

KW - interpretable embeddings

KW - model worker biases

KW - noisy crowd

KW - visual context

KW - context embedding networks

KW - CENs

KW - data collections

KW - crowd similarity

KW - multi-dimensional concept

KW - crowd embedding learning

KW - image set

KW - Visualization

KW - Context modeling

KW - Feature extraction

KW - Training

KW - Noise measurement

KW - Data models

KW - Crowdsourcing

U2 - 10.1109/CVPR.2018.00914

DO - 10.1109/CVPR.2018.00914

M3 - Conference contribution

SN - 978-1-5386-6421-6

SP - 8769

EP - 8778

BT - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Y2 - 18 June 2018 through 22 June 2018

ER -

The iNaturalist Species Classification and Detection Dataset

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Profiles

Oisin Mac Aodha

Cite this