Prediction-Guided Distillation for Dense Object Detection

Chenhongyi Yang*, Mateusz Ochal, Amos J Storkey, Elliot J Crowley

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Real-world object detection models should be cheap and accurate. Knowledge distillation (KD) can boost the accuracy of a small, cheap detection model by leveraging useful information from a larger teacher model. However, a key challenge is identifying the most informative features produced by the teacher for distillation. In this work, we show that only a very small fraction of features within a ground-truth bounding box are responsible for a teacher’s high detection performance. Based on this, we propose Prediction-Guided Distillation (PGD), which focuses distillation on these key predictive regions of the teacher and yields considerable gains in performance over many existing KD baselines. In addition, we propose an adaptive weighting scheme over the key regions to smooth out their influence and achieve even better performance. Our proposed approach outperforms current state-of-the-art KD baselines on a variety of advanced one-stage detection architectures. Specifically, on the COCO dataset, our method achieves between +3.1% and +4.6% AP improvement using ResNet-101 and ResNet-50 as the teacher and student backbones, respectively. On the CrowdHuman dataset, we achieve +3.2% and +2.0% improvements in MR and AP, also using these backbones. Our code is available at https://github.com/ChenhongyiYang/PGD.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022.
Subtitle of host publication17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part IX
PublisherSpringer
Pages123-138
Volume13669
ISBN (Electronic)978-3-031-20077-9
ISBN (Print)978-3-031-20076-2
DOIs
Publication statusPublished - 6 Nov 2022

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume13669
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Dive into the research topics of 'Prediction-Guided Distillation for Dense Object Detection'. Together they form a unique fingerprint.

Cite this