Integrated Perception with Recurrent Multi-Task Neural Networks

Hakan Bilen, Andrea Vedaldi

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc. However, a major advantage that natural intelligences still have is that they work well for all perceptual problems together, solving them efficiently and coherently in an integrated manner. In order to capture some of these advantages in machine perception, we ask two questions: whether deep neural networks can learn universal image representations, useful not only for a single task but for all of them, and how the solutions to the different tasks can be integrated in this framework. We answer by proposing a new architecture, which we call multinet, in which not only deep image features are shared between tasks, but where tasks can interact in a recurrent manner by encoding the results of their analysis in a common shared representation of the data. In this manner, we show that the performance of individual tasks in standard benchmarks can be improved first by sharing features between them and then, more significantly, by integrating their solutions in the common representation.
Original languageEnglish
Title of host publicationNIPS'16 Proceedings of the 30th International Conference on Neural Information Processing Systems
Place of PublicationBarcelona, Spain
Number of pages9
ISBN (Electronic)978-1-5108-3881-9
Publication statusPublished - 5 Dec 2016
Event30th Annual Conference on Neural Information Processing Systems - Barcelona, Spain
Duration: 5 Dec 201610 Dec 2016


Conference30th Annual Conference on Neural Information Processing Systems
Abbreviated titleNIPS 2016
Internet address

Fingerprint Dive into the research topics of 'Integrated Perception with Recurrent Multi-Task Neural Networks'. Together they form a unique fingerprint.

Cite this