Fish4Knowledge: Supporting humans in knowledge gathering and question answering wrt marine and environmental monitoring through analysis of multiple video streams

The study of marine ecosystems is vital for understanding environmental effects, such as climate change and the effects of pollution, but is extremely difficult because of the inaccessibility of data. Undersea video data is usable but is tedious to analyse (for both raw video analysis and abstraction over massive sets of observations), and is mainly done by hand or with hand-crafted computational tools. Fish4Knowledge will allow a major increase in the ability to analyse this data: 1) Video analysis will automatically extract information about the observed marine animals which is recorded in an observation database. 2) Interfaces will be designed to allow researchers to formulate and answer higher level questions over that database.

The project will investigate: information abstraction and storage methods for reducing the massive amount of video data (from 10E+15 pixels to 10E+12 units of information), machine and human vocabularies for describing fish, flexible process architectures to process the data and scientific queries and effective specialised user query interfaces. A combination of computer vision, database storage, workflow and human computer interaction methods will be used to achieve this.

The project will use live video feeds from 10 underwater cameras as a testbed for investigating more generally applicable methods for capture, storage, analysis and querying of multiple video streams.

The project will use live video feeds from 10 underwater cameras as a testbed for investigating methods for detecting, tracking, counting and recognising fish. The project will collect a large database of these detections and develop user tools that
allow marine biologists to explore the data and test hypotheses about fish populations and how they change over time.