Detection of bodies in maritime rescue operations using Unmanned Aerial Vehicles with multispectral cameras

In this work, we use Unmanned Aerial Vehicles (UAVs) equipped with multispectral cameras to search for bodies in maritime rescue operations. A series of flights were performed in open water scenarios in the northwest of Spain, using a certified aquatic rescue dummy in dangerous areas and real people when the weather conditions allowed it. The multispectral images were aligned and used to train a Convolutional Neural Network for body detection. An exhaustive evaluation was performed in order to assess the best combination of spectral channels for this task. Three approaches based on a MobileNet topology were evaluated, using 1) the full image, 2) a sliding window, and 3) a precise localization method. The first method classifies an input image as containing a body or not, the second uses a sliding window to yield a class for each sub-image, and the third uses transposed convolutions returning a binary output in which the body pixels are marked. In all cases, the MobileNet architecture was modified by adding custom layers and preprocessing the input to align the multispectral camera channels. Evaluation shows that the proposed methods yield reliable results, obtaining the best classification performance when combining Green, Red Edge and Near IR channels. We conclude that the precise localization approach is the most suitable method, obtaining a similar accuracy as the sliding window but achieving a spatial localization close to 1m. The presented system is about to be implemented for real maritime rescue operations carried out by Babcock Mission Critical Services Spain.