Detection and classification of fish species from acoustic data

Acoustic data from echo sounders and sonars are used as input to stock assessment in fisheries management models. Assessment and interpretation of acoustic data is challenging and require extensive expert work. Visual Intelligence collaborates with the Institute of Marine research (IMR) to develop models for detection and classification of fish species from acoustic data.

Acoustic data is collected from echosounders that measure how sound waves are reflected off the seabed and marine organisms below. The output is annotated by experts and will be utilized to train our deep learning models, but there are many related challenges that needs to be addressed.

Acoustic data gathered from the same location, but at different frequencies.

The volume of annotations is large, but the quality is variable, and the annotations are not suited for direct use for training deep learning models. Furthermore, characteristics of surveys and equipment may change over time. Finally, there will also be auxiliary information available from the surveys that can provide useful information.

For future use in stock assessment important issues are also challenges related to providing confidence and uncertainty measures and providing explainability and reliability. Solutions will also need to enable prototyping at IMR, where predictions will be tested for use in abundance assessment.

For acoustic data early work has investigated the use of deep learning for semantic segmentation of sand eel schools. Annotated surveys for more species will be made available during 2021.

An activity on acoustic classification of more species with different characteristics will be started. We will also start working on suitable models for exploitation of contextual information, such as how to include trawl samples and other auxiliary information to improve the performance of the network. Another activity related to analysis of acoustic data will focus on semi-supervised approaches. This to reduce the dependency on large amounts of annotated to better deal with an increasing volume of datasets that may present both new characteristics and new species. The aim is to investigate different semi-supervised models and approaches that can provide good prediction performance with only a few annotated data.


At a Visual Intelligence seminar Olav Brautaset from NR presented his releated work on marine acoustic classification.

Further reading

Performing objective measurements in ultrasound images
March 8, 2021
Exploiting limited data to perform objective measurements in ultrasound images of the heart.
Detection of sea mammals from aerial imagery
December 18, 2020
Better solutions are needed to estimate the populations of sea mammals, such as breeding seals, from aerial images of the sea ice.
Deep learning and AI in the medical domain
January 19, 2021
Overcoming the challenges of limited training data in the medical domain and laying the fundamentals for explainability and reliability.