Detection and classification of fish species from acoustic data

Acoustic data from echo sounders and sonars are used as input to stock assessment in fisheries management models. Assessment and interpretation of acoustic data is challenging and require extensive expert work. Visual Intelligence collaborates with the Institute of Marine research (IMR) to develop models for detection and classification of fish species from acoustic data.

Acoustic data is collected from echosounders that measure how sound waves are reflected off the seabed and marine organisms below. The output is annotated by experts and will be utilized to train our deep learning models, but there are many related challenges that needs to be addressed.

Acoustic data gathered from the same location, but at different frequencies.

The volume of annotations is large, but the quality is variable, and the annotations are not suited for direct use for training deep learning models. Furthermore, characteristics of surveys and equipment may change over time. Finally, there will also be auxiliary information available from the surveys that can provide useful information.

For future use in stock assessment important issues are also challenges related to providing confidence and uncertainty measures and providing explainability and reliability. Solutions will also need to enable prototyping at IMR, where predictions will be tested for use in abundance assessment.

For acoustic data early work has investigated the use of deep learning for semantic segmentation of sand eel schools. Annotated surveys for more species will be made available during 2021.

An activity on acoustic classification of more species with different characteristics will be started. We will also start working on suitable models for exploitation of contextual information, such as how to include trawl samples and other auxiliary information to improve the performance of the network. Another activity related to analysis of acoustic data will focus on semi-supervised approaches. This to reduce the dependency on large amounts of annotated to better deal with an increasing volume of datasets that may present both new characteristics and new species. The aim is to investigate different semi-supervised models and approaches that can provide good prediction performance with only a few annotated data.


At a Visual Intelligence seminar Olav Brautaset from NR presented his releated work on marine acoustic classification.

Further reading

Oil-spill detection and characterization of thickness
December 18, 2020
Visual Intelligence collaborates with KSAT to develop new models for detecting and characterizing oil spills.
Opening the black box of AI
January 19, 2021
Deep learning and AI models must become interpretable, explainable and reliable before they can be utilized in complex domains.
Deep learning and AI in the medical domain
January 19, 2021
Overcoming the challenges of limited training data in the medical domain and laying the fundamentals for explainability and reliability.