WILDetect: using Machine Learning to identify marine species

Highlights

• The study aims to develop a methodology for detecting maritime bio ecosystems
• Methodology employs Machine Learning and Reinforcement Learning techniques
• Employs several hybrid techniques to segment, split and count birds and can be applied to other species
• The outcome of the study is expected to benefit the entire environmental modelling community

Why was the study undertaken?

As marine development increases and technology progresses, there is a greater need to observe the large maritime environment efficiently, and to map the habitats of marine life, characteristics of species, and the diverse mix of maritime industries around them.

A new non-parametric approach, WILDetect, uses a mixture of supervised Machine Learning (ML) and Reinforcement Learning (RL) techniques to perform automated censuses in a highly dynamic marine ecosystem.

The experimental results suggest that automated data processing techniques – tailored for specific species – can be helpful in performing time-intensive marine wildlife censuses efficiently. They also help to establish ecological platforms/models to understand the underlying causes of trends in species populations, alongside the ecological change.

Why use Machine Learning?

Repetitive surveying of very large areas to observe trends and population fluctuations can have huge financial and time costs. In a typical marine survey programme, around half a million images might be taken over 12 months for a specific area. It is a labour-intensive task to separate this survey into positive images with targeted objects and negative images with no objects, and to count the objects in the images deemed positive (usually around 5%).

Long-term data that utilise standardised and structured methodologies are ideal for quantifying change in species populations. Unfortunately, such data do not exist for most biogeographic regions, due to the difficulties and high cost of manual methods. Automation of this work using an automated intelligent computer system would help the development of effective prospective environmental models with realistic inputs.

The study proposes a new supervised Machine Learning (ML) approach supported by Reinforcement Learning (RL) enabling user-model-data interaction that can detect, split and count birds, in particular, offshore gannets, in an automated decision-making way with high accuracy rates.

The proposed approach shows a new direction for the detection of species, particularly small species with a diverse background. Most importantly, it is a new direction for the classification of multispecies even if there is a strong resemblance between them. Current techniques (i.e. off-the-shelf approaches) cannot converge to a desired solution with high accuracy rates based on the features of datasets.

Why focus on birds?

Seabird population changes are good indicators of long-term and large-scale change in marine ecosystems, and are strongly influenced by threats to marine and coastal ecosystems (e.g. entanglement in fishing gear, overfishing of food sources, climate change, pollution, disturbance, direct exploitation, development, energy production). Research suggests that overall offshore bird populations are decreasing, and this loss of bird abundance signals an urgent need to address threats to avert future avifaunal collapse and associated loss of ecosystem integrity, function, and services.

APEM hold a wide range of gannet data, with geographical positions obtained from all around the world. This species is the focus of the study, which aims to test the developed approaches to help perform further autonomous bird censuses, paving the way for automated classification and counting of multispecies.

Our work

APEM conduct offshore digital wildlife surveys for the renewables sector, reliably capturing imagery all year round, in all lighting conditions and in sea states up to four. The data are captured on a variety of sensor formats, including both 35 mm and medium format from various manufacturers, in both single camera and multiple camera configurations, depending on the project requirements. The images are collected by these advanced cameras mounted in a small twin-engine aeroplane within a route in which all regions of interest are surveyed.

APEM aeroplane during a remotely-sensed aerial survey using advanced aerial high resolution photogrammetry

About the data

APEM have established a library consisting of around 1 million snags (cropped images with objects of interest). We aimed to incorporate all possible targeted positive images into the methodology, either for training and testing or evaluation and validation, to provide a large and representative training dataset to achieve good detection.

Three feature extraction techniques were employed in the methodology: Haar Cascades, Local Binary Patterns (LBP) and Histogram of Oriented Gradients (HOG). Each of these techniques acquires different features of objects using different mathematical modelling.

The accuracy rate of recognition is increased by combining 3 techniques, which is depicted by the yellow line in the figure below.

Discussion

From a conservationist point of view, prevention of regional and global extinction of species during industrial developments and environmental changes (e.g. climate change, habitat loss with rapid urbanisation and coastal disturbance, toxic pesticide use) is a social responsibility. In this sense, a species whose population is in decline needs to be identified urgently and should be protected with higher priority before it is too late.

Automated detection, locating, and monitoring of marine life along with the industry around the habitats of this ecosystem may be helpful to reveal current impacts, model future possible ecological trends, and determine required policies which would lead accordingly to a reduced ecological footprint and increased sustainability.

Advanced tools that enable effective monitoring of species are needed to observe and predict the likely effects of environmental changes on species, mostly caused by indispensable industrial developments, and to take urgent proper actions, e.g. rebuilding natural habitats to maintain/increase species counts.

Conclusion

The outcome of the study is expected to benefit the entire environmental modelling community.

With the applications of emerging fields of science and technology in new and existing industries, prominent companies and research organisations have been recently developing and deploying evolving technologies supported by location-independent advanced maritime mechatronics systems (AMMSs) to explore and exploit the resources in the tough marine landscape. This massively evolving industry, enabling enormous continuous human control in the maritime, has the potential to impact the marine ecosystem dramatically; in particular, the seabed, birds, turtles, and fish.

WILDetect can be primarily deployed by environmentalists, researchers, authorities, and policymakers to monitor the marine ecosystem for fulfilling their goals effectively. As marine development moves further out to sea, accurate modelling will be vital to monitor these impacts.

Within a holistic view, we aim to study other bird species and other marine species (e.g. turtles) as well as man-made maritime objects to observe the bio marine ecosystem with the possible environmental footprint in the short, mid, and long-term. Moreover, the automatic classification of maritime ecosystems based on a variety of species will be in our future plans to support all types of environmental models with near-real-time information with multiple species.

Read the full study, entitled WILDetect: An intelligent platform to perform airborne wildlife census automatically in the marine ecosystem using an ensemble of learning techniques and computer vision in ScienceDirect here.