Birds are hard to see and sometimes shy of humans, so call counts are the way that population sizes are usually estimated. Automatic acoustic recorders have made recording bird calls easier – but recorders generate a lot of data – a LOT of data! It can end up being just too time-consuming to analyse that data effectively and the potential information remains unmined.
Stephen Marsland, Nirosha Priyadarshani, Julius Juodakis and Isabel Castro from the School of Mathematics and Statistics at Wellington’s Victoria University and the School of Agriculture and Environment at Massey are developing software to analyse that data automatically and the good news is it’s open-source and easy for both research scientists and citizen science enthusiasts to use.
They describe the software, its development process and how it performed in a practical test of recordings from Wellington’s Zealandia Sanctuary, in an article recently published in Methods in Ecology and Evolution, a journal of the British Ecological Society. The article is freely available online.
“The relatively recent development of reliable automatic acoustic recorders has led to a huge amount of data being collected. Unfortunately, the subsequent discovery that these data are noisy, that many of the bird calls are far away from the microphone, and that the recordings still require tedious and error‐prone manual analysis means that much of these data are neglected, and the information it could potentially provide is not utilized. Although mathematicians, computer scientists and engineers have started to work on methods for the analysis of these data, it has proven to be a challenging topic, particularly once the need for high levels of accuracy in identification is considered.”
With funding assistance from the Royal Society of New Zealand Marsden Fund, National Science Challenge on Science for Technological Innovation and The Kiwi Recovery Group, the authors have developed their open‐source software to facilitate this.
“The software is part of a research project that aims at enabling all those interested in the management of wildlife (particular avian species in the first instance) to gain reliable information from acoustic field recordings. The software, which we call AviaNZ (with the ending being to emphasize its New Zealand roots, although it is not specific to New Zealand, nor to birds, although they have been our primary focus to date), is designed to be as user‐friendly as possible. We have considered the main tasks that people want to achieve with acoustic recordings, and to facilitate precisely those tasks, namely:
- Manually annotate bird calls from spectrograms and/or amplitude plots.
- Automatically detect bird calls and other noises.
- Easily train specialized filters to recognize the calls of particular species.
- Work in batch mode to run specialized filters over folders full of field recordings.
- Quickly and easily review the results of the recognition process (whether human or automatic).
- Produce output in forms that can be quickly interpreted in statistical software.
“We have particularly focused on the future‐proofing of our method, by which we mean that the outputs produced by the automatic processing (or by another human) can be quickly reviewed and corrected. Our aim was to ensure that the accuracy of any analyses that have been performed in the past, which would be purely manual, are comparable with those of the current time, by keeping the human in the loop – albeit it with a reduced role – and also with the analyses of the future, when the requirements for human involvement will be even further diminished.”
The software is free, runs under Windows, Linux and MacOS, and is available at https://avianz.net. According to the authors, it offers several advantages over other available products.
“While there is both free and commercial software available for bioacoustics (see for example the Wikipedia list), we do not know of others that focus on long‐time field recordings, facilitate human review and allow new filters for taxa of interest to the user to be developed within the software.”
The software offers a variety of outputs, chosen to match the standard methods of statistical analysis of bird abundance.
“The internal representation of bird calls that have been identified in AviaNZ is a simple human‐readable Python list. As such, these are easily transferred between users, or loaded into other programs as they are. However, we also provide the outputs of the program for each sound file processed in three different pages of a spreadsheet, which are as follows:
- start and end times of every bird call detected.
- presence/absence in each recording of any species of interest.
- presence/absence of each species of interest for pre‐defined time intervals (such as every 10 second block, or every minute).
“We have observed that when people find a sound that they are unsure about, they like to extract and share it with others, as both an image and a sound file. We therefore provide facilities to easily export the current section of the spectrogram in both formats. This has the added the benefit of enabling the easy creation of a ‘cheat sheet’ of examples of both standard calls of particular species, commonly confused species, and interesting variations, as can be seen on our website and also through the program.”
AviaNZ has the ability to train specialized filters to recognize individual species which allows fast, accurate data analysis.
“We have chosen to use filters for individual species for several reasons:
- the range of birds that are present in any given place is usually relatively limited compared to the worldwide range of species, and the user can easily specify this through their choice of filters;
- it increases the accuracy of identification of individual species that are of conservation interest (and while the statistical methods of abundance estimation can deal with false negatives, false positives are problematic);
- it enables confounding calls (which are identified as several species by their filters) to be easily identified and processed separately; it enables users to specify their own filters for their own particular species and noise levels, and to share them with others.
So what exactly does ‘training a filter’ involve?
“Creating a filter is a relatively simple process. The user chooses a number of files and manually segments the calls. The trained filters tend to work most effectively if the calls range from close to the recorder through to a good distance away, and if the recordings have different levels of noise in them. Typically, a small set of examples of each type of call (10–30) seems to be sufficient. The software then uses these examples to automatically establish the various expected parameters of the calls, and to train the filter, and the user evaluates the success of the filter on an independent test set.”
“Given our emphasis on keeping the human in the loop, particularly to avoid false positives for each species, we have chosen to break the outputs of a filter into the following three categories: very high likelihood that this call is from a particular species, very high likelihood that this call is not from that species and inbetween. It is particularly important that the user examines the calls in this final category.”
AviaNZ offers two methods for the human user to quickly and easily review the annotations provided by either another human, or the automatic filters. Any errors that were identified by the human are stored and can be used later for improving the filters.
A demonstration of the software, comparing manual and automatic detection of calls of the little spotted kiwi is given in the article.
“Audio recordings were collected from seven autonomous recorders during the night of 6 October 2018 from 22:00 to 04:00 in Zealandia. Recorders were attached to trees at 1.5 m in an area of approximately 600 by 800 m. Their positions were chosen to include a broad range of ambient noise sources, including a stream, a windy hilltop and a wind turbine.”
“Zealandia houses a translocated population of little spotted kiwi, which we chose as our species of interest. Both sexes call intermittently throughout the night. There are also other night callers, some of which can be confused with the kiwi, particularly the ruru. We demonstrate AviaNZ using different workflows on these data.”
“In the manual processing pipeline, observers loaded the recordings into AviaNZ, visually inspected the spectrogram and marked each kiwi call, using audio playback as needed. In the automated pipeline, the software was trained on manually annotated data using recordings from the same locations, but on the nights of September 14th and September 15th. Separate filters were trained for the male and female calls. Parameters were selected to obtain approximately 90% sensitivity on the training data. Eight 5 minute files were used to train the female filter (a total of 16 calls) and 31 files for the male (58 calls). After processing, the output of the filters was reviewed.”
While the automatic detection did produce false positives, human correction of these was far faster than manual review of the whole sound file.
“In the manual processing, 568 annotations were made, comprising a total call duration of 145 min. Annotation length ranged from 0.3 to 64 seconds; excluding annotations < 5 s, which usually indicate fragments of a single call, 445 calls remained. In the automated processing of the same recordings, our software identified 1,046 min of putative calls, 663 male and 383 female. Subsequent human review rejected most of these as false positives, leaving 147 min of calls as the final result from this pipeline (111 male, 36 female), including some missed in manual processing.”
“Manual processing took 319 min to process the 42 hr of recording (note that these are fairly sparse recordings, this time would grow massively with recordings featuring more calls), while automatic processing (which can be performed offline) took 119 min on a standard desktop machine, and manual review took 132 min (so about an hour less in elapsed time). Training a filter (including manual annotation of the training data, and testing a variety of parameter choices) took 4–5 hr.”
“While the number of false positives is a concern—and we are currently developing the methods to reduce them without negatively affecting the detection rate—the processing and review still takes far less time, and we believe that this is a good basis to develop from.”
The authors believe their software has potential for use by a variety of users, as well as being customizable and extendable for experienced specialists.
“AviaNZ is intended to be easy‐to‐use software that provides precisely the functionality that citizen scientists, wildlife managers and researchers in conservation and ecology need to analyse acoustic field recordings to monitor their species of interest. By keeping the human in the loop, we expect to be able to calibrate the outputs of the software in the future and also against manual data previously collected, and by designing it to incorporate new and improved filters, we expect the software to be reasonably future‐proofed. We are currently developing new methods for creating filters, for both species and individuals, and on the use of statistical sampling of calls to estimate abundance, starting from protocols for the use of acoustic recorders.”
Field testing has confirmed the program’s usability.
“AviaNZ has been field‐tested by a variety of community groups and conservationists, who have subsequently used it for tasks as diverse as detecting the presence of kiwi in a hitherto unknown area, call counts of a variety of New Zealand birds and comparison of human and manual detection of calls.”
“AviaNZ is a freely available open‐source standalone program. Our experience shows that it can be used by anybody quickly and easily. However, for experienced users it is easily customizable and extendable. By enabling everybody involved with acoustic bird recording to quickly and easily analyse their own data, while future‐proofing it by keeping the human in the loop, we are enabling acoustic field recordings to meet their potential.”
The full research article is published in Methods in Ecology and Evolution and freely available online.
