| Literature DB >> 34188884 |
Sebastian Lopez-Marcano1,2, Eric L Jinks1, Christina A Buelow1, Christopher J Brown1, Dadong Wang2, Branislav Kusy3, Ellen M Ditria1, Rod M Connolly1.
Abstract
Animal movement studies are conducted to monitor ecosystem health, understand ecological dynamics, and address management and conservation questions. In marine environments, traditional sampling and monitoring methods to measure animal movement are invasive, labor intensive, costly, and limited in the number of individuals that can be feasibly tracked. Automated detection and tracking of small-scale movements of many animals through cameras are possible but are largely untested in field conditions, hampering applications to ecological questions.Here, we aimed to test the ability of an automated object detection and object tracking pipeline to track small-scale movement of many individuals in videos. We applied the pipeline to track fish movement in the field and characterize movement behavior. We automated the detection of a common fisheries species (yellowfin bream, Acanthopagrus australis) along a known movement passageway from underwater videos. We then tracked fish movement with three types of tracking algorithms (MOSSE, Seq-NMS, and SiamMask) and evaluated their accuracy at characterizing movement.We successfully detected yellowfin bream in a multispecies assemblage (F1 score =91%). At least 120 of the 169 individual bream present in videos were correctly identified and tracked. The accuracies among the three tracking architectures varied, with MOSSE and SiamMask achieving an accuracy of 78% and Seq-NMS 84%.By employing this integrated object detection and tracking pipeline, we demonstrated a noninvasive and reliable approach to studying fish behavior by tracking their movement under field conditions. These cost-effective technologies provide a means for future studies to scale-up the analysis of movement across many visual monitoring systems.Entities:
Keywords: computer vision; connectivity; deep learning; dispersal; machine learning; object tracking; underwater video
Year: 2021 PMID: 34188884 PMCID: PMC8216886 DOI: 10.1002/ece3.7656
Source DB: PubMed Journal: Ecol Evol ISSN: 2045-7758 Impact factor: 2.912
FIGURE 1The study location in Tweed River Estuary, Australia, showing the camera array deployed in a fish passageway (two ended white arrow) between the rock wall channel and the seagrass meadow (green polygon). Each set of cameras consisted of three underwater cameras that recorded for 1 hr during a flood tide. Set 1 faced north and set 2 faced south. The distance between cameras (~3 m) and between sets (20 m) ensured nonoverlapping field of views. Map data: NearMap 2020
FIGURE 2Interaction between the object detection model and tracking architectures. The object detection model activates all three tracking architectures. For MOSSE and SiamMask, the tracker continues for 4 frames after the initial detection. For Seq‐NMS, the movement was determined by calculating the vector direction between two detections. For all architectures, a check was made to determine if the tracker continued, stopped, or a new tracker started. For MOSSE and SiamMask, the check was made after 4 tracking frames from the first detection. For Seq‐NMS, the check was made for all frames after the first detection. The interaction between detections and tracker occurred through the whole length of a video where the object detection model detected a yellowfin bream and was carried for all frames, videos, and cameras. All trackers provided a direction of movement for each frame where the interaction between the detection and tracking occurred successfully
Object detection mAP50 and the evaluation results of the Mask R‐CNN yellowfin bream model. The confusion matrix is shown as counts of individual fish, where the true positives were the correct detection of yellowfin bream. Yellowfin bream not detected were false negatives and misidentified objects were false positives
| Task | mAP50 | Confusion matrix | Average precision | Average recall | F1 | |||
|---|---|---|---|---|---|---|---|---|
| Ground‐truth | True positives | False positives | False negatives | |||||
| Object detection | 81% | 169 | 148 | 8 | 21 | 95% | 88% | 91% |
Confusion matrix for the three object tracking architectures (MOSSE, Seq‐NMS, and SiamMask) are shown as counts of individual fish, where the true positive means a bream was detected and tracked correctly for ≥50% of the time when it appeared on a video frame, otherwise, it was false negative. False positives were misidentified objects (e.g., algae or other fish) that were detected and tracked
| Architecture | Confusion matrix | Average precision | Average recall | F1 | ||
|---|---|---|---|---|---|---|
| True positives | False positives | False negatives | ||||
| MOSSE | 123 | 23 | 46 | 84% | 73% | 78% |
| Seq‐NMS | 129 | 9 | 40 | 93% | 76% | 84% |
| SiamMask | 121 | 19 | 48 | 86% | 72% | 78% |
FIGURE 3Proportion of the movement angles (up, down, right, left) for the ground‐truth and the three tracking architectures and for the two camera sets (Set 1: facing north and Set 2: facing south). The movement angles are spatial angles of horizontal yellowfin bream movement in two dimensions