Automated echolocation classifiers vary in accuracy for northeastern U.S. bat species

Abstract

Acoustic surveys of bat echolocation calls are an important management tool for determining presence and probable absence of threatened and endangered bat species. In the northeastern United States, software programs such as Bat Call Identification (BCID), Kaleidoscope Pro (KPro), and Sonobat can automatically classify ultrasonic detector sound files, yet the programs’ accuracy in correctly classifying calls to species has not been independently assessed. We used 1,500 full-spectrum reference calls with known identities for nine northeastern United States bat species to test the accuracy of these programs using calculations of Positive Predictive Value (PPV), Negative Predictive Value (NPV), Sensitivity (SN), Specificity (SP), Overall Accuracy, and No Information Rate. We found that BCID performed less accurately than other programs, likely because it only operates on zero-crossing data and may be less accurate for recordings converted from full-spectrum to zero-crossing. NPV and SP values were high across all species categories for SonoBat and KPro, indicating these programs’ success at avoiding false positives. However, PPV and SN values were relatively low, particularly for individual Myotis species, indicating these programs are prone to false negatives. SonoBat and KPro performed better when distinguishing Myotis species from non- Myotis species. We expect less accuracy from these programs for acoustic recordings collected under normal working conditions, and caution that a bat acoustic expert should verify automatically classified files when making species-specific regulatory or conservation decisions.