Deep machine learning identified fish flesh using multispectral imaging

Abstract

Food fraud is widespread in the aquatic food market, hence fast and non-destructive methods of identification of fish flesh are needed. In this study, multispectral imaging (MSI) was used to screen flesh slices from 20 edible fish species commonly found in the sea around Yantai, China, by combining identification based on the mitochondrial COI gene. We found that nCDA images transformed from MSI data showed significant differences in flesh splices of the 20 fish species. We then employed eight models to compare their prediction performances based on the hold-out method with 70% training and 30% test sets. Convolutional neural network (CNN), quadratic discriminant analysis (QDA), support vector machine (SVM), and linear discriminant analysis ( doi 10.13039/100003090, LDA; ) models perform well on cross-validation and test data. CNN and QDA achieved more than 99% accuracy on the test set. By extracting the CNN features for optimization, a very high degree of separation was obtained for all species. Furthermore, based on the Gini index in RF, 11 bands were selected as key classification features for CNN, and an accuracy of 98% was achieved. Our study developed a successful pipeline for employing machine learning models (especially CNN) on MSI identification of fish flesh, and provided a convenient and non-destructive method to determine the marketing of fish flesh in the future.

Graphical abstract

Highlights

• •

  Multispectral image segmentation is used to obtain sufficient data for establishing fish classification model.

• •

  The classification performance of 8 multi-classification models demonstrates the superiority of the CNN model.

• •

  By extracting the CNN features before the FC layer, a very high degree of separation was obtained for all species.

• •

  The feature selection results showed that 11 to 19 multispectral bands were sufficient to classify the existing fish species.