The Automated Plant Leaf Image Classification System using Deep Learning

Abstract

Knowledge of botany is necessary in order to classify plants accurately. Sometimes, even experts can misclassify a plant. To reduce errors that are made by a human, in this thesis, we aimed to invent an automated plant leaf classification system that could classify plants from leaves using deep learning techniques. The proposed method could also classify diverse plants and identify plant diseases from leaves. In this thesis, we presented three approaches to addressing the challenges of plant leaf classification.

In the first approach, we invented a method to classify various healthy plant leaf images taken in the laboratory, called the multiple-grid method. This method could extract robust features from the local area using different feature extraction methods: histogram of oriented gradients (HOG), local binary patterns (LBP), and color histogram. Hence, the principal component analysis (PCA) technique was proposed to reduce the size of the feature and finally fed to the machine learning techniques: support vector machine (SVM) and multi-layer perceptrons (MLP). The proposed method achieved high accuracy in plant leaf image recognition.

In the second approach, the leaf images (healthy and diseased) taken in the natural environments were classified using the ensemble convolutional neural networks (CNNs) method. For the CNN models, we created various CNN models based on five architectures: MobileNetV1, MobileNetV2, Xception, DenseNet121, and NASNetMobile. The CNN models were fine-tuned with different parameters, including optimizers, batch sizes, and data augmentations. For the ensemble learning method, we classified the output probabilities of 3 CNN models (called 3-EnsCNNs) and 5 CNN models (called 5-EnsCNNs) with three different ensemble learning methods: unweighted majority vote, unweighted average, and weighted average. As a result, the ensemble CNN with the weighted average method outperformed other ensemble learning methods on three different plant leaf datasets.

In the third approach, we automatically selected the best-CNN models using the ant colony optimization (ACO) algorithm used in the ensemble CNN method. According to the ACO algorithm, we first proposed the new fitness function computed by the loss and error while training the CNN models. Second, the learning rate schedule was included in the ACO algorithm to decrease the fitness value between each CNN model while training the ACO algorithm. We compared the performance of two learning rate schedules: the time-based and cyclical learning rate, and found that two learning rate schedules contributed to improving the ACO algorithm. Consequently, the proposed ACO algorithm outperformed the existing methods on mulberry leaf and turkey plant datasets.

We also found that many deep learning techniques could be proposed for automated plant leaf image classification. However, when we focus on the ensemble CNNs method, we should have an automated method to select the best-CNN models. Further, the proposed ACO algorithm is one of the best solutions for creating an automated plant leaf classification system. Adding the new robust CNN models to the system enables the proposed method to train the ACO algorithm and automatically choose the best-CNN models.

-