CentralBark Image Dataset and Tree Species Classification Using Deep Learning

Abstract

The task of tree species classification through deep learning has been challenging for the forestry community, and the lack of standardized datasets has hindered further progress. Our work presents a solution in the form of a large bark image dataset called CentralBark, which enhances the deep learning-based tree species classification. Additionally, we have laid out an efficient and repeatable data collection protocol to assist future works in an organized manner. The dataset contains images of 25 central hardwood and Appalachian region tree species, with over 19,000 images of varying diameters, light, and moisture conditions. We tested 25 species: elm, oak, American basswood, American beech, American elm, American sycamore, bitternut hickory, black cherry, black locust, black oak, black walnut, eastern cottonwood, hackberry, honey locust, northern red oak, Ohio buckeye, Osage-orange, pignut hickory, sassafras, shagbark hickory silver maple, slippery elm, sugar maple, sweetgum, white ash, white oak, and yellow poplar. Our experiment involved testing three different models to assess the feasibility of species classification using unaltered and uncropped images during the species-classification training process. We achieved an overall accuracy of 83.21% using the EfficientNet-b3 model, which was the best of the three models (EfficientNet-b3, ResNet-50, and MobileNet-V3-small), and an average accuracy of 80.23%.

1. Introduction

Proper tree species identification is a crucial tool for sustainable forest management, climate change mitigation, and biodiversity conservation. By accurately identifying trees, practitioners, landowners, and the general public can better understand the current state of a forest ecosystem and implement appropriate management [1]. However, manual identification is tedious, and there are multiple methods for identifying trees through leaves, buds, fruit, and bark observation. Although leaves are commonly used for identification in the spring, summer, and early fall seasons, their reliability decreases greatly once they senesce and fall off the trees for the winter. Using fruit or flowers is limited, as once they fall, it becomes difficult to attribute them to a particular tree. Another way to identify trees is the use of buds in the spring, but buds are hard to reach for tall trees and slight differences within species may be difficult to identify from a distance [2]. By far, the most reliable and well-agreed-upon method to identify trees is by using the bark, which is accessible and does not vary too much throughout the year. Tree bark exhibits various patterns, shapes, and textures that make for a more reliable identification method [3]. However, much like the twig buds, a trained eye is required to identify trees accurately based on the bark.

Foresters, botanists, and other professionals involved in forest management need to have a solid understanding of these methods to accurately identify tree species and make informed decisions regarding forest management. By doing so, they can ensure the long-term health and sustainability of forest ecosystems.

Over the past two decades, various image-processing studies have attempted to improve the accuracy of tree identification based on bark by treating it as a texture-recognition task [4]. Typically, these studies employ a two-step method, which involves extracting features from images and then feeding them into linear classifiers, e.g., Support Vector Machine, or non-linear classifiers, e.g., Multilayer Perceptron. Some of these methods include Gabor filter banks as proposed by Chi et al. [5]; co-occurrence matrices; histogram and auto-correlation methods, as applied by Wan et al. [6]; and the Grey-Level Co-occurrence Matrix with Long Connection Length Emphasis, as employed by Song et al. [7]. Newer studies, namely Kim et al. [8], have used state-of-the-art computer-vision class activation mapping (CAM) to differentiate and classify bark patterns and further understand the nested groups of parameters in CNNs. Other studies have added color features, such as Wan et al. [6], or utilized handcrafted features, such as the shape, color, structure, and orientation of bark with the help of Canny filters, hue histograms, and Gabor filters, as proposed by Ratajczak et al. [9]. Boudra et al. [10] introduced the Statistical Macro Binary Pattern (SMBP), a variant of the Local Binary Pattern that represents the intensity distribution within the macrostructure of large spatial support by one macro pattern code. Fekri-Ershad [11] used Local Ternary Patterns and fed them to the Multilayer Perceptron, while Remes and Haindl [12] introduced rotationally invariant multispectral textural features and reported 90.4% accuracy on BarkNet [3] using the nearest neighbor classifier.

With the help of artificial intelligence (AI), specifically deep learning methods, and Convolutional Neural Networks (CNNs), Lecun et al. [13] demonstrated that accurate identification can be achieved by providing a quantitative approach to exploring unique features in each tree’s bark. Deep learning is a machine learning technique that teaches computers to learn what comes naturally to humans. In this case, teaching computers to become as experienced as professional foresters is the overarching goal. Several studies, e.g., [1,3,14,15,16], have successfully used CNNs for bark identification, reporting equally good or better accuracy compared to texture classification methods, with the added benefits of easy implementation and end-to-end training.

Deep learning methods require a large dataset of labeled bark images to be effective. A few datasets have been created, such as BarkTex [17], TRUNK12 [9], and BarkNet 1.0 [3]. These tree bark datasets contain bark images of hardwood species throughout the US, Canada, and parts of Europe, to supplement the deep learning methods, but each dataset varies in its usefulness due to its availability and the content of similar species.

The objective of our project is to develop AI tools for tree bark identification, allowing people without extensive professional training to identify trees and allowing for the automation of the tree identification process—a task that is greatly in demand in the field of digital forestry. An important step in this effort is to develop a dataset of annotated images that is then used to train the AI model. While a handful of regional datasets, although limited by a small number of species and images, already exist, we undertook the task of creating one the most comprehensive databases of bark images of hardwood trees in the eastern United States.

We aimed to create a dataset of raw images with zero cropping and focused on the US Central Hardwood region and parts of the Central Appalachian regions (Indiana, Illinois, and Ohio). The creation of this dataset includes 25 different hardwood species. These species are a combination of the most commercially valuable species as well as the most common species that foresters may encounter within the region. Along with the species, the diameter at breast height (DBH) recorded in inches, the moisture condition (wet vs. dry) of the bark at the time, GPS location, the time, and the date stamp, along with camera metadata, were captured. While the related works provide very good results, our goal is to improve deep learning identification efforts and adapt them to US hardwood species while also potentially learning more about the forests in which the trees reside.

2. Materials and Methods

2.1. Study Area

The data collection areas primarily comprised state-owned forests and parks located in Indiana, Illinois, and Ohio, with some recreational parks also serving as data collection areas. This allowed us to capture a wide range of light conditions, as public parks typically have more open environments that allow for more light to hit each tree.

We selected forest sites based on the forest type and similarity of the desired species on the dataset list to the species composition in the area (Figure 1). We also spread out the data collection sites to ensure variation in the bark attributes within species, as some species are known to exhibit different characteristics based on their geographic locations and microenvironments. Furthermore, it was necessary to spread out the data collection locations since some species in the dataset only grow in certain portions of the region of interest. For example, sweetgum is a species that grows prolifically in southern Indiana due to the abundance of hills that provide a lowland habitat.

2.2. Image Collection

We chose 25 tree species commonly found in the Central Hardwood and Appalachian regions. Our goal was to collect data on 250 individual trees per species. With 4 images per tree, we collected nearly 1000 images per species. For each species, the first 150 trees were identified, photographed, and labeled by a trained dendrologist. The remaining 100 trees of each species were collected by either the same trained dendrologist or other trained forestry practitioners, including forestry graduate students, students in dendrology class, and state foresters. Labels for these images were then verified by the trained dendrologist. We selected forest sites based on the forest type and similarity of the desired species on the dataset list to the species composition in the area. We captured and recorded images in various seasons, weather, and light conditions. To capture the images, a combination of smartphone cameras, including iPhones 10, 11, and 14 Pro, were used. The ArcGIS Fieldmaps app was used for capturing tree locations and general data management. Figure 2 shows the final image results of the following data collection protocol. Furthermore, the data collection protocol is outlined in the following steps:

• Identify the tree species and record its GPS location in the app.
• Record the diameter at breast height to the nearest inch, ensuring that it is greater than the 8-inch diameter threshold.
• Attach a bright color tag of known size (1″ × 3″) to a tree face prior to taking a picture as a reference for the later measurement of the DBH of the tree using AI.
• Take the first photo on the north face of the tree and repeat the process, moving counterclockwise (north, west, south, east) until the data collector returns to the initial starting position.
• Ensure the entire tree is in view of the frame, and limit as much background as possible.
• Record the bark moisture condition as either wet or dry.
• Data collection is spread over four seasons of the year and different times of the day to include a variety of conditions.

2.3. Data Description

We captured 25 species, 4697 individual trees, and 19,147 images for this study.

Table 1. Species list and dataset statistics for the CentralBark dataset. The #Trees column indicates how many individual trees were used while capturing the images, while the #Images column indicates the total number of training data images captured for each species.

	Species	Common Name	#Trees	#Images
1	Tilia americana	American basswood	67	283
2	Fagus Grandifolia	American beech	280	1.132
3	Ulmus Americana	American elm	156	640
4	Platanus occidentalis	American sycamore	239	981
5	Cary cordiformis	Bitternut hickory	149	609
6	Prunus serotina	Black cherry	284	1.173
7	Robinia pseudoacacia	Black locust	154	631
8	Quercus velutina	Black oak	211	843
9	Juglans nigra	Black walnut	286	1.174
10	Populus deltoides	Eastern cottonwood	279	1161
11	Celtis occidentalis	Hackberry	202	846
12	Gleditsia triacanthos	Honeylocust	36	171
13	Quercus rubra	Northern red oak	278	1158
14	Aesculus glabra	Ohio buckeye	32	155
15	Maclura pomifera	Osage-orange	154	616
16	Carya glabra	Pignut hickory	142	574
17	Sassafras albidum	Sassafras	96	400
18	Carya ovata	Shagbark hickory	278	1156
19	Acer saccharinum	Silver maple	150	642
20	Ulmus rubra	Slippery elm	27	121
21	Acer saccharum	Sugar maple	290	1238
22	Liquidambar styraciflua	Sweetgum	48	201
23	Fraxinus americana	White ash	150	610
24	Quercus alba	White oak	322	1327
25	Liriodendron tulipifera	Yellow poplar	315	1305
	Total		4697	19,147

lists the dataset statistics we used to conduct the experiment in this paper. Due to the limited occurrence and distribution of some species within the region and study time constraints, we did not reach our initial goal of 1000 images for all species.

2.4. Deep Neural Model for the Tree Species from Bark

In recent studies based on Convolutional Neural Networks (CNNs), researchers demonstrated that accurate identification can be achieved by utilizing a quantitative approach to exploring unique features in each tree’s bark. Several studies have successfully used CNNs for bark identification, reporting equally good or better accuracy when compared to texture classification methods, with the added benefit of easy implementation and end-to-end training. Deep learning methods require a large dataset of labeled bark images to be effective. A few datasets have already been created, such as BarkTex, TRUNK12, and BarkNet. They contain bark images of hardwood species throughout the US, Canada, and parts of Europe to supplement the deep learning methods, but each dataset varies in its usefulness due to its availability and content of similar species. The datasets were upscaled in the number of images by cropping them. However, this does not improve variability, as the images are from the same trees.

2.5. CentralBark Dataset and Deep Neural Models

We selected CNNs EfficientNet-b3 [18], ResNet-50 [19], and Mobilenet-V3-small [20] to conduct the baseline study and test the effectiveness of our dataset in bark identification. ResNet-50 and EfficientNet-b3 have 25.55M and 12.23M parameters, respectively. MobileNet-V3-small is a lighter architecture that reduces parameter space to 2.54M.

During the training of all our models, we used the Adam optimizer [21] with a learning rate of 0.01 and cross-entropy loss. We also incorporated standard computer-vision data-augmentation techniques such as image rotation, flipping, and grayscale transformations. We trained our models using a mini-batch size of 32 and an input size of 224 × 224, to enhance their performance and robustness. To demonstrate the effectiveness of our dataset, during the testing phase, we conducted center cropping of the test images and then rescaled them to the desired input size. By doing so, we were able to evaluate the base capability of our dataset.

We employed a stratified five-fold cross-validation method without overlapping, where 20% of the data was reserved for testing in each fold. For every experiment, we used four of the folds (80% of the data) for training, with 70% of those data used as the training set and 10% used as the validation set. The remaining fold (20% of the data) was used as the test set. We split the data at the tree level. Our system was implemented on a desktop computer equipped with an Intel Core i7-8700K CPU @ 3.70 GHz ×12, 32 GB of memory, and an NVIDIA GeForce RTX 1080 Ti GPU. We used PyTorch 1.13 as the framework to implement all the models.

3. Results

We have demonstrated the capability of EfficientNet-b3 to perform accurate species identification from bark images of 25 Central Hardwood and Appalachian region species. Using our dataset, we achieved 83.21% overall accuracy with the EfficientNet-b3 model. EfficientNet-b3 has several architectural advantages compared to ResNet and MobileNet-v2 that likely contribute to its superior performance on many tasks.

In addition to the success of the CNN model, we also created a large and comprehensive dataset named CentralBark, with over 19,000 images. This dataset can be used to further the research on bark classification for forestry and natural resources and AI learning applications, as there exists an urgent need for large, standardized bark image datasets for tree bark classification.

Table 2. CentralBark dataset performance vs. all three models (EfficientNet-b3, ResNet-50, and MobileNet-V3-small).

Species	Common Name	EfficientNet-b3 (Accuracy %)	ResNet-50 (Accuracy %)	MobileNet-V3-Small (Accuracy %)
Tilia americana	American basswood	65.72	53.00	44.88
Fagus grandifolia	American beech	96.02	93.11	95.67
Ulmus americana	American elm	69.53	70.47	68.59
Platanus occidentalis	American sycamore	90.52	92.35	86.75
Cary cordiformis	Bitternut hickory	63.38	68.97	55.17
Prunus americana	Black cherry	75.70	74.17	71.44
Robinia pseudoacacia	Black locust	89.22	88.91	86.21
Quercus velutina	Black oak	87.66	84.82	75.56
Juglans nigra	Black walnut	81.18	75.04	63.2
Populus deltoides	Eastern cottonwood	87.17	82.26	75.88
Celtis occidentalis	Hackberry	75.53	70.57	69.74
Gleditsia triacanthos	Honeylocust	61.99	42.11	53.8
Quercus rubra	Northern red oak	91.28	86.53	85.75
Aesculus glabra	Ohio buckeye	61.29	60.00	55.48
Maclura pomifera	Osage-orange	95.94	93.18	94.32
Carya glabra	Pignut hickory	84.32	87.46	75.61
Sassafras albidum	Sassafras	72.75	71.00	71.25
Carya ovata	Shagbark hickory	88.06	85.12	83.56
Acer saccharinum	Silver maple	80.84	82.09	69.94
Ulmus rubra	Slippery elm	51.24	46.28	33.88
Acer saccharum	Sugar maple	85.14	83.2	78.84
Liquidamber styraciflua	Sweetgum	58.21	51.74	48.26
Fraxinus americana	White ash	74.43	73.93	67.7
Quercus alba	White oak	87.49	90.13	69.4
Liriodendron tulipifera	Yellow poplar	87.28	87.66	89.73
Overall		83.21	81.37	76.11

shows each model’s performance by species from our dataset. We used micro-average accuracy, which treats each sample as the same weight when averaged. The EfficientNet-b3 architecture achieved the highest accuracy with a score of 83.21%. The second-best performing model was ResNet-50, with an accuracy score of 81.37%, followed by MobileNet-V3-small, with a score of 76.11%. It is worth noting that the performance order of these models is consistent with their performance on the ImageNet dataset [22].

The confusion matrix in Figure 3 reveals that some tree species are incorrectly identified by the EfficientNet-b3 model, resulting in false positives. This inaccuracy can be attributed to three main factors: the diameter at breast height (DBH) or the age of the tree, the color and physical appearance of the bark, and subtle differences within species. For instance, when trees are smaller in diameter, species like sugar maple (Acer saccharum) and hackberry (Celtis occidentalis) have very similar bark. Hackberry can be identified by its smooth bark when young, but it soon develops warty growths as the tree matures [23]. Consequently, when young, sugar maples appear smooth, and then, as they mature, the ridges and furrows become more pronounced. Because of the similarities between the two species, we ran into inaccuracies in our confusion matrix. However, we found that with the increasing diameter (age), the models were able to differentiate between them with greater results. Similarly, black cherry (Prunus serotina) and white oak (Quercus alba) can be confused because of their shared blocky and flaky bark pattern, despite their different morphology. Also, the bark of cherry tends to bleach when exposed to sunlight, changing its shade from a darker gray to a lighter one, which can further complicate identification.

Moreover, the differences in bark appearance within species are another significant source of confusion, as observed in species such as American elm (Ulmus americana) vs. slippery elm (Ulmus rubra) or pignut hickory (Carya glabra) vs. bitternut hickory (Carya cordiformis). There are often minute differences in the bark pattern between species of the same genus, such as the interlacing pattern in the tight bark of pignut and bitternut hickory. Professional foresters identify these species based on touch. Wojtech and Wessels [24] point out that the American elm has a mottled, grayish brown, and spongy bark that can be compressed with the thumb, while the slippery elm is firmer and not easily compressed. However, this kind of tactile input cannot be observed by AI without human intervention. To improve accuracy, AI needs more data on these subtle differences within species.

4. Discussion

We found that EfficientNet-b3 has several architectural advantages compared to ResNet and MobileNet-v2 that likely contribute to its superior performance on many tasks. First, EfficientNet uses a compound scaling method that jointly scales the network width, depth, and resolution, allowing for a more optimal allocation of resources. This balanced scaling approach enables EfficientNet to achieve better accuracy and efficiency. Additionally, EfficientNet employs MBConv (Mobile Inverted Bottleneck Conv) blocks, which are a more efficient version of the bottleneck blocks used in ResNet. MBConv blocks use depth-wise separable convolutions and squeeze-and-excitation (SE) modules, which help improve the network’s representational power while keeping the computational cost low. EfficientNet-b3 also has a larger receptive field compared to ResNet and MobileNet-v2 due to its deeper architecture and the use of larger kernel sizes in the early layers. This allows the network to capture more contextual information, which can be beneficial for certain tasks. Finally, EfficientNet is designed to maximize accuracy for a given computational budget, allowing it to achieve higher accuracy with fewer parameters compared to ResNet and MobileNet, making them more parameter-efficient.

One of the limitations we face with our dataset is that some of the images we collected may not have bark texture throughout the entire area, as shown in Figure 4 in a sample image of black oak (Quercus velutina). Additionally, there are cases where a single image may contain multiple trees, making it difficult to isolate the bark texture of the target tree. These limitations can potentially impact the performance of our model if we simply perform center cropping and pass the processed image to the CNN classifier. The classifier may become confused in such extreme cases, leading to inaccurate classification results.

Furthermore, Figure 5 shows the relationship between the sample data size and the classification accuracy. Species with larger amounts of data generally perform better with each of the three models we used in our experiments. In our dataset we were able to capture ample amounts of images for species that are generalists, ones that grow in many different types of habitats, making searching for them relatively easy. Species that have lower classification accuracy such as sweetgum do not have nearly as much sample data as they require a very specific type of habitat or location making them harder to find in our region of interest.

Another challenge we faced in our dataset was that there was a weak correlation between model accuracy and DBH in Figure 6. In theory, there should be a strong positive correlation between accuracy and larger-diameter trees because as the trees grow larger, they begin to form unique characteristics that make them easier to identify. With that stated, we believe that if we had a threshold DBH of 14 inches rather than 8 inches, then we would see an increase in model performance across the board.

5. Conclusions

We created a large and comprehensive dataset while also laying a framework for an easy and repeatable protocol that allows datasets to be created and added to. Our dataset is unique as it contains many attributes other than just the images of the tree bark, such as diameter, the moisture condition of tree bark, GPS location, images from different cardinal directions, and different light conditions. This dataset can be used for further research on bark classification for forestry and AI applications.

We also demonstrated the capability of our dataset against three different existing CNN models against our own dataset, in performing accurate species identification from bark images of 25 Central Hardwood and Appalachian region species. Using our dataset, we achieved the following results: EfficientNet-b3 achieved an accuracy of 83.21%, ResNet50 achieved 81.37%, and MobileNet-V3-small achieved 76.11%. It is worth noting that we did not crop or alter the images in any way, and the results were achieved using the raw images.

To overcome the limitations of this study, more work is needed to improve our current model, including the gathering of additional images. It is essential to ensure that the images in the dataset are representative of the entire population of tree species, to avoid any bias in the training of the model. We are currently collecting images in the north and south of the eastern US to include more species and regional variations. Additionally, we need to address the issue of defects, moisture conditions, and DBH, which can adversely affect classification accuracy.

Future research can also focus on extending the scope of the dataset beyond the Central Hardwood and Central Appalachian regions to include other regions around the world. Through Purdue University’s Institute for Digital Forestry, we are expanding the database to contain twice as many hardwood species and some softwoods. Species expansion and data collection have begun in Maine and Georgia to represent the northeast and southeast regions of the United States. This will enable us to identify and classify tree species across various geographic locations, thereby making a significant contribution to the conservation and management of forest resources and helping move AI further in the direction of simplifying forestry-related tasks. Overall, our study has shown the potential of using deep learning methods in bark identification and has laid the groundwork for further research in this field.