Integrating Sigmoid Calibration Function into Entropy Thresholding Segmentation for Enhanced Recognition of Potholes Imaged Using a UAV Multispectral Sensor

Abstract

This study was aimed at enhancing pothole detection by combining sigmoid calibration function and entropy thresholding segmentation on UAV multispectral imagery. UAV imagery was acquired via the flying of the DJI Matrice 600 (M600) UAV system, with the MicaSense RedEdge imaging sensor mounted on its fixed wing. An endmember spectral pixel denoting pothole feature was selected and used as the base from which spectral radiance patterns of a pothole were analyzed. A field survey was carried out to measure pothole diameters, which were used as the base on which the pothole area was determined. Entropy thresholding segmentation was employed to classify potholes. The sigmoid calibration function was used to reconfigure spectral radiance properties of the UAV spectral bands to pothole features. The descriptive statistics was computed to determine radiance threshold values to be used in demarcating potholes from the reconfigured or calibrated spectral bands. The performance of the sigmoid calibration function was evaluated by analyzing the area under curve (AUC) results generated using the Relative Operating Characteristic (ROC) technique. Spectral radiance pattern analysis of the pothole surface revealed high radiance values in the red channel and low radiance values in the near-infrared (NIR) channels of the spectrum. The sigmoid calibration function radiometrically reconfigured UAV spectral bands based on a total of 500 sampled pixels of pothole surface obtained from all the spectral channels. Upon successful calibration of UAV radiometric properties to pothole surface, the reconfigured mean radiance values for pothole surface were noted to be 0.868, 0.886, 0.944, 0.211 and 0.863 for blue, green, red, NIR and red edge, respectively. The area under curve (AUC) results revealed the r² values of 0.53, 0.35, 0.71, 0.19 and 0.35 for blue, green, red, NIR and red edge spectral channels, respectively. Overestimation of pothole 1 by both original and calibrated spectral channels was noted and can be attributed to the presence of soils adjacent to the pothole. However, calibrated red channel estimated pothole 2 and pothole 3 accurately, with a slight area deviation from the measured potholes. The results of this study emphasize the significance of reconfiguring radiometric properties of the UAV imagery for improved recognition of potholes.

1. Introduction

Pothole detection is critical for maintaining roads and vehicle safety [1]. The progression in remote sensing technology has offered exciting avenue for studying detection and characterization of potholes [2]. The improvements in the accuracy and spatial resolution of remotely sensed images now enable the detection and characterization of small objects such as potholes. Unmanned Aerial Vehicle (UAV) systems have recently become the most commonly reliable remote sensing platforms for detecting potholes due to their ability to detect features at an ultra-spatial resolution of a few centimeters [3]. As such, segmentation of these UAV images has become the basis for pothole characterization and improving segmentation accuracy is of great interest to remote sensing scientists. When imaged using UAV systems, image segmentation becomes the basis for characterizing potholes. Although there are several remote sensing image segmentation methods capable of detecting and characterizing potholes such as spatial-based image segmentation [4], hybrid-technology-based image segmentation [5] and semantic-based image segmentation [6], spectral-based image segmentation has been, and is still, of great interest to the remote sensing community [7]. This method segments an image by analyzing the pixel values [8], through either feature space clustering [9] or thresholding [10]. Of these spectral-based approaches, thresholding is recognized as the most viable and practical way for extracting information from UAV imagery [11], and potholes are no exception.

The thresholding approach performs under the supposition that the distribution pixel intensity in the imagery has two different peaks, and the threshold value can be determined by any intensity in the valley that separates them [12]. The threshold is defined as a probability point which confines the criteria for allowing certain processes on the image [13]. As such, the statistical perception known as “entropy” is often used for the image thresholding [14]. This approach, conventionally popular for image binarization, segments image into categories of pixels with values that are either lower, greater than or equal to the threshold [15]. The inclusion of the thresholding approach in machine learning and automated detection of pothole in various studies also highlights the significance of this approach [16]. In a study by Kim and Ryu [17], the thresholding technique achieved 88% accuracy in extracting pothole regions. The thresholding approach is efficient when there is a clear distinction between a target feature and the background, such as a pothole and asphalt, exhibiting a sharp contrast in their spectral reflectance. However, the thresholding approach has been commonly deployed on grayscale images [18]. As these images are the product of converted multispectral imagery, they contain less details about potholes [14]. Because multispectral imagery contains additional information such as hue and saturation [19], this emphasizes a need to shift focus towards multispectral image thresholding [20].

Image thresholding is not without disadvantages. It relies on information at pixel level, disregarding pixel association with neighboring pixels [21]. Exacerbating the situation are radiometric errors associated with ultra-spatial resolution UAV images, which enable them to detect even smaller features [22], resulting in undesirable noise. These radiometric errors emanate because of the local image contrast, tonal range, random noise and radiometric resolution [23]. Without minimizing or removing these errors, poor segmentation of potholes may occur, subsequently complicating their characterization [22]. Therefore, to minimize uncertainties surrounding thresholding performance, calibration of UAV multispectral imagery to potholes is required prior to the deployment of thresholding classifier. Uncalibrated models tend to either underestimate or overestimate the probability of outcomes [24], subsequently making insignificant decisions regarding pothole size. Performed prior to image classification, calibration adjusts the image radiometric properties to closely match the pothole features [25]. The outcome of the UAV multispectral data calibration must be the degree to which a pixel belongs to a pothole class, instead of a pixel directly becoming a member of a pothole class [26]. Calibration of UAV multispectral data to potholes can be achieved by deploying either isotonic regression (also known as ROC convex hull approach) [27] or Platt-based sigmoid [28]. Although the isotonic regression approach is capable of calibrating probability of classifier scores [29], it is subjected to model overfitting when datasets are smaller, such as when dealing with few potholes.

The Platt-based sigmoid approach has demonstrated the ability to overcome the shortfalls associated with the isotonic regression calibration model [30]. This approach was inspired by biological neural networks and has been deployed extensively in training machine learning algorithms, especially as an activation function in artificial neural networks [31]. Of these neural networks, the multilayer perceptron (MLP), trained with backpropagation learning, has become the most popular neural network on which sigmoid calibration function is deployed [32]. This calibration model can fit a logistic regression classifier to UAV multispectral data using the value 0 or 1 to produce a binary image. These binary values can serve as the pothole output points at which the sigmoid function saturates [33]. This approach is important if thresholds are required for demarcation of pothole surface from multispectral imagery [26], especially when the pothole class requires quantification. The sigmoid calibration function can calibrate potholes by relying on two parameters, namely (a) a location parameter (m), which specifies the sigmoidal midpoint, where the calibrated score is 0.5, and (b) a shape parameter (γ), which specifies the slope of the sigmoid at the midpoint [34]. In this case, the probability outcomes of the calibration model are thresholded to reduce mischaracterization of features [35]. The responsibility of thresholding is to convert a projected probability or score into a certain class. Several studies on pothole characterization based on UAV multispectral imagery focused on improving methods and techniques for detecting potholes [18,36]. However, the deployment of these approaches without them being subjected to a calibration process poses a question regarding their ability to compensate for information loss due to noise associated with ultra-spatial resolution of UAV sensors. To our best knowledge, there is currently no study that has characterized potholes by integrating entropy thresholding segmentation classifier and sigmoid calibration function. In the quest for optimized characterization of potholes from UAV multispectral imagery, we proposed an approach which combine sigmoid calibration function and entropy thresholding segmentation classifier.

2. Materials and Methods

2.1. Experimental Site

This investigation was carried out in a section of the R523 road which runs from Sibasa Town to the N1 national road, which runs from Cape Town to Beit Bridge, which is the border with Zimbabwe. The road section on which the experiment was carried out is situated at Nzhelele Village in the Limpopo Province. The experimental site is located between 22°43′30″ S; 30°0′45″ E and 22°52′30″ S; 30°06′30″ E grid reference, in the northern part of South Africa, within the Vhembe District Municipality. The site was selected based on the availability of the UAV imagery containing tarred road which was subjected to potholes. Figure 1 presents the location of the experimental site in relation to South Africa.

2.2. Methodology

The experiment for pothole recognition in this study was achieved by deploying the sequence of methods and techniques as shown in Figure 2.

2.2.1. Unmanned Aerial Vehicle (UAV) Data

The UAV data used in this experiment was the imagery captured using the MicaSense RedEdge imaging sensor, embedded on a fixed wing of the DJI Matrice 600 (M600) UAV system supplied by the Da-Jiang Innovations, Shenzhen, China. This multispectral imaging sensor scanned the road pavement at 8.55 cm spatial resolution at 47.2° horizontal field of view (HFOV). The MicaSense RedEdge sensor captured the road pavement using five narrow spectral channels, i.e., blue (475 nm), green (560 nm), red (668 nm), near-infrared (840 nm) and red-edge (717 nm) wavelengths of the electromagnetic spectrum.

Table 1. Characteristics of the UAV system employed to acquire image for this study.

System	Characteristics	Specification
Platform	Mass	6 kg
	Flying height	120 m
	Areal extent	8711 m squared
	Scanning duration	13 min 28 s
	Speed	10 ms−1
	Visible satellites	13
	Overlap	75%
	Side lap	75%
	Dimension	1.2 m
Red-Edge Sensor	Spectral channels	B, G, R, NIR, R_Edge
	Focal length	5.5 mm
	Field of view	7.2 degrees
	Mass	150 g
	Image resolution	1280 × 960 mm
	Spatial resolution	8.5 cm

provides a detailed description of the UAV system employed to acquire the imagery used in this study. The advantage of employing this narrow-band camera was based on its ability to take precise measurements of certain wavelengths, resulting in more detailed and accurate land surface information. A single UAV image with five (5) spectral channels was subsequently acquired to capture asphalt surface containing potholes. Detailed information regarding the imagery used is provided in Table 1.

2.2.2. Field-Based Data

Field-based data, in the form of pothole diameters, were measured using the Stramm 5 m × 16 mm M2554 P2 measuring tape. Moreover, the locations of the potholes were also verified using the Garmin eTrex 22x Handheld GPS^® supplied by the Garmin Southern Africa (PTY) Ltd., Johannesburg, South Africa. The GPS provided a consistent and accurate method of determining the location [37]. The field-measured pothole diameter data was used as the base from which pothole size was computed.

2.3. UAV Sensor Calibration

The camera calibration was performed prior to UAV take-off, using the black–white chessboard pattern MicaSense Calibrated Reflectance Panel (CRP). This process was carried out to readjust the camera with irradiance properties of the land surface during the flight period [38]. The square located at each corner of the CRP served as a positional reference point. The CRP was placed at approximately 1 m distance to cover the sensor’s field of view. The UAV sensor was rotated 360° degrees around the calibration panel along its center axis, and then held towards the incident light direction to avoid the influence of light reflectance from surrounding features. The UAV status indicator is expected to emit a green light after calibration; if it blinks red, it means there is strong interference and recalibration is required. Upon the successful detection of the CRP by the sensor, light-emitting diode was flashed to indicate a successful recording of incident radiation.

2.4. Image Pre-Processing

UAV images are usually subjected to distortion caused by the camera movement during landscape scanning. This subsequently reduces the quality of the image, underscoring the need for image pre-processing. In this study, pre-processing was carried out to correct geometric and radiometric distortions incurred in the image scenes, as follows.

2.4.1. Geometric Correction

Specifically, the image orthorectification process was carried out to spatially fuse the image scenes captured on each spectral channel, to form a single image based on the common rows and columns. Subsequently, geometric correction was carried out in order to spatially register the orthomosaic imagery. The imagery was spatially registered to WGS-1984-UTM-Zone-36S spheroid. The atmospheric correction process was not carried out in this study because the UAV system operated below the atmosphere; the recorded reflected radiations were not subjected to interaction with the atmosphere. The imaged scene obtained from the UAV system extended beyond the asphalt road to cover crop fields and adjacent vegetation. As such, the image sub-setting process was carried out to extract only the surface covered by asphalt road, using the “Extract By Mask” tool embedded in ArcMap GIS environment.

2.4.2. Conversion of DN to Radiance

Because the MicaSense RedEdge imaging sensor records images in 16-bit format, radiometric correction models are required to convert raw pixel values into spectral reflectance values. This is especially true when the desired results are reflectance maps [39]. For this study, the unitless digital number (DN) values of the image were converted into at-sensor radiance, using Equation (1) adopted from [40]:

$$L\left(\lambda \right)=V\left(x,y\right)\times \left(\frac{a_1}{g}\right)\times \frac{DN\left(\lambda \right)-D{N}_{BL}}{t_e+a_{2}y-a_3{t}_{e}y}$$

(1)

where L_(λ) denotes the spectral radiance (measured in W/m²/sr/nm); DN is the raw digital number of a given pixel and DN_BL is the black level value (usually obtained from metadata file; a₁, a₂ and a₃ are radiometric calibration coefficients; t_e is the image exposure time; and g denotes the sensor gain setting [39].

2.4.3. UAV Image Filtering

The image filtering process was performed to enhance the visibility of potholes from the imagery, under a supposition that there is a steep gradient where land feature properties start to change in the image. Based on this supposition, the edges were identified by deriving the intensity values and finding locations where derivative is maximum. The gradient in the imagery was computed using Equations (2) and (3) adopted from [41]:

$$\frac{\partial f\left(x,y\right)}{\partial x}=∆x=\frac{f\left(x+dx,y\right)-f\left(x,y\right)}{dx}$$

(2)

$$\frac{\partial f\left(x,y\right)}{\partial x}=∆y=\frac{f\left(x,y+dy\right)-f\left(x,y\right)}{dy}$$

(3)

where d_x and d_y compute distance along the x and y directions.

The gradient discontinuity in the image was then determined by deploying Equation (4):

$$M=\sqrt{∆x^2+∆y^2}$$

(4)

2.5. Characterization of Potholes by Thresholding Classifier

In this study, two experiments were conducted: the first experiment was conducted based on uncalibrated (original) UAV spectral bands, and the second experiment was conducted using the calibrated UAV spectral channels, as follows.

2.5.1. Characterization of Potholes by Thresholding Classifier

A total of 25 points denoting pothole surface were randomly created from RGB image to produce a vector shapefile. The vector shapefile was overlain on each UAV spectral channel, and the values of pixels on which the points were digitized were extracted and appended to the point shapefile, using the “Extract Mult Values to Points” module in ArcGIS version 10.8 software. The extracted pixel values were used to compute descriptive statistics for pothole surface reflectance across the channels of UAV sensor’s spectrum. Upon the successful computation of the statistics, the pothole surface was then demarcated by using minimum and maximum spectral radiance values as thresholds for each UAV spectral channel, such that

$$L(\lambda )=\left\{\begin{array}{c}1ifL\left(i,j\right)\ge T_{min}and{\le T}_{max}\\ 0,otherwise\end{array}\right.$$

(5)

where L denotes spectral radiance value of UAV image band, i denotes pixel at i-th location, j denotes pixel at j-th location, T_min is the minimum threshold and T_max is the maximum threshold.

The values that fall within the upper bound and the lower bound were the representative of pothole surface. The binary image was reclassified into two classes, where class 0 indicated non-pothole surface (i.e., pixel values falling outside the thresholds) and class 1 indicated pothole surface (i.e., pixel values falling outside the thresholds).

2.5.2. Sigmoid Calibration Function

The sigmoid activation function of the backpropagation multilayer perceptron (MLP) neural network was used to account for the effect of variability over the sampled spectral radiance denoting pothole surface. The Platt-based sigmoid calibration function was used to calibrate spectral radiance properties of UAV imagery to potholes. The sigmoid function managed this by employing a nonlinear transformation of the linear regression model to linearly separate spectral radiance values of pothole surface and non-pothole surface based on Equation (6) adopted from [28]:

$$\mathsf{\sigma }\left(x\right)\frac{L}{1+{exp}^{-k(x-x_0)}}$$

(6)

where:

L denotes supremum of the values of the function; σ (x) is the sigmoid function; exp ≈ 2.71 is the base of the natural logarithm; k denotes the steepness of the sigmoid curve; x and x₀ are the parameters fitted into the sigmoid function through maximum likelihood prediction function according to [28] (x₀ denotes the value of function’s midpoint).

In this study, the sigmoid function for spectral radiance of each dependent UAV spectral channel was set to saturate at 1.0, so that activation function became gradually more linear in nature.

2.5.3. Improving Sigmoid Performance by Minimizing Loss Function

The performance of the sigmoid loss function was improved by deploying the gradient descent method. In this case, learning a loss function was defined, and then minimized by optimizing sigmoid function. This method was achieved by evaluating the parameters x and x₀ fitted into sigmoid function by deploying Equation (7) adopted from [42]:

$$\underset{x,x_0}{\mathrm{argmin}}\{-{\sum }_i{y}_i\log \left(p_i\right)+(1-y_i)\log \left(1-p_i\right)\}$$

(7)

where

$$p_i=\frac{1}{1+{exp}^{\left(x{f}_i-x_0\right)}}$$

(8)

2.5.4. Performance Evaluation of the Sigmoid Calibration Function

The validation of the sigmoid model performance was carried out by employing the ROC technique embedded on TerrSet 18.3 geospatial monitoring software package. This was achieved by using the sigmoid calibrated bands as against the coded image used as training mask. In this case, each sigmoid calibrated spectral band was treated as the test data, whereas the training mask image was treated as the ground truth. All the pothole features in the image were considered; otherwise, random sampling would have been employed. Subsequently, random sampling was used to select points on which spectral pixel values for pothole surface. The thresholds were selected, and the number of thresholds were set to 100. This allowed the specification of the interval number and the split of categories on the basis of value range that are automatically defined. Ultimately, the output for each input image was plotted with the percentage of true positives on the vertical axis and the percentage of false positives on the horizontal axis, and the area under the curve (AUC) was calculated using Equation (9) adopted from [43]:

$$AUC={\sum }_{i-1}^n[x_{i+1}-x_i]\ast \left[y_i+\frac{y_{i+1}-y_i}{2}\right]$$

(9)

where n represents the number of thresholds, x_i denotes the rate of the false positives for threshold and y_i denotes the rate of true positives for threshold.

Upon the successful performance evaluation of the sigmoid activation model, potholes were classified using the thresholding technique explained in Section 2.5 in this study.

2.5.5. Field Determination of Pothole Size

The area of pothole was calculated as a function of the field-measured average diameter per pothole, using Equation (10).

$$A=\pi {\left(\frac{d}{2}\right)}^2$$

(10)

where:

A is the area of a pothole,

pi (π) equals 3.14, and

d is the field-measured diameter

Subsequently, the pothole area derived from the spectral radiance data generated from the sigmoid model. This was then compared with the field-measured pothole area, with a view to determine the accuracy of the model.

3. Experimental Results

Through visual interpretation of the RGB image, a total of three (3) potholes were identified in the study area. Figure 3 provides the RGB image of the experimental site, with three (3) road potholes identified and numbered.

3.1. Spectral Radiance Pattern Analysis of Pothole Surface from UAV Original Bands

Upon the successful masking of asphalt, radiance values for each spectral channel were presented in

Table 2. Descriptive statistics of radiance samples obtained from original spectral channels.

Original Spectral Band	N	Min	Max	μ	σ	p-Value
Blue	25	0.855	0.969	0.896	0.032	0.008
Green	25	0.875	0.992	0.936	0.029	0.001
Red	25	0.937	0.984	0.962	0.012	0.002
NIR	25	0.208	0.337	0.273	0.028	0.021
Red edge	25	0.675	0.953	0.872	0.078	0.017

. Descriptive statistics of the spectral radiance data sampled from pothole surface were generated in this study, guided by the null hypothesis that “there were no significant differences in spectral radiance properties of pothole surface across original spectral channels of UAV imagery”. Table 2 presents the descriptive statistical analysis of radiance properties of the potholes.

Minimum spectral radiance values at N = 25 were 0.855 (blue), 0.875 (green), 0.937 (NIR) and 0.675 (red edge) spectral channels for UAV imagery. These spectral channels were also found to have maximum radiance values of 0.969, 0.992, 0.984, 0.337 and 0.953, in that order. For these respective spectral channels, the corresponding mean radiance values were found to be 0.896, 0.936, 0.962, 0.273 and 0.872. Their spectral radiance standard deviation values were also found to be 0.032, 0.029, 0.012, 0.028 and 0.078, respectively, with the computed p-values of 0.008, 0.000, 0.002, 0.021 and 0.017, respectively. Then, the spectral radiance pattern of the potholes was computed at different spectral channels. Figure 4 presents the behavior of the potholes at different original spectral bands.

Based on the descriptive statistics presented in Table 2, the potholes were subsequently demarcated based on the analysis of their upper and lower radiance threshold values. Figure 5 provides demarcated potholes based on the original spectral channels of UAV imagery.

3.2. Spectral Radiance Pattern Analysis of Pothole Surface from UAV Calibrated Bands

In this study, the presence of potholes was predicted by training the MLP with sigmoid calibration function on various spectral channels of the UAV imagery.

Table 3. Learning parameters for sigmoid calibration function-trained MLP.

Sigmoid Calibration Function for Predicting Potholes from Backpropagation MLP
		Blue Channel	Green Channel	Red Channel	NIR Channel	Red Edge Channel
Input specifications	Independent image	B, G, R, N, R_edge	B, G, Red, NIR, R_edge	B, G, R, N, R_edge	B, G, Red, NIR, R_edge	B, G, R, N, R_edge
	Dependent variable images	Blue channel	Green channel	Red channel	NIR channel	Red edge
	Training mask	Boolean image	Boolean image	Boolean image	Boolean image	Boolean image
	Max. training pixels used	500	500	500	500	500
	Max. testing pixels used	500	500	500	500	500
Network topology	No. of input layer nodes	5	5	5	5	5
	No. of output layer nodes	1	1	1	1	1
	No. of hidden layers	1	1	1	1	1
	Layer 1 nodes	5	5	5	5	5
Training parameters	Training type	Automatic	Automatic	Automatic	Automatic	Automatic
	Learning rate type	Dynamic	Dynamic	Dynamic	Dynamic	Dynamic
	Learning rate start	0.01	0.01	0.01	0.01	0.01
	Learning rate end	0.001	0.001	0.001	0.001	0.001
	Momentum factor	0.5	0.5	0.5	0.5	0.5
	Sigmoid constant	1	1	1	1	1
Stopping criteria	Iteration	10,000	10,000	10,000	10,000	10,000
	Actual training pixels	490	497	474	483	496
	Actual testing pixels	491	497	475	483	497
	Learning rate	0.0010	0.0010	0.0010	0.0010	0.0010
	RMS	0.01	0.01	0.01	0.01	0.01
	Training RMS	0.0322	0.0334	0.0345	0.0185	0.0349
	Testing RMS	0.0344	0.0365	0.0325	0.0196	0.0350
	R.sqr	0.942	0.987	0.772	0.979	0.986

provides detailed information regarding the parameters used to train MLP with sigmoid calibration function.

The sigmoid calibration model neural network was applied across several spectral channels of UAV imagery, with a view to determine the spectral channel on which the sigmoid calibration function provides the best pothole estimate.

Table 4. Descriptive statistics of radiance samples obtained from calibrated spectral channels.

Calibrated Spectral Bands	N	Min	Max	μ	σ	p-Value
Blue	25	0.848	0.911	0.868	0.032	0.004
Green	25	0.783	0.937	0.886	0.021	0.001
Red	25	0.925	0.961	0.944	0.011	0.002
NIR	25	0.18	0.32	0.211	0.024	0.018
Red edge	25	0.311	0.906	0.863	0.066	0.014

shows the descriptive statistics generated from the radiance samples obtained from calibrated spectral channels.

Based on the descriptive statistics presented in Table 3, the potholes were also demarcated based on the analysis of their upper and lower radiance threshold values. Figure 6 shows potholes as demarcated on each calibrated spectral channel.

3.3. Spectral Comparative Analysis of Pothole Surface from Original and Calibrated Bands

Upon the successful sampling of spectral radiance data from both original and calibrated spectral bands, the sampled data were plotted on linear graph to determine the difference between pothole radiance properties obtained from of original bands and those which were sampled from calibrated spectral bands. Figure 7 shows the spectral radiance properties of potholes derived from original and calibrated bands.

Generally, calibrated spectral bands exhibited low spectral radiance values when compared with uncalibrated spectral bands.

3.4. Performance Evaluation of the Sigmoid Calibration Function

The performance of the sigmoid calibration function in reconfiguring UAV radiometric properties to potholes was evaluated by analyzing AUC results generated by the ROC model. Figure 8 presents the AUC-ROC curve results explaining the performance of the sigmoid calibration model.

Based on the results, the AUC values for blue, green, red, NIR and red edge spectral channels of UAV imagery were found to be 0.53, 0.35, 0.71, 0.19 and 0.35, respectively. By implication, the sigmoid calibration function showed better performance in radiometrically reconfiguring the red channel of UAV imagery (Figure 7c). Moreover, the performance of the sigmoid calibration function on blue, green, NIR and red edge spectral channels was weak (Figure 7a,b,d,e).

3.5. Field Verification of the Pothole Area

Upon the successful characterization of the pothole surface using original and calibrated spectral channels, the area covered by each pothole was determined and compared with those measured on field. Figure 9 provides a comparison of the pothole area predicted via original and calibrated spectral channels.

From Figure 8, the area of the measured potholes 1 (Figure 8a), pothole 2 (Figure 8b) and pothole 3 (Figure 8c) were to be 3.11 m², 1.68 m² and 0.83 m², respectively. From Figure 8a, both the original and calibrated spectral channels overestimated the size of the pothole. Overestimation of pothole 1 can be attributed to the presence of soils adjacent to the pothole. However, calibrated red channel (Figure 8b, c) estimated pothole 2 and pothole 3 accurately, with a slight area deviation from the measured potholes. By implications, the sigmoid calibration function demonstrated the ability to reduce noise in UAV imagery, leading to improved accuracy in the recognition of potholes.

4. Discussion

The significance of timely and accurate detection of potholes for maintenance has long been emphasized [44]. In recent years, studies have attempted to segment large scale images for feature recognition [45,46]. However, detection and characterization of potholes from multispectral data without reconfiguring radiance properties to pothole surface may lead to inaccurate determination of pothole size [47]. The study was aimed at calibrating radiometric properties of UAV multispectral imagery to pothole features based on sigmoid calibration function embedded in MLP algorithm. In most cases, the detection of potholes is done manually by inspection, with GPS, recording the location of the pothole at a precision of at most 3 m. However, the accuracy of the GPS location at this precision is questionable, especially for typical roads with a width of about 8 m. In the quest for accurate timely detection of potholes, remote sensing has been endorsed due to its reasonable temporal resolution [48]. In the quest for improved detection of potholes using spectral unmixing of Sentinel-2 raw imagery, Gebreegziabher [49] obtained low accuracy, and this could be associated with the inability of Sentinel-2 sensor in detecting small possible features. Recent studies highlighted the significance of UAV imagery in predicting potholes [50], due to its ability to collect small features of a few centimeters size. UAV-based remote sensing, in particular, has been deemed a suitable option for offering high-spatial resolution images that are capable of detecting potholes. Movia et al. [51] noted that, though ultra-spatial-resolution Unmanned Aerial Vehicle sensors are capable of retrieving multiple forms of landscape information, they are prone to issues related to classification due to noise caused by the presence of smallest possible detectable features. As such, optimized recognition of features from these sensors based on a single or multiple goodness-of-fit numerical approach is required [52]. The sigmoid calibration function algorithm facilitated a successful reconfiguration of spectral radiance properties of UAV imagery to the pothole surface. The presence of the sigmoid function in MLP allows this algorithm to learn abstract features from raw inputs, which is useful for many applications, such as image recognition [34].

A more reliable optimization function must take measurement of uncertainties into account [53]. In this investigation, the reliability of the results obtained using the sigmoid calibration function were evaluated by employing ROC area under curve. Although validation approaches such as the area under margin (AUM) are also capable of evaluating the performance of sigmoid calibration function, this technique was noted to lack the ability to account for account model confidence [53]. The area-under-the-ROC-curve approach has been extensively deployed in the pattern recognition domain [54]. Using this approach, thresholds can also be derived for optimal adaptation to a change in class prior. Fawcett and Niculescu-Mizil [55] noted that ROC-AUC approach can optimize reconfiguration outcomes even when the classifier is poorly calibrated. Generally, pothole surface was noted to have a higher spectral reflectance than asphalt surface in all spectral channels but blue channels. In their analysis of field-based measurements of asphalt road spectra, Li et al. [56] noted that asphalt surfaces under varying aging conditions showed relatively diverse spectral reflectance patterns. Du et al. [57] noted that spectral reflectance in the visible-infrared (VNIR) spectrum increases with the increase in tarred road age. The sigmoid calibration function results revealed its optimization ability lying on the red spectral channel of UAV imagery.

The calibrated spectral channels characterized the pothole surface based on the spectral values that were generally lower than those of the original spectral channel. The performance evaluation results of the sigmoid calibration function in reconfiguring spectral radiance properties to the pothole surface revealed AUC values of 0.53, 0.35, 0.71, 0.19 and 0.35 for blue, green, red, NIR and red edge spectral channels, respectively. The ROC evaluates the model performance based on the AUC values that range between 0 and 1. For a model to be acceptable, the AUC value must be at least 0.5, with a value closer to 0 indicating a perfect model [58]. The ROC results of the sigmoid function evaluation-produced focus was based on the Area Under Curve (AUC), and the sigmoid calibration model developed based on red channel had AUC value of 0.71, which was the better performance in comparison with the sigmoid calibration models developed based on other spectral channels of the UAV imagery. This is due to the fact that when asphalt materials peel off, the spectral reflectance patterns change to the soil beneath asphalt, and the fact that soil type and brightness substantially influence the reflectance in the red channel of spectrum [59]. Particularly, the deployment of relative operating characteristics (ROC) graphs is progressively being commended for performance evaluation of classifiers [60]. Subsequently, the spectral reflectance of potholes from each spectral channel generated through the prediction by the sigmoid calibration model was compared with that of potholes generated from the original spectral channels of the UAV imagery, and all the original spectral channels were more reflective compared to the predicted spectral channels. This is related to the adjustment of high spectral reflectance, such as soil particles scattered around potholes, that tend to cause noise. However, the ability of the absolute value of spectral band to determine the shape of the sigmoid function depends on the spectral properties of the image to be reconfigured. In other spectral bands, road marks (painted in white) were recognized as potholes due to similarity in spectral reflectance properties. Lachiche and Flach [61] noted that when a single threshold is generated from the calibration process, the classification error produced in ROC is insignificant. Moreover, all modeling methods perform under specified assumptions, some of which may not hold in a given application, leading to miscalibration [62]. However, the shape of sigmoid function can be restrictive, and it often cannot produce well-calibrated probabilities when the instances are distributed in feature space in a biased fashion (e.g., at the extremes, or all near a separating hyper plane) [62].

Segmenting image is considered a critical component of image processing and analysis, but it can be very difficult, especially on high-resolution remote sensing images, because different landscape features can have similar spectral properties and the same landscape feature can exhibit varying spectral characteristics [63]. Ultimately, it is challenging to identify the ideal threshold value for a scene or landscape, suggesting that user involvement is necessary [64]. However, it can be assumed that nearby pixels often have the same class label (smoothness assumption) based on the spatial autocorrelations among those pixels [65]. Determining the best thresholding algorithm is not the only conclusion that can be drawn from the results. While threshold-based segmentation works well on uniform backgrounds, it faces limitations in situations where landscape features are subjected to varying illumination conditions. Threshold value definition is based on the pixel intensity of the image, where pixels whose intensity values below threshold are assigned to region 1, and the other pixels are assigned to region 2 [66]. We can easily include extraneous pixels that are not part of the desired region, and we can just as easily miss isolated pixels within the region (especially near the boundaries of the region). These effects get worse as the noise gets worse, simply because it’s more likely that a pixel’s intensity does not represent the normal intensity in the region. The goal of classification algorithms is to predict the class label itself, not its probability. It follows that in order to do its job, the classifier must get a good estimation of the critical region P(Y = 1|x) ≈ 1/2, while it is irrelevant to get a better fit on very high and very low class probabilities.

5. Conclusions

The current study was aimed at enhancing recognition of potholes imaged by UAV multispectral sensor. This was achieved by reconfiguring radiometrically properties of UAV imagery to potholes based on sigmoid calibration function technique embedded in the MLP algorithm and the thresholding approach. The efficacy of the sigmoid calibration function approach was assessed by examining the ROC based AUC. Thresholding facilitated a successful categorization of potholes from both original and calibrated spectral bands. Radiometric comparison of original and calibrated spectral bands revealed that the sigmoid calibration function reduced the noise in ultra-high-spatial resolution imagery. In conjunction with the thresholding technique, sigmoid calibration function also improved the detection of pothole surfaces. Overall, the findings of this research highlight the importance of radiometrically reconfiguring UAV multispectral imagery to potholes for enhanced recognition of potholes. In conclusion, this paper highlighted the ongoing importance of entropy thresholding segmentation and sigmoid calibration function in addressing road and transportation issues that hinder the achievement of sustainable development goals.