Quality Control of Cement Clinker through Operating Condition Classification and Free Calcium Oxide Content Prediction

Abstract

Recent advances in artificial intelligence (AI) technologies such as deep learning open up new opportunities for various industries, such as cement manufacturing, to transition from traditional human-aided manually controlled production processes to the modern era of “intelligentization”. More and more practitioners have started to apply machine learning methods and deploy practical applications throughout the production process to automate manufacturing activities and optimize product quality. In this work, we employ machine learning methods to perform effective quality control for cement production through monitoring and predicting the density of free calcium oxide (f-CaO) in cement clinker. Based upon the control data measured and collected within the distributed control system (DCS) of cement production plants and the laboratory measurements of the density of free lime in cement clinker, we are able to train effective models to stabilize the cement production process and optimize the quality of cement clinker. We report the details of the methods used and illustrate the superiority and benefits of the adopted machine learning-based approaches.

1. Introduction

Recent developments in sensing and computing facilities, high-speed networks, and storage capabilities enable practitioners in various industrial fields to collect and archive a sheer volume of datasets from different perspectives (e.g., performance measurements, control parameter values, etc.) and across the entire industrial production process (raw materials preparation, production quality evaluation, etc.). The availability of such unprecedented amounts of data together with the recent advances of artificial intelligence (AI) technologies such as ensemble learning, artificial neural networks (ANNs), etc., stimulate the incorporation of machine learning (ML)-based approaches into industrial manufacturing. For example, many such efforts in the cement industry are already underway to test and refine machine learning approaches to improve the control of their production devices including raw mills [1,2], rotary kiln [3,4], ball mills [5,6,7], conveyors [8,9], blenders [10], as well as other related manufacturing activities such as cement clinker quality control [11], concrete porosity prediction [12], energy consumption estimation [13], electricity cost optimization [14], hydrating behavior prediction [15], fault detection and diagnosis [4], etc.

The process control of cement manufacturing and production is traditionally conducted manually by human operators. These human operators usually adopt an iterative approach between empirically adjusting the values of control parameters through the distributed control system (DCS) of the cement production plants and observing the corresponding status measurements to stabilize and optimize the manufacturing process and product quality of their respective industrial domains. In the new “intelligent” approach of industrial system control, artificial intelligence (AI) engineers employ model-based optimization and data-driven methods such as machine learning (ML) algorithms to construct pre-trained models to predict, control, and optimize factory production performance. In these approaches, human operators, who are usually industrial experts, will assist AI practitioners in model training and parameter tuning by incorporating domain-specific knowledge. To support process efficiency and cement product quality, one must take into consideration many factors including the discharging speed of raw materials, the temperature of heating devices, the rotation speed of kiln, the fan speed of the ventilation system, and the pressure of the grate cooler, etc. Various sensors are deployed to monitor, measure, and record these factors, and based on which, predictive models are trained and utilized to optimize characteristics that directly influence the cement product quality.

It has long been well-recognized that the cement product quality in a cement plant is largely determined by the cement clinker quality in the rotary kiln [16], in which the content of free calcium oxide (f-CaO) is one of the most important cement quality indicators. The clinker free calcium oxide refers to the calcium compounds that exist in cement materials that do not react with silicates to form hydration products, whose content in the cement clinker directly manifests the strength, hardening time, shrinkage, and other properties of cement products. Therefore, accurately predicting the content of f-CaO in cement clinker is of great significance for the control and optimization of cement manufacturing process.

In this work, we employ a two-step data-driven approach to realize accurate prediction of free calcium oxide content in cement clinker to help optimize the cement production process. Our proposed method first preprocesses the DCS control data and rules out the records that are insignificant or negative in predicting the f-CaO content. This is done by training a ResNet50 [17] deep learning model to classify real-time thermal images showing the material discharge positions in the grate cooler and accordingly identify the DCS data records that are measured and collected under abnormal operating conditions of the rotary kiln. Such classification not only can clean up the dataset and improve the accuracy of f-CaO content prediction, but also can help detect potential problems such as operating anomalies of cement production devices, thus improving the stability and safety of the production process. Secondly, based on the preprocessed DCS data, our proposed method predicts the f-CaO content in cement clinker under normal operating conditions of rotary kiln using the XGBoost algorithm [18], which has the advantage of high performance and high accuracy in capturing the correlation between various features (i.e., process status measurements in DCS) and the f-CaO content. The experimental results show that by correlating the operating conditions of a rotary kiln with the free calcium oxide content in cement clinker, monitoring and control of the clinker production process can be realized and the cement clinker quality and production efficiency can be improved. The adopted method based on image recognition/classification and the XGBoost algorithm can effectively predict the f-CaO content in cement clinker, and provide valuable guidance and optimization strategies for cement clinker production. Our work in this paper concurs with the recent trend of fusing artificial intelligence and high computational power into industrial manufacturing to save production costs while simultaneously producing higher-quality products.

The rest of this paper is organized as follows. We conduct a survey of related work in Section 2. Section 3 reviews the background of the manufacturing process of cement clinker and presents the problem of cement quality control from a data-driven perspective. Section 4 details the design of the proposed two-step learning-based f-CaO content prediction method. Section 5 conducts experiments to evaluate the proposed approach and showcase how Industry 4.0 is transforming the future of the industry of construction materials. Section 6 concludes our work and discusses future work.

2. Related Work

Artificial intelligence (AI) technologies and machine learning (ML) algorithms have been increasingly utilized to solve hard problems that inherit nondeterminism and randomness in engineering practice of a variety of fields including complex system control [19,20], object detection [21], communication networks [22], oil pipeline monitoring [23] and leak detection [24], industrial control system protection [25], waste material recycling [26], etc. In recent years, more and more AI/ML practitioners collaborate with experts in different industrial domains to apply data-driven methods to help solve domain-specific problems and achieve the “Smart Manufacturing” and “Industry 4.0” goals of integrating computing machine intelligence into their respective manufacture production processes [1,2,3,4,5,6,7,10,11,12,13,14,15]. Numerous surveys about the applications of these AI/ML methods to different manufacturing industries have been published. For example, Li et al. in [27] gave a comprehensive review about the adoptions of ML methods in concrete science that have a positive impact. Nian et al. reviewed reinforcement learning (RL) methods and surveyed their applications in industrial process control [28]. Rahman et al. in [29] sketched out a generic framework for using learning-based technologies to solve problems in complex industrial processes. Ramasamy et al. focused on the strategies of tuning predictive controls model and conducted a comprehensive survey covering both analytical methods such as ordinary differential equations and learning-based methods such as neural networks [30].

Here, we particularly focus on the applications of optimization techniques in the concrete and cement industries and present an up-to-date survey about the adoptions of machine learning (ML) and other advanced control methods in various stages and processes of the concrete and cement productions, spanning from energy efficiency [13,31], concrete porosity [12], free lime content [32,33,34], ball milling [6,7], to advanced process control and optimization [30,35]. Lin et al. back in 2006 [36] already adopted data-driven approaches to develop soft sensors for systematically monitoring the cement quality. Later, in 2012, Yuan et al. adopted feedforward neural network with a radial basis function (RBF) activation to perform prediction of free calcium oxide content in clinker and obtained results much better than methods such as linear/non-linear regression [33]; and in 2016, Pani et al. in [37] developed soft sensing models based on feed-forward artificial neural network and fuzzy inference to monitor clinker quality online and observed acceptable results. In more recent years, Chatzilenas et al. applied various machine learning methods to predict energy consumption of the grinding process in cement mills and reported accuracy improvement of nearly one order of magnitude above their baseline [31]. Guzmán et al. used artificial neural networks and genetic algorithms to optimize the cost of electricity in cement plants with electricity costs regulated country by country and factors affecting electricity price taken into account [14]. Cao applied regression tree ensembles to predict the porosity (a durability indicator of concrete) of concrete containing supplementary cementitious materials and further empirically identified important dominating features that impact the concrete porosity, i.e., curing days, water/binder ratio, and aggregate content [12]. Liu et al. conducted similar work using support vector machine (SVM) ensembles in [32]. Zhao et al. tried to solve the same free lime content modeling problem using a particle swarm optimization (PSO) algorithm [34]. Andreatta et al. developed soft sensors based on a backpropagation neural network model to perform online prediction of the cement fineness inside of a ball mill [7]. Li et al. incorporated convolutional neural network (CNN) into a modeling framework and then invoked transfer learning methods to use pre-train models to predict ball milling performance of cement production [6]. Zanoli et al. applied advanced process control techniques to tune the critical process variables of clinker rotary kiln and clinker grate cooler to achieve minimization of energy consumption in terms of fuel, coal, and electricity [35].

Besides applications to the control and optimization of the cement manufacturing process itself, learning-based methods have also started to be applied in the transportation of industrial products and the safety assurance of industrial control systems. For example, Dwivedi et al. used deep learning to detect the cracks of longitudinal conveyor belts used in construction tunnels [9], such an approach may also be adopted in concrete industry as conveyor is an indispensable component in cement production as well. Raman et al. used ML algorithms to learn the process dynamics and control strategies deployed in industrial control systems, which led to easier and faster development of outlier and intrusion detectors [25]; similarly, such approach could and should be employed in cement production to ensure the safety of operational plants, which seemed to be inevitable with the intelligentization of concrete manufacturing process.

3. Background and Problem Statement

We first briefly review the background of the manufacturing process of concrete cement clinker and formulate the problem of cement clinker quality control via f-CaO content prediction.

3.1. Production Process of Cement Clinker

Most modern cement plants use a dry process to produce cement by first heating crushed materials including clay, slate, blast furnace slag, etc., and then cooling and grinding the resultant substance to create the fine powder of the clinker. Such dry processes are more thermally efficient than the traditional wet process and thus are more environmentally friendly. Cement clinker results from pyroprocessing, and there are several main steps in such a calcination process of clinker production, including:

• Preparation and pre-treatment of raw materials, where quarried limestone is crushed along with clay, slate, blast furnace slag and other components, to a proper size of no more than 3 inches;
• Pre-heating, where the crushed raw materials are heated in the multi-stage pre-heater and then fed into a rotary kiln to raise their temperature to approximately 1370 °C (about 2500 °F); see in Figure 1;
• Calcination and sintering in rotary kiln, where as the raw materials travel along the rotary kiln towards the firing zone, the raw materials lose moisture and other components and form a rock-like substance called clinker that are mineral lumps roughly one inch in diameter; see in Figure 1;
• Cooling the grate cooler, where the hot cement clinker leaves the rotary kiln, enters into the grate cooler to get cooled/quenched rapidly to prevent further reactions and preserve the desired mineralogical composition, and the heat is recycled to the pre-heater to improve efficiency, save financial cost, and reduce the environmental impact of the process; see in Figure 1;
• Grinding in ball mills, where the cool clinker is milled to form a fine powder, which is the final product known as cement;
• Post-treatment, where additional cement components, e.g., calcium sulphate to control the setting time, are added;
• Packaging and transportation, where the finished cement products are bagged, moved via conveyor, and finally shipped.

The various steps in the calcination process are closely related and mutually constrained. They have a tight coupling and together build a complete clinker calcination thermodynamic system, Figure 1 shows a simplified schematic view of such calcination and sintering process and the corresponding cement plant devices. The operating conditions in the whole aforementioned process must be carefully controlled to obtain the desirable cement properties.

3.2. Prediction of Free Calcium Oxide (f-CaO) Content

As stated in Section 3.1, the cement clinker production is a very complex process that involves various components and many manipulatable parameters. The ultimate objective of the production process control is to obtain the best quality of the clinker by iteratively adjusting the control parameter values through the distributed control system (DCS) of the cement plants in response to the observations and measurements of the quality attributes of the cement clinker.

The free calcium oxide (f-CaO) in cement clinker, which refers to the calcium oxide (CaO) present in the cement clinker that has not participated in the chemical reactions and exists in a free state, reflects the quality of the clinker to a significant extent. The residual unreacted calcium oxide left in the clinker after the burning process only starts to hydrate slowly only after the cement has undergone hydration, hardening, and achieved a certain level of strength. This hydration of free calcium oxide is usually accompanied by a certain degree of uneven expansion, which can potentially lead to a decrease in cement strength, causing cracking or even collapse and resulting in poor cement stability. Therefore, the content of free calcium oxide (f-CaO) in cement clinker is commonly regarded as one of the most important indicators used to assess cement quality.

Currently, most cement manufacturing plants use manual laboratory methods to determine and measure the f-CaO content in cement clinker, which takes around one hour from sampling the clinker to obtaining the test results. This approach has certain drawbacks: it has a lag in providing guidance for adjustments in the operating conditions of cement rotary kilns, and the test results are susceptible to various factors such as the technical skills and experience of the human operators. As a result, there is a certain level of measurement inaccuracy, delay, and error in the test results of f-CaO content in cement clinker, making it difficult to be used for promptly detecting and addressing issues during the cement production.

It is difficult to adjust control parameters of cement production and achieve satisfactory results by first constructing analytical models of the complex cement manufacturing process and then relying on the test results of f-CaO content to conduct the adjustments due to poor process understanding and delayed delivery of f-CaO content data. Because of this, using soft sensors based on data-driven models derived from the actual input–output process data, which in many cases are readily available in different forms of distributed control system (DCS) deployed in most cement plants, is becoming a common approach adopted in the intelligent transformation practice of cement industry. By monitoring the status measurements in real time, such soft measurement methods can provide accurate data support for predictive process control, thereby addressing challenges in establishing traditional non-linear analytical process models. It is particularly suitable for cement production, which is a continuous process and is influenced by different equipment at various stages, to establish non-linear models by extracting features from historical multidimensional data and accurately predicting f-CaO content in cement clinker.

To save the cost and provide more accurate and timely guidance for the process control of cement production, we address the limitations of traditional manual laboratory methods for f-CaO content tests and train machine learning models that characterize the relationship between key measurable variables in the process of cement production and the content of f-CaO in cement clinker. Based on the models trained using datasets of time-series measurements, we predict the f-CaO content in cement clinker at a future time based on the ongoing monitoring and recording of the processing status of cement production.

More formally, the prediction problem of the f-CaO content in cement clinker can be formulated as follows. We are given a set of control data records continuously measured and collected in the DCS system up to time point T, denoted as $D_T=\{{\mathbf{x}}_i=\langle x_{i,1},x_{i,2},\dots ,x_{i,j},\dots ,x_{i,N}\rangle ,y_i\}$, where $i=1,2,\dots ,T$ signifies the time points when DCS monitoring sensors collect status data of the cement production plants and $j=1,2,\dots ,N$ is the number of the data attributes (i.e., features) in the dataset; our ultimate goal is to predict the near future f-CaO content $y_{t^{\prime }}$ in cement clinker at time $t^{\prime }>T$, based on $D_T$.

The attributes in the feature vector ${\mathbf{x}}_i$ have different ranges of possible values in a parameter space of multiple dimensions of different magnitudes, and the DCS measurement dataset $D_T$ can be viewed as samples from this space. These samples are used to generalize to the whole feature set and the manufacturing process for prediction purposes, and we use machine learning methods to train predictive models based on historical DCS control and measurement data. More specifically, we utilize the time-series sensor measurements exported from the DCS system of a cement plant as a training dataset $D_T$ that consists of N laboratory test results of f-CaO content, i.e., $D_T=\{({\mathbf{x}}_1,y_1),({\mathbf{x}}_2,y_2),\dots ,({\mathbf{x}}_N,y_N)\}$, where ${\mathbf{x}}_i,i=1,2,\dots ,N$ is an instance of $\mathbf{x}$ that collectively determines the corresponding f-CaO content $y_i$ at time point i. We propose employing machine learning-based methods to estimate $y_{t^{\prime }}$ based on $D_T,t^{\prime }>T$, such that the predicted (estimated) value ${\widehat{y}}_{t^{\prime }}$ is close enough to the ground truth $y_{t^{\prime }}$ for all testing samples $({\mathbf{x}}_t,y_t),t\le T$, and can be applied to arbitrary future cases (i.e., when $t^{\prime }>T$) with high accuracy based on $D_T$.

4. Content Prediction of Free Calcium Oxide (f-CaO) in Cement Clinker Using Machine Learning

In this section, we detail the adopted machine learning approach to perform prediction of f-CaO content in cement clinker.

4.1. Overview

Figure 2 sketches out schematic steps of the proposed two-step data-driven method.

Firstly, we utilize the image datasets that reflect production conditions such as clinker appearance, fly ash, brightness and shape of the flame inside the rotary kiln, to identify and judge the current operating conditions. Through image classification, it determines whether the current operating condition is normal. If the current condition is normal, we proceed by reading production status data from the sensors deployed on the cement production line (this is done via the DCS system) and obtaining results of the free lime content of cement clinker from the laboratory tests. We then preprocess the sensor-collected DCS dataset and select the most important (relevant) features for later model training. If the current conditional is abnormal, we alarm the DCS control system and provide the image classification results to help guide the adjustment of the control parameters that either be performed by human operators or an adaptive control agent of the DCS if available.

Secondly, based on the image classification results and their associated DCS datasets as well as the corresponding selected important features, we invoke a proper machine learning algorithm (e.g., XGBoost) to train a prediction model and use it to predict the values of the f-CaO content in near future time.

4.2. Identification of Abnormal Operating Conditions in Cement Production

The identification of abnormal operating conditions in this work is done by training a ResNet50 [17] deep learning model to classify real-time thermal images showing the material discharge positions in the grate cooler. The ResNet50 model incorporates an additional convolutional layer within each residual block inside of the network to increase its depth and thus model generality and typically performs well in image classification tasks. In the context of our problem, the ResNet50 model learns from the “flying ash” features in real-time images to determine their respective normalities of operating conditions.

The loss function of the ResNet50 model is an enhanced version of the cross-entropy loss, known as the focal loss. This focal loss function is defined as follows:

$$FL\left(p_t\right)=-{(1-p_t)}^{\gamma }log\left(p_t\right),$$

where $\gamma$ is a constant, and its value being 0 makes the focal loss degenerate to the regular cross-entropy loss; $p_t$ is defined as follows:

$$p_t=\left\{\begin{array}{c}p,\mathrm{if}y=1\hfill \\ 1-p,\mathrm{otherwise}\hfill \end{array},\right.$$

with p being the model’s estimated probability for the ground-truth class with label $y=1$.

Such a focal loss in ResNet50 [38] naturally solves the class imbalance problem, and therefore is particularly suitable to our problem since most of the real-time thermal images are collected under normal operating conditions in the cement production line, while the ones showing phenomenons that reflect abnormal operating conditions are the minority, considering the fact that most of the time the cement production line can operate and continuously produce cement clinkers without major problems.

The trained ResNet50 model in this work compares and classifies the thermal images showing the material discharging positions inside of a cement grate cooler. It does so by assessing whether the central bright area of the real-time image is obstructed by flying smoke and sank, known as “flying ash”. This assessment is used to determine the status of abnormal operating conditions. As shown in Figure 3, if there is substantial smoke and ash obstruction in the center area, the model examines whether there is smoke and ash on both sides of the image as well. If significant smoke and ash is detected, the “flying ash” phenomenon is determined, thus abnormal operating conditions are detected. On the other hand, as shown in Figure 4, if the central area of the thermal image is brightly lit and there is no obvious smoke or ash on either side, the condition is categorized as no “flying ash”; thus, normal operating conditions are detected.

To study the reliability and have better interpretability of the adopted models in this work, we leverage the deep features learned by the ResNet neural networks and generate heatmaps based on the Class Activation Maps (CAM) [39] to highlight the regions in the thermal images that are crucial for determining a particular class. The results presented in Figure 5 show the parts of the thermal image that are “looked at” by the neural networks, where the heatmap results mark out the regions that are the most salient for making a classification decision. Contrasting Figure 5a and Figure 5b, one can clearly see that the trained ResNet50 models can identify the materials discharge positions from the thermal images of grate cooler. Over these discharge positions, the “flying ash” may appear when the cement clinker production is operating under abnormal conditions, which eventually would lead to poor clinker quality and impaired accuracy of f-CaO content prediction. Therefore, preprocessing the thermal image datasets using classification itself can also be viewed as an alarming mechanism, and based on which, ruling out the DCS control data under abnormal operating conditions help increase the accuracy of f-CaO content prediction (see Section 5 for such experimental results). Note that the images shown in Figure 5 represent the ones fed into the neural networks models, which are resized and thus have lower resolutions than those in Figure 3 and Figure 4.

4.3. Feature Selection in the Distribute Control System Data

We invoke a simple iterative permutation approach to evaluate the significance of various features in the DCS system dataset on the performance of the predictive models (other automated feature selection methods, e.g., [40], may also be used). This assessment is conducted using the permutation feature importance metric applied to the obtained prediction results, which involves randomly shuffling the features and accordingly measuring the corresponding changes in the prediction results of the trained models, thus quantifying the importance of each feature and its effects on the predictive model. Based on this evaluation, the following five features, out of many others, demonstrate substantial importance during the process of random shuffling, and are consequently selected for model training:

• Inlet temperature of waste heat power generation. The inlet temperature of waste heat power generation refers to the temperature of the waste heat entering the power generation system, which reflects the cooling efficiency of the grate cooler. A high inlet temperature of waste heat power generation indicates poor cooling efficiency of the grate cooler, resulting in slow cooling of the clinker and inadequate dispersion of internal heat, leading to an unstable and thus lower-quality clinker structure. If the grate speed is stable but the inlet temperature of waste heat power generation is too low, it could suggest that the clinker temperature is insufficient.
• Secondary air temperature. The secondary air refers to the air that is reintroduced into the rotary kiln after undergoing heat exchange at the kiln outlet. Maintaining an appropriate temperature for the secondary air is important in cement production. An optimal secondary air temperature can facilitate the sintering of clinker particles, enhancing the degree of sintering and consequently improving the strength and stability of the cement. Moreover, maintaining an appropriate secondary air temperature can stimulate the hydration reaction of gypsum, leading to the production of an appropriate amount of hydrated calcium sulphoaluminate, which, in turn, enhances the cement’s property of resistance to sulfate erosion.
• Flame temperature in rotary kiln. The rotary kiln flame temperature refers to the temperature of the combustion flame inside a rotary kiln used in cement production. This temperature is a critical parameter as it directly affects the efficiency of the combustion process, the quality of product formation, and energy consumption. An appropriate flame temperature can enhance the sintering degree of clinker, stimulate the formation of mineral phases, and facilitate the growth of crystals within the clinker, contributing to an increase in the strength and stability of the resulting cement clinker.
• Clinker temperature. Clinker temperature refers to the temperature of the solid material that emerges from the rotary kiln in cement production after undergoing various chemical reactions and thermal processes. It is one of the key factors that affect the degree of sintering in clinker formation. Increasing the clinker temperature enhances the fusibility of coal ash, making it more prone to react with the mineral materials inside of the rotary clinker. This improvement in reactivity can lead to enhancements in cement quality. Clinker temperature changes can potentially impact the morphology and structure of clinker particles, subsequently influencing the particle distribution and flow characteristics of the resulting cement clinker.
• Grate bed pressure. Grate bed pressure refers to the pressure exerted on the surface of grate coolers in cement production. The pressure on the grate bed is an important operational parameter in controlling the cooling process. Adjusting the pressure on a grate bed can influence the distribution of clinker particles. Appropriate pressure can lead to an even distribution of clinker particles on the grate bed, reducing particle accumulation and adhesion, which is beneficial for rapid cooling of the clinker. Controlling the pressure on a grate bed also affects the thermal balance of the bed. Maintaining proper pressure can help reduce overheating of the grate bed, ensuring its stable operational state and enhancing the sintering efficiency of the clinker.

4.4. Preprocessing of the Dataset

The distributed control system (DCS) is now embracing a changing role in cement production, from traditional manufacturing process automation [41] to data-driven intelligent control and optimization.

The dataset of time-series type used for the predictive model training in this study is sourced from the DCS system of a cement production line. Besides the standard data preprocessing performed in a typical machine learning project such as outlier removal, data imputation, data smoothing, etc., we also perform non-trivial data reduction/aggregation to accelerate the model training and improve the prediction performance, as follows.

Since the monitoring sensors within the DCS system measure various indicative factors every 2 s; in total, 1800 data points per hour are obtained from the DCS sensors and are then stored in the DCS database system. The approach adopted in this study involves initially smoothing the DCS data to reduce noise and abrupt variations in the original data, enhancing data readability and interpretability. Subsequently, the data are downsampled through averaging to form a 2-D matrix format of M by N, where $M=60$ is the number of data records per hour, and N represents the number of selected features, to eliminate redundant information from the data and expedite the training process. Consequently, this preprocessing transforms every 30 original DCS data points to create a single data point (i.e., from 1800 records to 60 records per hour) for the predictive model training.

4.5. Prediction Model Based on XGBoost

The XGBoost algorithm is employed to predict the quality of cement clinker based on the free lime content. The XGBoost algorithm [18] uses gradient boosting trees as an ensemble learning method, iteratively training a series of decision tree models for prediction. In this approach, each decision tree model functions as a weak classifier, and even though their predictive ability might be relatively weak, the XGBoost algorithm combines them iteratively to obtain a strong classifier. This enables the modeling of complex nonlinear relationships and enhances prediction accuracy.

Consider a training dataset $D_T=\{{\mathbf{x}}_i=\langle x_{i,1},x_{i,2},\dots ,x_{i,j},\dots ,x_{i,N}\rangle ,y_i\},i=1,2,\dots ,T$, a loss function $l(y_i,{\widehat{y}}_i)$, and a regularization term $\mathsf{\Omega }\left(f_k\right)$, the overall objective function of XGBoost algorithm can be denoted as:

$$\mathcal{L}\left(\varphi \right)=\sum _{i}l(y_i,{\widehat{y}}_i)+\sum _k\mathsf{\Omega }\left(f_k\right),$$

where $\mathcal{L}\left(\varphi \right)$ represents the objective function in the linear space, i stands for the i-th data sample, k denotes the k-th decision tree, ${\widehat{y}}_i$ signifies the predicted value corresponding to the i-th sample ${\mathbf{x}}_i$, i.e., ${\widehat{y}}_i={\sum }_k{\widehat{f}}_k\left({\mathbf{x}}_i\right)$, and ${\sum }_k\mathsf{\Omega }\left(f_k\right)$ represents the complexity of the k-th tree.

Recall Figure 2 and Section 4.1, and note that since the predicted values under normal operating conditions are more valuable as references for guiding the control and optimization of the cement production, we utilize a ResNet50 deep learning model to classify and determine the operating conditions of the production line based on real-time thermal images, distinguishing between normal and abnormal operating conditions. Further, the DCS data records that are associated with the abnormal operating conditions are ruled out. In other words, the dataset used in the aforementioned model training in this section is the one that is frequently sampled from the production line data via the DCS system, with data records that reflect abnormal operating conditions eliminated, and is then preprocessed following the method in Section 4.4.

5. Evaluation

We present the evaluation results of the proposed apporach in this section. Our evaluations are conducted based on a dataset collected from the DCS control system and corresponding operating condition monitoring system in a real-life cement production factory, consisting of frame images extracted from monitoring videos and tabular records exported from the DCS databsae. Three models ResNet50+XGBoost, XGBoost, and LSTM [42] are compared.

5.1. Experimental Settings

The implementations of the experimental evaluation, including model training, prediction, performance measurements, are all done using Python programming language along with open-source libraries including Scikit-Learn [43], TensorFlow [44], and dmlc/xgboost [45]. In the experiments, we first split a dataset of 5037 records into training and testing sets following an 80–20% ratio, and subsequently feed the training set into the XGBoost algorithm for model training. Grid search and cross validation are both used to tune the parameters of the XGBoost algorithm. More specifically, within the pre-defined parameter ranges and with a properly chosen step size, an iterative process was executed to explore all possible combinations of hyperparameters. Based on the results of cross-validation evaluations between each combination, the algorithm sequentially adjusted parameters to seek the optimal hyperparameter configuration. Due to the incorporation of regularization terms and control over tree complexity, the XGBoost algorithm demonstrates a degree of robustness against noise and outliers within the training data.

5.2. Results

The experimental results indicate that the XGBoost algorithm based on operating condition recognition and classification can predict the content of f-CaO with relatively higher accuracy (Figure 6) in comparison with that without the preprocessing based on image classification (Figure 7). As illustrated in Figure 6,Figure 7 and Figure 8, the “ground truth” (red) lines represent the actual values of the f-CaO content in cement clinker, i.e., the results obtained from laboratory tests, while the “predicted” (blue) lines represent the predicted f-CaO content in cement clinker. As shown in Figure 6, the overall mean absolute error (MAE), mean squared error (MSE), root mean square error (RMSE) of the predictions with preprocessing based on image classification are approximately $0.21$, $0.08$, and $0.28$, respectively; a 16%, 27%, and 15% improvement than that in Figure 7 without the preprocessing. The prediction accuracy improvement of the proposed method over LSTM is also significant, specifically, a 25%, 38%, and 22% upgrade than that in Figure 8 are observed in our evaluations.

To illustrate the robustness and generalizability of the proposed solution, we present in Figure 9 the training loss and validation loss of the adopted models. The results in Figure 9 show that: (i) the training loss first plunges down with every epoch, hinting that the models learn and absorb information from the datasets; (ii) the validation loss, during the same epochs, is slightly higher than the training loss and also drops smoothly, indicating that the training gets the model more seasoned with the training datasets and is not under-fitting; and (iii) as the epochs continue, the training loss reaches the plateau, and the validation loss keeps decreasing and becomes smaller than the training loss, showing that the models are not over-fitting. In sum, the trained models both have a good fit to the training datasets and generalize well to be used for new datasets.

6. Conclusions and Future Work

We applied machine learning (ML) methods to perform effective quality control for cement production by identifying abnormal operating conditions in the clinker manufacturing process and predicting the content of free calcium oxide (f-CaO) in cement clinker. Our ML approach was based on the control data measured and collected in the distributed control system (DCS) of a real-life cement production plant.

Considering that the predictive results under normal operating conditions are more valuable as references to optimize the cement production process, we trained a ResNet50 deep learning model to identify abnormal operating conditions based on real-time thermal images and further eliminated associated DCS data under abnormal operating conditions. The model training in this work was based on frequently sampled production line data, which enhanced the information richness concerning various measurable indicators of the cement production process within the data samples with respect to the prediction of the f-CaO content. We performed various data preprocessing on the DCS production line data, including outlier detection, data smoothing, as well as downsampling, to reduce data redundancy and minimize the effects of abnormal values and abrupt variations that existed in the original dataset on the model training.

It is of our interest to further improve the performance of the both steps in the proposed methods by adopting more advanced models such as RNN, Prophet, Attention, etc. as well as other helpful sophisticated approaches such as automated feature selections.

In addition, in observation of the techniques developed in the field of explainable artificial intelligence (XAI), we would like to adopt them in our work of integrating AI technologies to transform and transition traditional production factories to smart manufacturing industries. For example, using techniques such as feature importance ranking to further refine feature engineering/selection to speed up the model training and improve the solution efficiency; using techniques such as saliency maps and layer-wise relevance propagation to highlight and correlate the inputs and outputs of the trained machine learning models. Such work will significantly improve the transparency and interpretability of the models used in our product solutions, which will help the domain users to conceive trustworthy in our AI-based solutions in various manufacturing industries.

Besides the cement industry, it is also of our great interest to explore the opportunities of adopting AI/ML technologies in various other manufacturing and production industries including petroleum industry, e.g., oil/gas production optimization, pipeline monitoring and leak detection; energy industry, e.g., power grid safety monitoring, power consumption forecasting, etc.