Transferability of Multi-Objective Neuro-Fuzzy Motion Controllers: Towards Cautious and Courageous Motion Behaviors in Rugged Terrains

Abstract

This study is motivated by the need to develop generic neuro-fuzzy motion controllers for autonomous vehicles that may traverse rugged terrains. Three types of target problems are investigated. These problems differ in terms of the expected motion behavior, including cautious, intermediate, and courageous behaviors. The target problems are defined as evolutionary multi-objective problems aiming to evolve near optimal neuro-fuzzy controllers that can operate in a variety of scenarios. To enhance the evolution, sequential transfer optimization is considered, where each of the source problems is defined and solved as a bi-objective problem. The performed experimental study demonstrates the ability of the proposed search approach to find neuro-fuzzy controllers that produce the required motion behaviors when operating in various environments with different motion difficulties. Moreover, the results of this study substantiate the hypothesis that solutions with performances near the edges of the obtained approximated bi-objective Pareto fronts of the source problems provide better transferability as compared with those that are associated with performances near the center of the obtained fronts.

1. Introduction

Increasing interest from the car industry in autonomous driving and the availability of new technologies have given rise to vast research efforts on the control of autonomous vehicles (e.g., [1,2]). However, such studies have hardly dealt with off-road autonomous driving. This study is motivated by the expected difficulties that a motion control system of an autonomous vehicle will face when traversing rugged terrains. For such off-road situations, it is envisioned that a range of motion behaviors might be of interest, spanning from a cautious behavior to a courageous one. As a first step towards the development of such control systems, this study considers an academic problem that applies a dynamic model of a cart that interacts with various environments. These environments differ by their motion difficulties, which are expressed by various friction functions and varying speed limitations. An attractive approach to address the development challenges of such controllers is to use a multi-objective evolutionary neuro-fuzzy optimization approach (e.g., [3,4]). The use of such an evolutionary approach, as compared with a traditional neuro-fuzzy approach, e.g., ANFIS [5], has three main advantages. First, it avoids the need to rely on an expert to determine the relevant fuzzy rules. Second, it avoids the need to create data sets for the tuning of the parameters of the membership functions. In addition, the use of an evolutionary search is most suited for solving multi-objective optimization problems, i.e., to find a set of Pareto-optimal solutions [6]. In this kind of optimization problem, no prior articulation of the objective preferences is provided. As described in the following, this study utilizes the above three advantages while focusing on the use of multi-objective optimization in conjunction with transfer optimization.

This study deals with a Sequential Transfer Optimization (STO) approach [7,8]. Namely, it assumes the existence of relevant knowledge from solving multi-objective source problems to the solution of multi-objective target problem(s). Moreover, this study involves an evolutionary STO in which a genetic transfer is employed. Namely, the knowledge is transferred via the use of individual solutions of the source problems to serve as an initial population for solving the target problems [9,10].

When defining the target optimization problem for the development of robot motion controllers, it is important to realize the difference between specialized and non-specialized controllers [11]. Specialized controllers are optimized for a particular environment, whereas non-specialized controllers are optimized for a set of environments. For the current motivation, it appears that the development of controllers for rugged terrains should focus on non-specialized controllers that may cope with a variety of terrains. According to [11], defining the problem as a meta-problem, which aims at performance optimization for a set of environments, produces both specialized and non-specialized neuro-controllers.

Following [11], this study considers the target problem as a meta-problem, i.e., the problem is a multi-objective optimization problem in which each objective is associated with a different environment. In this study, we define three types of behavior-based meta-problems. These problems differ by the desired behavior, including cautious, courageous, and intermediate behaviors. For example, we aim at non-specialized controllers that produce courageous behaviors in each one of the environments.

We seek to extract knowledge from a set of source problems to support solving the three types of target problems considered. The source problems are defined as optimization problems that differ by the source environment. Each of these problems is defined as a bi-objective Pareto-optimization problem in which one of the objectives corresponds to a cautious behavior and the other one corresponds to a courageous behavior. Each of these bi-objective source problems is solved using Pareto-optimization. Solving a set of source problems, which differ by the considered environment, results in a set of Pareto-optimal sets. The union of these sets is expected to hold relevant knowledge to support solving the target problems. Namely, it is expected to hold controllers of different behaviors that are relevant to different environments. While the diverse solutions of the union set are considered an advantage, they raise the following difficulty.

In general, a union set of Pareto-optimal sets of solutions from several source problems has a large cardinality. This cardinality is expected to be much beyond the size of the population that is needed to solve the target meta-problem. This study aims to answer a major question that results from the large cardinality of the union set. Namely, what subset of the individual solutions of the bi-objective source problems is more suited to serve as the initial populations for solving the target meta-problem? To provide an answer to this research question, two types of knowledge extraction techniques are suggested and investigated. The first type involves an initial population of edge controllers, i.e., controllers with performance vectors that are located close to the edges of the obtained fronts. The second type of knowledge extraction is based on an initial population of center controllers, i.e., controllers whose performance vectors are located close to the center of the obtained fronts. In addition, these types of knowledge extraction are compared with random initialization. We compare the three types of initial populations with respect to the three behavior-based target meta-problems considered here. We aim to substantiate the hypothesis that the use of edge controllers is a preferred approach regardless of the considered target problem.

Given the state-of-the-art, as presented in Section 2.3, the main contributions of this paper are:

(a)

It is the first study that defines target meta-problems that aim to find non-specialized Neuro-Fuzzy Controllers (NFCs) that produce various motion behaviors.

(b)

It is the first study that applies knowledge extraction from solving multi-objective source problems to support finding non-specialized NFCs that produce various motion behaviors for environments that are different due to their motion difficulties.

(c)

It is the first study that raises the hypothesis that the genetic transfer of edge solutions of multi-objective source problems is preferred over that of the center solutions. Furthermore, it is the first study to substantiate the hypothesis with respect to the considered type of evolutionary neuro-fuzzy control problem.

The rest of this paper is organized as follows. Section 2 describes the background for this study and locates this study with respect to the state-of-the-art. Section 3 introduces the problem formulation and the proposed solution approach. Next, in Section 4, details are provided on the experimental study. Finally, Section 5 summarizes and concludes this study.

2. Background

This section provides the background for this study and a state-of-the-art analysis related to the contributions of this study. First, Section 2.1 presents the basic principles of evolutionary multi-objective optimization. Then, Section 2.2 describes the use of such optimization in the design of NFCs. Finally, Section 2.3 provides the positioning of this research with respect to the state-of-the-art.

2.1. Multi-Objective Optimization

Engineering design commonly involves decisions under conflicting objectives. In such situations, improving performance in one objective may deteriorate the performance in the other objectives. To make decisions based on the understanding of the performance tradeoffs, more than two decades ago, engineers started to apply Pareto-based optimization [6]. This type of optimization results in a set of Pareto-optimal solutions and their associated Pareto-front in the objective space. This allows a posteriori solution selection based on assessable performance tradeoffs that are exposed by the obtained front. The following highlights the fundamentals of Pareto optimization.

Pareto-based multi-objective optimization is commonly described as a vector optimization problem with M objectives to be optimized. Without the loss of generality, the problem can be defined as a minimization problem as follows:

$$\underset{\mathit{x}\in X}{\min }\mathit{F}\left(\mathit{x}\right)$$

(1)

where $X$ is the set of feasible solutions and $\mathit{F}={\left[F_1\left(\mathit{x}\right),F_2\left(\mathit{x}\right),\dots ,F_M\left(\mathit{x}\right)\right]}^T$ is the performance vector.

Let two solutions of (1), $\mathit{a}$ and $\mathit{b}$, be associated with $\mathit{F}\left(\mathit{a}\right)={\left[F_1\left(\mathit{a}\right),\dots ,F_M\left(\mathit{a}\right)\right]}^{\mathit{T}}$ and $\mathit{F}\left(\mathit{b}\right)={\left[F_1\left(\mathit{b}\right),\dots ,F_M\left(\mathit{b}\right)\right]}^{\mathit{T}}$, respectively. Then, $\mathit{a}$ is said to dominate $\mathit{b}$, denoted by $\mathit{a}\preccurlyeq \mathit{b}$, if the following condition is satisfied:

$$\forall i\in \left\{1,\dots ,M\right\},F_i\left(\mathit{a}\right)\le F_i\left(\mathit{b}\right)\mathrm{a}\mathrm{n}\mathrm{d}\exists j\in \left\{1,\dots ,M\right\}\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}{F}_j\left(\mathit{a}\right)<F_j\left(\mathit{b}\right)$$

(2)

Without loss of generality, for a given minimization problem, the Pareto-optimal-Set (PS) is the set of all Non-Dominated Solutions (NDS) of the feasible set $X$. Namely, a solution $\mathit{a}\in X$ is called Pareto-optimal if and only if there is no solution $\mathit{b}\in X$ for which $\mathit{b}\preccurlyeq \mathit{a}$. The set of performance vectors corresponding to the aforementioned set is known as the Pareto-Front (PF). Solving (1) means finding the PS and the associated PF.

Usually, there is no analytical solution to real-life Multi-objective Optimization Problems (MOPs) [12]. Therefore, computational techniques are commonly employed to obtain at least good approximations of the PS and PF. Dedicated evolutionary algorithms for solving MOPs are termed Multi-Objective Evolutionary Algorithms (MOEAs). MOEAs, including NSGA-II [13], which is used in this study, follow the ideas of natural evolution, such as selection, recombination, and mutation, aiming to evolve a population of solutions towards the Pareto-optimal solutions. With over two decades of proven success, MOEAs are the leading algorithms for solving MOPs. This is evident from reviews such as in [12,14,15].

2.2. Neuro-Fuzzy Systems and Control

Fuzzy Inference Systems (FISs) are computational frameworks that implement fuzzy reasoning to make decisions. The mathematical basis, which was founded by L. Zadeh (1965), defines linguistic variables, fuzzy sets, and logical operators extending crisp set theory [16]. Fuzzy sets are defined using Membership Functions (MFs), while Fuzzy Rules (FRs) express relationships between such sets. The structure and parameters of FRs and MFs determine the mapping of given inputs to outputs. There are at least two main differences between mapping by an FIS as compared with mapping by a Neural Network (NN). First, an FIS can be formed based on expert knowledge, while an NN commonly requires data. Second, an FIS is interpretable, while an NN is considered to be a black box. The main advantage of using an NN is that the mapping can be learned and adapted. A Neuro-Fuzzy (NF) approach synergistically combines the main features of FISs and NNs. The advantages of the NF approach over the FIS approach have created new research and application opportunities [3,17,18,19]. As evident from the literature, over the years, both FISs and NF systems have been applied to control design [20,21,22]. The common approach to tuning NF controllers is supervised learning based on some data (e.g., [23,24]). However, the design of NFCs can be based on evolutionary techniques [25] to reach the desired optimum/optima without adhering to any data (e.g., [26]).

Using the concept of Pareto optimization for the design of controllers is not new (see review [27]). However, applying Pareto-based optimization/adaptation to the design of neuro-fuzzy systems is rare (e.g., [4,28,29]). Several studies tried to find Pareto-optimal fuzzy controllers with static structure, e.g., [30,31,32], whereas in [26], such studies were extended to also include the rule structure.

2.3. Positioning of This Research

The idea of defining and solving meta-problems for obtaining specialized and non-specialized controllers has been recently suggested in [11]. The current study is the first study to use this idea in the context of evolutionary neuro-fuzzy control.

Applying STO to solving neuro-fuzzy control problems is not new [33,34,35,36,37,38,39,40]. However, in this study, several uncommon/unique aspects concerning the use of STO for such problems should be noted. First, commonly in STO studies on solving neuro-fuzzy control problems, the knowledge that is transferred is data, which is used in the context of supervised learning (e.g., [33,34,35]). In contrast, the current study applies a different type of knowledge that is transferred, namely, a genetic type. Second, the majority of STO studies on solving neuro-fuzzy control problems are based on using just one source problem (e.g., [37]), while the current study is based on the use of multiple source problems. Moreover, in contrast to the current study, to the best of our knowledge, no study on NF systems concerns source problems that are defined as MOPs. To substantiate this claim, we conducted the following Scopus search:

(TITLE-ABS-KEY ((neuro-fuzzy OR “neuro fuzzy”))) AND (“source problem” OR “source task” OR sequential) AND (“transfer learning” OR “transfer optimization”). A manual check of all the resulting documents revealed no study that deals with source problems that are defined as MOPs.

While the above positioning is restricted to STO studies that are concerned with neuro-fuzzy problems, to the best of our knowledge, this study contains a major difference with respect to any STO study. We claim that this study is the first to raise and investigate the hypothesis that the use of edge solutions of multi-objective source problems for genetic transfer is preferred over the use of the center solutions. To substantiate this novelty assertion, we conducted the following Scopus search:

(TITLE-ABS-KEY ((Pareto OR multi-objective OR “multi objective”) AND transfer)) AND (“transfer learning” OR “transfer optimization”) AND (sequential OR “source problem” OR “source task”). A manual check of all the resulting documents revealed no study that concerns the suggested hypothesis.

For convenience,

Table 1. Positioning Table with respect to STO Neuro-Fuzzy studies.

Related Study	Evolutionary Driven	Pareto-Based Source	Application
Aouf et al., 2018 [33]	X	X	Indoor robotics
Juang & Bui 2019 [34]	X	X	Indoor robotics
Sell & Coupland 2015 [35]	X	X	Classification
Fouladvand et al., 2015 [36]	X	X	Indoor robotics
Chou & Juang 2018 [37]	V	X	Indoor robotics
Ferdaus et al., 2019a [38]	V	X	Aerial vehicles
Ferdaus et al., 2019b [39]	X	X	Aerial vehicles
Ferdaus et al., 2018 [40]	V	X	Aerial vehicles
Current study	V	V	Offroad robotics

summarizes the positioning results according to the main research attributes as well as the application focus of the related studies. In the table, X/V refers to non-existing/existing features, respectively.

3. Problem Description and Solution Approach

This paper deals with using transfer optimization to solve a target problem, which is defined as a set of SOPs in Section 3.1. In addition to the target problem definition, this section describes the proposed research methodology. The main steps of the methodology are described in Figure 1. First, starting from the left side of the figure, several MOPs are generated, which serve as source problems. Next, each source MOP is solved separately using MOEA, as described in Section 3.2. Solving each of the source problems results in a set of non-dominated solutions. Then, the obtained solutions are used to form an initial population, which is to be used for solving the target problem. This step, which is the knowledge extraction step, is detailed in Section 3.3. In this study, we examine two alternatives for generating the initial population. Finally, each of the extracted populations is used as an initial population to solve the target problem. To evaluate the obtained results, the target problem is also solved using an initial random population. In this study, the well-known NSGA-II [13] is used to solve both source and target problems. However, it should be noted that many alternative MOEAs can be used. In case the considered MOPs would involve a large number of objectives, then it is recommended to use an algorithm that was designed to cope with more than three objectives (see [41]).

3.1. Target Meta-Problem

The considered target problem is a multi-objective fuzzy control meta-problem in which each objective is associated with finding an optimal controller for a specific task. This meta-problem is defined as follows. Let $P_T=\left\{P_T^1,P_T^2,\dots ,P_T^{N_T}\right\}$ be a given set of $N_T$ fuzzy control problems. Then, the target meta-problem is formulated as follows:

$$\underset{\mathit{a}\in X_T}{\min }\left[f\left(P_T^1;\mathit{a}\right),f\left(P_T^2;\mathit{a}\right),\dots ,f\left(P_T^{N_T};\mathit{a}\right)\right]$$

(3)

where $X_T$ is the feasible set of fuzzy controllers for the target meta-problem, and $f$ is a scalar objective function that is used to evaluate the controllers’ performance. The main goal is to search for a set of non-specialized controllers, i.e., controllers that evolved to provide a successful behavior in all of the given set of control problems, following the approach in [11].

3.2. Source Problems and Their Solutions

In this study, each source problem is formulated as a MOP. Let $P_s=\left\{P_s^1,P_s^2,\dots ,P_s^{N_s}\right\}$ be a given set of $N_s$ source fuzzy control problems. Each of these problems is formulated as follows:

$$\underset{\mathit{a}\in X_{\mathit{si}}}{\min }\left[f_1\left(P_s^i;\mathit{a}\right),f_2\left(P_s^i;\mathit{a}\right),\dots ,f_M\left(P_s^i;\mathit{a}\right)\right]$$

(4)

where $X_{si}$ is the set of feasible fuzzy controllers of the $i^{th}$ source problem. Each fuzzy controller is evaluated using scalar objective functions, $f_1,\dots ,f_M$. Assuming no prior objective preferences, the solution of (4) results in a set of non-dominated solutions for each of the source problems.

3.3. Knowledge Extraction and Research Hypothesis

This study deals with the STO of NFCs. As mentioned in the introduction section, two types of knowledge extraction techniques are proposed and studied here. The first type involves an initial population of edge controllers, i.e., controllers whose performance vectors are located close to the edges of the obtained fronts. The second type of knowledge extraction is based on an initial population of center controllers, i.e., controllers whose performance vectors are located close to the line connecting the ideal and the nadir points. For a given front, the ideal point is a vector of the best objective values, whereas the nadir point is a vector of the worst objective values of the considered front.

This study deals with bi-objective source problems. Hence, the following presents a step-by-step procedure of the knowledge extraction approach as implemented in this study. Given a non-dominated set (NDS) of solutions and their performance vectors (PVs) of a source problem, then:

• Define the number of transferred edge-controllers ($2n_{ec}$), such that ${2n}_{ec}\ll \left|NDS\right|$
• For each edge:
  • Find the edge PV and store the associated solutions in a set of extracted edge controllers (EEC).
  • Select the $n_{ec}-1$ controllers that their PVs are the closest to the edge PV and add them to EEC.
• Define the number of transferred center controllers ($n_{cc}$), such that $n_{cc}\ll \left|NDS\right|$.
• Find the line connecting the ideal and the nadir points.
• Select the $n_{cc}$ controllers for which their PVs are the closest to the line found in 4 and store them in a set of extracted center controllers (ECC).

Repeat this process for all source problems and create the union of the EECs (UEECs) and the union of the ECCs (UECCs). Each of these unions contains extracted knowledge as needed to investigate the suggested hypotheses, as presented below.

Figure 2 illustrates these two types of controllers. In the figure, an example of normalized PVs is shown and marked as dots. Assuming that ${2n}_{ec}=n_{cc}=4$, the PVs of the EECs are marked by circles, whereas squares mark the PVs of ECCs.

The main research question that this study aims to solve is which of the proposed two types of knowledge extraction techniques is preferred from a transferability viewpoint. It is hypothesized that the edge controllers are preferred over the center ones. This hypothesis is intuitively justified by the expected higher diversity of the edge controllers. In addition, a secondary hypothesis is examined, which suggests that each of the proposed types of knowledge extraction techniques is better than the case of using random initial population, i.e., the case of no knowledge transfer.

4. Numerical Study

This section is organized as follows. First, Section 4.1 provides a detailed description of the source and target problems. Section 4.2 describes the applied NFC. Then, the comparison methods are given in Section 4.3. Next, in Section 4.4, the experimental setup is described. Finally, the obtained results and their analysis are provided in Section 4.5.

4.1. Source and Target Problems

The considered numerical study is based on simulations of a theoretical cart and environment interaction model. The study involves a set of constrained motion source control problems of a point-mass cart, and an additional set of target scenarios, as presented in the following. In this study, the cart is assumed to travel horizontally along a linear path. The linear path is defined by start and goal points. Each path is characterized by a friction function and a speed limitation function. These functions could be viewed as representing various types of motion difficulties as expected in rugged terrains. The equation of motion of the considered cart model and the involved constraints are:

$$\left\{\begin{array}{l}\ddot{x}\left(t\right)=-\frac{c\left(x\right)}{m}\dot{x}\left(t\right)+\frac{1}{m}U\left(t\right)\\ s.t.\begin{array}{c}\left|\dot{x}\left(t\right)\right|\le V_O\left(x\left(t\right)\right)\le V_{\mathrm{m}\mathrm{a}\mathrm{x}}\\ \left|U\left(t\right)\right|\le U_{\mathrm{m}\mathrm{a}\mathrm{x}}\end{array}\end{array}\right.$$

(5)

where $x\left(t\right)$ is the position of the cart, $m$ is the mass of the cart, $0\le c\left(x\right)\le 1$ is a damping coefficient as a function of the position $x\left(t\right)$, $U\left(t\right)$ is the control force, $V_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the speed limit of the cart, and $U_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the saturation value of the control force. In addition, the path imposes a speed constraint function $V_O\left(x\right)$. An interaction between the cart and a given path is termed here as a scenario.

Both the target and the source optimization problems, which are defined in Equations (6) and (7), use the following two cost functions. The first function reflects a desire to find NFCs that bring the cart from the initial position $x_0$ to the goal point $x_g$ as fast as possible, whereas the second one corresponds to the case of safe motion behavior. These are defined as follows:

$$J_{fast}=1-\frac{T_{\mathrm{m}\mathrm{i}\mathrm{n}}}{T_g}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{T}_{\mathrm{m}\mathrm{i}\mathrm{n}}=\frac{\left|x_g-x_0\right|}{V_{\mathrm{m}\mathrm{a}\mathrm{x}}}$$

(6)

$$J_{safe}=\underset{0}{\overset{T_g}{\int }}\left|c\left(x\left(t\right)\right)·\dot{x}\left(t\right)\right|dt$$

(7)

where $T_g$ is the time it takes to reach the goal point, using the considered NFC, and $T_{\mathrm{m}\mathrm{i}\mathrm{n}}$ is the time it would take to reach the goal if the entire motion involves the maximal speed limit. During the evolutionary process, a controller is considered numerically feasible if the following constraints are met, where $T_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the simulation time limit.

$$\begin{array}{c}\exists T_g<T_{\mathrm{m}\mathrm{a}\mathrm{x}}s.t\begin{array}{c}x\left(T_g\right)=x_g\end{array}\\ \left|\dot{x}\left(t\right)\right|<V_O\left(x\left(t\right)\right)\forall t<T_{\mathrm{m}\mathrm{a}\mathrm{x}}\end{array}$$

(8)

Figure 3 presents the $c\left(x\right)$ and $V_O\left(x\right)$ functions that are used in the source problems scenarios, whereas Figure 4 and Figure 5 present them for the first and the second set of the target problems, respectively. In these figures, the solid and the dashed curves present $c\left(x\right)$ and $V_O\left(x\right)$, respectively. It should be noted that the functions used in the source and target problems were randomly created. In particular, for each $c\left(x\right)$, a random number of Gaussian functions was selected, and their parameters were also randomly selected. Any Gaussian that exceeded the value of 0.8 was truncated. The $V_O\left(x\right)$ functions were created as a function of the resulting $c\left(x\right)$ functions.

In this study, each of the target sets is composed of three different scenarios, as given in Figure 4 and Figure 5. For each target set, three alternative MOPs are examined. These are defined as follows:

$$\begin{array}{c}MO{P}_1:\underset{\mathit{a}\in X_T}{\min }\left[J_{fast}\left(P_T^1;\mathit{a}\right)+J_{safe}\left(P_T^1;\mathit{a}\right),J_{fast}\left(P_T^2;\mathit{a}\right)+J_{safe}\left(P_T^2;\mathit{a}\right),J_{fast}\left(P_T^3;\mathit{a}\right)+J_{safe}\left(P_T^3;\mathit{a}\right)\right]\\ MO{P}_2:\underset{\mathit{a}\in X_T}{\min }\left[J_{fast}\left(P_T^1;\mathit{a}\right),J_{fast}\left(P_T^2;\mathit{a}\right),J_{fast}\left(P_T^3;\mathit{a}\right)\right]\\ MO{P}_3:\underset{\mathit{a}\in X_T}{\min }\left[J_{safe}\left(P_T^1;\mathit{a}\right),J_{safe}\left(P_T^2;\mathit{a}\right),J_{safe}\left(P_T^3;\mathit{a}\right)\right]\end{array}$$

(9)

The three alternative MOPs are defined in accordance with the knowledge extraction techniques (see Section 3.3). $MO{P}_1$ corresponds to the center controllers, whereas $MO{P}_2$ and $MO{P}_3$ correspond to the edge controllers, i.e., to the fast and safe controllers, respectively. In total, we examine six target problems, i.e., three alternative MOPs, each with two target sets. These three MOP definitions are expected to encourage three types of behaviors. Namely, $MO{P}_2$ and $MO{P}_3$ are expected to encourage courageous behavior and cautious behavior, respectively, whereas $MO{P}_1$ is expected to encourage intermediate behavior. In the context of this study, it should be noted that these terminologies correspond not only to the speed but also to the level of motion difficulty as presented in $c\left(x\right)$.

The $i^{th}$ bi-objective source problem is given by:

$$\underset{\mathit{a}\in X_s^i}{\min }\left[J_{fast}\left(P_s^i;\mathit{a}\right),J_{safe}\left(P_s^i;\mathit{a}\right)\right]$$

(10)

where $X_s^i$ is the feasible set of fuzzy controllers, as described in Equation (8).

4.2. Fuzzy Controllers

The cart is controlled by a neuro-fuzzy inference system. The controller receives three inputs, including the relative location of the goal point $e\left(t\right)$, the normalized velocity of the cart $v\left(t\right)$, and an effective damping coefficient $c_{eff}\left(t\right)$. Each of these inputs is mapped into the range of [−1, 1], as given in Equations (11)–(13), respectively. The NFC maps these inputs into a force $U\left(t\right)$ to be applied to the cart, as described in Equation (5).

$$e\left(t\right)=\left\{\begin{array}{cc}1& \frac{x_g-x\left(t\right)}{x_g-x_0}>1\\ -1& \frac{x_g-x\left(t\right)}{x_g-x_0}<-1\\ \frac{x_g-x\left(t\right)}{x_g-x_0}& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(11)

$$v\left(t\right)=\frac{\dot{x}\left(t\right)}{V_{\mathrm{m}\mathrm{a}\mathrm{x}}}$$

(12)

$$c_{eff}\left(t\right)=-1+c\left(x\left(t\right)\right)+c\left(x\left(t\right)+dx\right)$$

(13)

where $dx$ is a pre-defined number, which has been set as $0.1 \mathrm{m}$.

In the current implementation, the NFC is based on zero-order Takagi-Sugeno (TS) rules [42]. The main reason for choosing TS rules, in which the rules result in crisp outputs, is that it avoids the need for defuzzification. Selecting zero-order TS rules reduces the number of search parameters for the evolutionary search; hence, it results in a major reduction in the required computational resources. Each input belongs to one of three fuzzy sets. These fuzzy sets are based on the general Gaussian membership function. This results in 27 combinations of the TS rules, where each rule is considered as follows:

$$\mathrm{I}\mathrm{f}e\left(t\right)\mathrm{i}\mathrm{s}{A}_i\mathrm{a}\mathrm{n}\mathrm{d}v\left(t\right)\mathrm{i}\mathrm{s}{B}_j\mathrm{a}\mathrm{n}\mathrm{d}{c}_{eff}\left(t\right)\mathrm{i}\mathrm{s}{C}_k\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{n}U\left(t\right)=U_f$$

(14)

where $i,j,k\in \left\{\mathrm{1,2},3\right\}$ are the indices of the inputs’ fuzzy sets $A_i,B_j,C_k$, respectively. $f\in \left\{1,\dots ,27\right\}$ is the index of the output $\left|U_f\right|\le U_{max}$. The final crisp conclusion is calculated as the average of the rule conclusions.

The decision variables of the optimization problem include the parameters of $A_i,B_j,C_k$, and $U_f$. Given that each general Gaussian function has two parameters, and the total number of rules is 27, then the total number of decision variables results in 9 (input sets) $·$ 2 (parameters) + 27 (rules) = 45 variables.

4.3. Comparison Methods

In this study, the substantiations of the research hypotheses are achieved by comparing the results of solving the target meta-problems as obtained by each of the knowledge extraction techniques and by the random initialization technique (see Section 3). In particular, two indicators are used to evaluate the transfer capabilities of the compared techniques based on the Hyper-Volume (HV) metric [43]. The first indicator, which is inspired by the Asymptotic Performance (AP) measure [44], calculates the final value of the HV for each of the compared approaches. The second indicator, which is based on the Time-to-Threshold (ToT) measure [44], is calculated as the number of generations it took HV to reach a threshold. In addition, the obtained best performance in each scenario of the target meta-problem is also compared.

Given the stochastic nature of the applied MOEA, 31 runs were conducted for each approach to make statistical inferences. As commonly done when comparing MOEAs, the medians and Inter-Quartile Ranges (IQRs) are used rather than the means and the standard deviations. For the statistical comparisons, the Wilcoxon rank-sum test was applied for each pair of compared techniques.

4.4. Experimental Setup

Here, NSGA-II [13], which is a well-known MOEA, is applied to solving the optimization problems presented in Equations (9) and (10). The run parameters that were applied for solving each of the source MOPs and the target MOPs are summarized in

Table 2. Run parameters.

Parameter	Signal Source	Target
Number of generations	$300$	$200$
Population size	$100$	$60$
Penalty value	$100$
Weight Mutation (WM) mechanism	$\mathrm{P}\mathrm{o}\mathrm{l}\mathrm{y}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{i}\mathrm{a}\mathrm{l}$
WM parameters (probability, distribution index)	$\left(\mathrm{0.2,15}\right)$
Weight Crossover (WC) mechanism	$\mathrm{S}\mathrm{B}\mathrm{X}$
WC parameters (probability, distribution index)	$\left(\mathrm{0.8,20}\right)$
Simulation time limit $T_{\mathrm{m}\mathrm{a}\mathrm{x}}$	$20\left[\mathrm{s}\right]$
Cart mass $m$	$1\left[\mathrm{k}\mathrm{g}\right]$
Force saturation value $U_{\mathrm{m}\mathrm{a}\mathrm{x}}$	$50\left[\mathrm{N}\right]$
Starting point $x_0$	$-10\left[\mathrm{m}\right]$
Starting Velocity $\dot{x}\left(0\right)$	$0[\mathrm{m}/\mathrm{s}]$
Goal point $x_g$	$0\left[\mathrm{m}\right]$
Speed limit $V_{\max }$	$5[\mathrm{m}/\mathrm{s}]$

. In this study, the applied stopping criterion of NSGA-II is the total number of generations. It should be noted that the run parameters for the source problems were selected to ensure convergence in all source problems. To deal with the constraint problem, a penalty approach is applied. Here, a non-feasible controller is penalized by adding a high penalty value, which practically serves as a death penalty. It should be noted that the population size for the target meta-problem is a result of the applied knowledge extraction technique (see Section 4.5.1).

4.5. Experimental Results and Analyses

This section, in which the results are presented, is organized as follows. First, in Section 4.5.1, typical results obtained by solving one of the ten source control problems are presented, including the knowledge extraction from the presented results. Then, the results of the target meta-problems are presented and analyzed in Section 4.5.2.

4.5.1. Demonstration of Knowledge Extraction

In this section, typical results are presented based on solving a randomly selected source control problem. Figure 6 shows the approximated Pareto front as obtained by solving the problem in Equation (10).

In the figure, each point represents the performance vector of a non-dominated NFC. As expected, the tradeoff between the objectives is evident. This tradeoff is further shown in Figure 7. In Figure 7, the behaviors of the edge controllers are presented. The left solid curves show the position and the velocity versus time for the fastest NFC, and the right solid curves show these behaviors for the safest one. The dashed curves show the corresponding damping coefficient and speed limit. It is evident from Figure 7 that the velocity, as obtained by applying the fastest NFC, quickly reaches the vicinity of the speed limit. In this case, the goal point is reached in less than 2.5 s. In contrast, the application of the safest NFC results in a much lower velocity, and the goal-point is reached after about twenty seconds.

The convergence curve of the run of the presented case is shown in Figure 8. This figure shows the HV measure at each generation. It is suggested from the figure that convergence is reached.

Following the method presented in Section 3.3, the knowledge extraction from the results of solving the presented source problem is presented in Figure 9. In the left plot, the extraction of center controllers, which are marked as squares, is presented. The circles in the right plot present the selected edge-controllers. It should be noted that for this demonstration, six controllers were selected from each source problem for each of the controller types.

Given that ten source problems are used in this study, and that each of these problems provides six edge controllers and six center controllers, then the population size is sixty for solving each of the corresponding meta-problems.

4.5.2. Results and Analysis of the Target Meta-Problems

Table 3. Transfer capabilities comparison—target set 1.

Problem	Indicator	Random	Center	Edge
$MO{P}_1$ Target set 1	ToT	33 (±20)	12 (±7.5)	3 (±2)
	AP	0.88124 (±0.0601)	0.91951 (±0.0314)	0.93812 (±0.0107)
	Best-F1	0.00829 (±0.0079)	0.00230 (±0.0018)	0.00355 (±0.0011)
	Best-F2	0.01216 (±0.0098)	0.00258 (±0.0012)	0.00127 (±0.0001)
	Best-F3	0.00546 (±0.0022)	0.00493 (±0.0020)	0.00375 (±0.0010)
$MO{P}_2$ Target set 1	ToT	24 (±197)	2 (±2)	2 (±149)
	AP	0.90159 (±0.944)	0.95214 (±0.094)	0.96308 (±0.730)
	Best-F1	0.03682 (±1.071)	0.02305 (±0.023)	0.01784 (±0.816)
	Best-F2	0.01970 (±1.094)	0.00620 (±0.037)	0.00387 (±0.825)
	Best-F3	0.02136 (±1.087)	0.01029 (±0.018)	0.00849 (±0.820)
$MO{P}_3$ Target set 1	ToT	43 (±150)	8 (±148)	5 (±149)
	AP	0.75407 (±0.589)	0.79441 (±0.640)	0.85067 (±0.643)
	Best-F1	0.00830 (±0.063)	0.00425 (±0.183)	0.00596 (±0.085)
	Best-F2	0.01917 (±0.067)	0.00570 (±0.210)	0.00183 (±0.096)
	Best-F3	0.00492 (±0.032)	0.01093 (±0.466)	0.00799 (±0.048)

and

Table 4. Transfer capabilities comparison—target set 2.

Problem	Indicator	Random	Center	Edge
$MO{P}_1$ Target set 2	ToT	24 (±197)	2 (±2)	2 (±149)
	AP	0.21958 (±0.387)	0.67058 (±0.236)	0.70545 (±0.116)
	Best-F1	0.00568 (±0.007)	0.00258 (±0.003)	0.00081 (±0.001)
	Best-F2	0.00480 (±0.003)	0.00135 (±0.001)	0.00151 (±0.001)
	Best-F3	0.00369 (±0.005)	0.00220 (±0.002)	0.00082 (±0.001)
$MO{P}_2$ Target set 2	ToT	4 (±163)	1 (±1)	2 (±150)
	AP	0.95268 (±0.780)	0.96454 (±0.077)	0.97688 (±0.770)
	Best-F1	0.00818 (±0.836)	0.00625 (±0.020)	0.00398 (±0.831)
	Best-F2	0.01925 (±0.828)	0.01686 (±0.014)	0.01099 (±0.831)
	Best-F3	0.00772 (±0.836)	0.00648 (±0.0236)	0.00475 (±0.830)
$MO{P}_3$ Target set 2	ToT	129 (±122)	15 (±9)	13 (±9)
	AP	0.61719 (±0.178)	0.73587 (±0.080)	0.73441 (±0.057)
	Best-F1	0.00525 (±0.003)	0.00293 (±0.002)	0.00093 (±0.001)
	Best-F2	0.00571 (±0.003)	0.00374 (±0.002)	0.00462 (±0.001)
	Best-F3	0.00445 (±0.004)	0.00246 (±0.001)	0.00145 (±0.001)

summarize the results of the ToT, AP, and the best performances in each of the objectives for target sets 1 and 2, respectively. For each method, the tables outline the medians and IQRs of the indicators. In the tables, the superior values are marked in bold numbers. In general, better medians are obtained when using the edge controller for the initialization as compared to either using the center controllers or the random controllers. In addition, better results were obtained by the center-controllers than with the random ones.

The results of the rank-sum Wilcoxon test are presented in

Table 5. Statistical comparisons.

Indicator	Edge Better Than Center	Edge Better Than Random	Center Better Than Random
ToT (+/−/≈)	3/1/2	5/0/1	6/0/0
AP (+/−/≈)	5/0/1	5/0/1	4/1/1
Best-F1 (+/−/≈)	4/2/0	6/0/0	5/0/1
Best-F2 (+/−/≈)	3/1/2	5/0/1	5/0/1
Best-F3 (+/−/≈)	5/0/1	5/0/1	5/0/1
Total (+/−/≈)	20/4/6	26/0/4	25/1/4

, which allows statistical inference. In the table, + is assigned for comparison in which the statement is statistically correct, − is assigned for cases where the opposite statement is statically correct, and ≈ is assigned for cases where the obtained p-values are larger than the significance level, i.e., the compared cases are statistically equivalent. For example, the entry 3/1/2 in the 1st row and column of the table means that the statement that using the edge controllers was better than using the center ones was found to be correct in three out of the six meta problems, etc. The last row of the table provides the accumulated numbers with respect to all the indicators.

As shown in the table, when comparing initialization using the edge controllers, as opposed to using the center controllers, there are 20 comparisons in which using the edge controllers was better than using the center controllers. The opposite statement was found to be correct in 4 comparisons and no statistical superiority was found in 6 of the 30 comparisons. These observations suggest that, in general, edge controllers are better than the center-controllers for the considered meta-problems in terms of the obtained values of most indicators. These findings substantiate the first hypothesis of this study.

When comparing initialization by the edge controllers with that of the random controllers, the accumulated results show that there was no comparison in which the latter controllers were superior. When comparing initialization by the center controllers with that of the random controllers, there was only one case out of the 30 in which the random ones were superior. These findings substantiate the second hypothesis of this study.

4.5.3. Detailed Demonstration of the Transferability Results

The following presents some details of the results and analysis for the case of $MO{P}_1$ with the first target set. The boxplots in Figure 10 show the statistical results of the HV metric, as obtained by each initialization. The left plot in the figure presents the results achieved by the edge controller initialization, the results of center controller initialization are presented in the center plot, and the right plot shows the result of the random initialization. The results of this demonstration suggest that the best results are achieved by the edge controller initialization. It can be observed that the edge results are better in terms of higher median values as well as lower variance and outliers. It is also evident that the center controller initialization is superior to those achieved when a random initialization is used.

To make statistical inferences, the rank-sum test, with a significance level of 0.05, is applied for each comparison.

Table 6. Statistical tests (p-values).

Indicator	Center vs. Edge	Random vs. Edge	Random vs. Center
ToT	2 × 10−11	1 × 10−11	5 × 10−9
AP	8 × 10−5	3 × 10−10	4 × 10−6
Best-F1	7 × 10−4	7 × 10−9	4 × 10−9
Best-F2	1 × 10−7	1 × 10−11	9 × 10−11
Best-F3	7 × 10−5	2 × 10−4	4 × 10−1

outlines p-values as obtained by the test. Bold results represent comparisons where initialization by the edge controllers is statistically superior, whereas underlined results are for comparisons where initialization by the center controllers is superior.

The obtained results show that edge controllers are better than the center controllers for the considered meta-problem in terms of the obtained values of most indicators. In addition, the results indicate that both the center and edge controllers are better than the random ones for the considered problem. These findings substantiate the two hypotheses of this study (see Section 3.3).

4.5.4. Demonstration of the Obtained Behaviors

This section provides discussion and analysis regarding the obtained behaviors by solving each of the target meta-problems, as defined in Section 4.1. Figure 11 and Figure 12 present the behaviors of randomly selected NFCs as obtained by solving the three MOPs for target set 1 and target set 2, respectively. Each row of the figures provides the results for a specific scenario of the target sets as obtained by the three MOPs. The solid curve in each of the figures shows the velocity versus time, whereas the dashed curves show the corresponding speed limit.

Considering the results of $MO{P}_2$ (in the middle panels), as compared with those of solving $MO{P}_1$ and $MO{P}_3$, it is observed that the obtained NFCs provide higher velocities without any violation of the speed limits. As expected from the definition of $MO{P}_2$, its solutions exhibit safe yet courageous behaviors. Furthermore, when comparing the results of solving $MO{P}_1$ and $MO{P}_3$, it can be observed that the solutions of $MO{P}_3$ resulted in slower motions. As expected from the definitions of these two problems, $MO{P}_3$ exhibits the most cautious behaviors, while $MO{P}_1$ exhibits intermediate behaviors.

The following provides a statistical comparison between the NFCs of $MO{P}_2$ and those of $MO{P}_3$, which aims to highlight the clear difference between cautious and courageous behaviors. In particular, the presented statistics are obtained for the case of using initialization by the edge NFCs, using 31 runs per each of the MOPs as applied per each scenario of each of the target sets. The statistics are provided with respect to an average-time to reach the goal point at each of the presented scenarios.

Table 7. Behavior comparison of time to goal point.

Target Set	Scenario	Edge Initialization
		$\mathit{M}\mathit{O}{\mathit{P}}_2$	$\mathit{M}\mathit{O}{\mathit{P}}_3$
1	1	8.140 (±0.66)	15.73 (±4.70)
	2	18.70 (±1.09)	19.97 (±2.40)
	3	9.650 (±0.80)	16.70 (±4.28)
2	1	17.45 (±0.52)	19.02 (±0.32)
	2	15.35 (±0.35)	17.30 (±0.60)
	3	19.95 (±0.06)	19.99 (±0.02)
(+/−/≈)			(6/0/0)

summarizes the medians and IQRs of the average times. Each of these averages is calculated as follows. For each of the runs, the average is taken over the entire set of the obtained approximated Pareto optimal NFCs (of each MOP and target set). To obtain this average time, first, each of these NFCs of the run is applied to the considered scenario to find its resulting time to reach the goal point. Next, these times are used to find the average time for the run of the considered scenario. In addition, the last row of Table 7 provides the accumulated results of the Wilcoxon rank-sum tests over the six scenarios. As expected, these results clearly show that the average times as obtained using the definition of $MO{P}_2$ are statistically shorter than those obtained by $MO{P}_3$.

In summary, the time-to-goal results in Table 7 further emphasize the differences between the outcomes of courageous and cautious behaviors. While not presented here, similar conclusions about the time to the goal point were obtained not just for the initialization by the edge controllers but also by the other two types, including initialization by center controllers and by random controllers. Namely, each of the initialization types resulted in the expected behavior as associated with the considered MOP.

5. Summary and Conclusions

This study concerns the evolutionary search for generic neuro-fuzzy motion controllers in conjunction with the use of sequential transfer optimization. The search aims to evolve near optimal neuro-fuzzy controllers that can operate in a variety of scenarios. Three types of target meta-problems are defined, which differ by the expected motion behavior, including cautious, intermediate, and courageous behaviors. Each of these meta-problems is defined as a multi-objective problem, where each objective is associated with optimizing the performance in a particular scenario.

A unique kind of sequential transfer optimization is proposed that involves multiple source problems, where each of the source problems is defined and solved as a bi-objective problem. The use of multiple source problems aims to diversify the source solutions with respect to scenarios, whereas the use of the two objectives aims to diversify the source solutions with respect to the obtained behaviors. This kind of multiple-source problem raises the dilemma of which of the source solutions should be transferred to solve the target problems. The main research hypothesis is that solutions with performances near the edges of the obtained approximated bi-objective Pareto fronts provide better transferability as compared with those that are associated with performances near the center of the obtained fronts.

Given the stochastic nature of the applied search technique, a statistical numerical study was conducted. First, ten source problems were defined and solved. Next, a total of six target meta-problems were defined. Each of these target problems was solved using three types of initializations including random initializations and two additional types in accordance with the aforementioned hypothesis.

The main contribution of this study is that it substantiates the hypothesis that edge controllers should be preferred over the center ones. This has a pragmatic impact on applying sequential transfer optimization to the development of neuro-fuzzy controllers for the considered motion control type of problems. This is because the proposed approach produces useful information out of the large and diverse knowledge that is created by solving many Pareto-based source problems. As shown in this study, and similar studies, using selected solutions from source problems to serve as an initial population to solve a target problem is much more efficient as compared with the traditional use of a random initial population. In addition, this study shows that the proposed approach is useful for finding neuro-fuzzy controllers that produce different motion behaviors when operating in various environments with different motion difficulties.

This study assumed that experts select the source problems based on their cognitive abilities to choose relevant problems to the considered target problems. It is suggested to investigate in the future the influence of restricting the transferred solutions based on similarity measures among tasks (e.g., [8]). It should be noted that the current study defines and substantiates the considered hypotheses for bi-objective source problems. Future work should also deal with extending the hypotheses to source problems with more than two objectives. In particular, the definition of edge controller should be revised and investigated.