Page 20 - Read Online

Page 2 of 16 Zander et al. Complex Eng Syst 2023;3:9 I http://dx.doi.org/10.20517/ces.2023.11

too complex for effective use in the fuzzy setting. Such applications tend to include video games and other
[5]
interactive environments .
Reinforcement learning (RL) comprises a collection of learning algorithms that allow optimization of various
[6]
controlsystems. RLtrainsagentstoadapttoagivenenvironmentviaarewardsystem . Theagenttakesaction
by assessing the state of the environment, and RL allows the agent to maximize the expected reward. Such
algorithms are well suited for complex environments; RL has shown great success in games [7] and industrial
[8]
applications .
Deep RL [9] has recently seen significant developments as agents have successfully outperformed humans in
[7]
games such as Go , Poker [10] , and video games [11] . This approach also finds effective employment in appli-
cations such as autonomous vehicles [12] , UAVs [13] , and fine-tuning of large language models [14] . However,
RL models and related deep learning architectures, in particular, tend to have a significant drawback in their
relative lack of explainability.

Explainable machine learning is a subject of extensive research [15–17] concerned with rendering relevant al-
gorithms more understandable, transparent, and trustworthy. Explainable fuzzy AI has afforded significant
developments in this domain due to characteristics such as amenability to visualization and expression in nat-
ural language [18–20] . Notably, literature discussing directed RL in the setting of explainable fuzzy AI proves
scarce despite recent advancements in the former and a capacity for mutual benefit; exploration of genetic
algorithms proves more common historically [21–23] .

To aid in developing RL algorithms capable of producing explainable models, fuzzy RL-based architectures [24]
show promise. Fuzzy Q-learning was proposed and studied in a series of research papers [25–27] primarily
concerned with performance in control applications rather than explainability in general. Additionally, these
explorations are disconnected from recent developments in deep RL. This presents a gap in the literature.
However, in [28] , the author developed Takagi-Sugeno-Kang (TSK) fuzzy systems with successful applications
in various environments where Deep Q-Learning Network (DQN) [29] has illustrated effectiveness. Similar to
an Adaptive Network-Based Fuzzy Inference System (ANFIS) [30] , this approach leverages RL to train an agent
in these environments and shows the promise of such experimentation.

This paper offers further practical study of RL-based TSK fuzzy systems and ANFIS architectures as solutions
for developing explainable AI. The latter is compared to traditional Deep Q-learning algorithms. After a pre-
liminary section where we provide an overview of the conceptual building blocks of our systems, we discuss
the successful and winning application of RL-based TSK systems to the Asteroid Smasher framework and cor-
responding 2023 Explainable AI competition [31] . Additionally, we offer a comparison of ANFIS and DQN
in the CartPole environment. The value of the relationship between RL and fuzzy systems to explainability is
highlighted and discussed in both cases.

2. PRELIMINARIES

2.1. Fuzzy systems
Fuzzy sets are defined as functions : → [0, 1] and are interpreted as sets with a continuum of membership
[1]
grades . They are used in modeling under uncertainty, especially in a rule-based environment.
Fuzzyrulesdescribeanimprecisecause-effectrelationshipbetweencertainvariablescontrollingagivensystem.
Fuzzy rules are of the form
If is then is
where and are fuzzy sets on domains and , respectively. Fuzzy rules can be organized into fuzzy rule