Open Access

The electrification of transportation requires the development of smart charging management systems for electric vehicles to optimize grid performance and enhance user satisfaction. However, existing methods often reduce multi-objective problems to single-objective formulations, limiting their ability to balance conflicting objectives and requiring iterative runs for diverse solutions. In this study, we propose a Multi-Objective Evolutionary Reinforcement Learning (MOEvoRL) framework to optimize electric vehicle charging strategies and discover multiple policies within a single training run. Our approach focuses on maximizing the batteries’ state of charge, increasing photovoltaic power consumption, reducing peak loads, and smoothing the overall load on the grid. Simultaneously, it adheres to essential grid constraints, such as load balancing and grid connection node limits, to ensure grid stability, efficiency, and real-world applicability. MOEvoRL utilizes the exploratory power of Evolutionary Algorithms and the sequential decision-making strengths of Reinforcement Learning. By employing neuroevolution, we optimize the weights and topologies of policy networks. Our approach employs the Non-dominated Sorting Genetic Algorithm II, Strength Pareto Evolutionary Algorithm 2, and a modified NeuroEvolution of Augmenting Topologies as optimizers and benchmarks their performance against the gradient-based Multi-Objective Deep Deterministic Policy Gradient (MODDPG) and a single-objective DDPG that simplifies multiple objectives into a single objective using a linear scalarization function. The results show that MOEvoRL approaches are superior to MODDPG in terms of generalization, robustness, constraint compliance, and multi-objective optimization capabilities. In contrast, DDPG exhibits poor and unstable performance. This positions MOEvoRL as a robust strategy for managing electric vehicle charging while optimizing local grid loads.

This paper presents MAHF, a software framework for the highly flexible construction of metaheuristics from individual components and the subsequent evaluation of these algorithms. At that, MAHF is developed specifically for the experimental analysis of the algorithmic behaviour during the optimization process with a focus on the influences of the algorithm’s components. Furthermore, uncommon and incompletely examined operators or frameworks of “novel” metaheuristics are included as well, so that their usefulness can be assessed. In the following, we will elaborate on MAHF’s structure and its general goals and application possibilities. Concerning MAHF’s component structure, we will provide examples of its usage and extension to ensure that it is reusable by others as well.

The Machine Health (MH) sector—which includes, for example, Predictive Maintenance, Prognostics and Health Management, and Condition Monitoring—has the potential to improve efficiency and reduce costs for maintenance and machine operation. This is achieved by data-driven analytics applications, utilising the vast amount of data collected by sensors during machine runtime. While there are numerous possible fields of application, the overall complexity of machines and applications in scientific publications is still low, preventing MH technologies from being implemented in many real-world scenarios. This may be the result of a diffuse understanding of the term complexity in the publications of this field, which results in a lack of focus towards the core problems of real-world MH applications. This article introduces a new way of discerning complexity in data-driven MH applications, enabling an effective discussion and analysis of present and future MH applications. This is achieved by creating a new taxonomy based on observations from relevant literature and substantial domain knowledge. Using this newly introduced taxonomy, we categorise recent applications of MH to demonstrate the usefulness of our approach and illustrate a still-prevalent research gap based on our findings.

To fill the increasing demand for explanations of decisions made by automated prediction systems, machine learning (ML) techniques that produce inherently transparent models are directly suited. Learning Classifier Systems (LCSs), a family of rule-based learners, produce transparent models by design. However, the usefulness of such models, both for predictions and analyses, heavily depends on the placement and selection of rules (combined constituting the ML task of model selection). In this paper, we investigate a variety of techniques to efficiently place good rules within the search space based on their local prediction errors as well as their generality. This investigation is done within a specific LCS, named SupRB, where the placement of rules and the selection of good subsets of rules are strictly separated in contrast to other LCSs where these tasks sometimes blend. We compare a Random Search, (1,λ)-ES and three Novelty Search variants. We find that there is a definitive need to guide the search based on some sensible criteria, i.e. error and generality, rather than just placing rules randomly and selecting better performing ones but also find that Novelty Search variants do not beat the easier to understand (1,λ)-ES.

Analysing the behaviour of metaheuristics comprehensively and thereby enhancing explainability requires large empirical studies. However, the amount of data gathered in such experiments is often too large to be examined and evaluated visually. This necessitates establishing more efficient analysis procedures, but care has to be taken so that these do not obscure important information. This paper examines the suitability of clustering methods to assist in the characterisation of the behaviour of metaheuristics. The convergence behaviour is used as an example as its empirical analysis often requires looking at convergence curve plots, which is extremely tedious for large algorithmic datasets. We used the well-known K-Means clustering method and examined the results for different cluster sizes. Furthermore, we evaluated the clusters with respect to the characteristics they utilise and compared those with characteristics applied when a researcher inspects convergence curve plots. We found that clustering is a suitable technique to assist in the analysis of convergence behaviour, as the clusters strongly correspond to the grouping that would be done by a researcher, though the procedure still requires background knowledge to determine an adequate number of clusters. Overall, this enables us to inspect only few curves per cluster instead of all individual curves.

With the increasing capabilities of machine learning (ML) and other artificial intelligence (AI) methods comes a growing interest from many fields of application to employ these methods to increase automation of work tasks and improve the efficiency and effectiveness of operations. However, the systems will only see effective use if they are trusted by those responsible for the task itself. False predictions and flawed decisions can have detrimental effects, for example, on human life in medical applications or financial interest in industry. Therefore, it is reasonable for stakeholders of these systems to want to understand the reasoning of AI systems. Methods that can make this insight available to stakeholders have increasingly be summarized under the term explainable AI (XAI). While approaches exist towards making black-box models explainable, the use of inherently explainable models can be more straightforward and promising. One family of algorithms producing inherently explainable models are. Learning Classifier Systems (LCSs). Despite their name, they are a general rule-based ML (RBML) method and representatives for all major ML tasks have been proposed. To classify LCSs based on their mode of operation, this work introduces a new system that is more precise than the current stateof- the-art and based on descriptive ML terminology. While most researchers in the past have focused primarily on LCSs’ algorithmic aspects, this work adopts a distinct perspective by approaching them through the lens of optimization. It discusses LCSs with regards to typical tasks involved in creating an ML model and what specific elements have to be optimized and how this is typically done. Critically, the task of model selection is usually performed by some metaheuristic component and involves the subtasks of how many rules to use and where to place them. This work also proposes a template to assess use case–specific explainability requirements based on multiple stakeholders’ inputs and extensively demonstrates its usage in a real-world manufacturing setting. There, stakeholders indeed request XAI models over black-box approaches and, according to their answers, LCSs should be a good fit. Additionally, the results laid out what LCS models in that application should look like which is, however, not achievable with the major state-of-the-art LCSs. Therefore, a new LCS, called the Supervised Rule-based learning system (SupRB), is introduced in this work that is simpler than previous LCSs with clearer optimization objectives and models that can fulfil the stakeholders’ requirements. In extensive testing on real-world data, SupRB demonstrates its capabilities of producing small yet accurate models that outperform those of well-established methods. This work also investigates numerous possible extensions for each component of SupRB with a special focus on its optimizers and presents the findings of the multiple studies in a comprehensive manner based on descriptive statistics, visualizations of results, and rigorous statistical testing. Then various paths for future research and application of SupRB are laid out which can advance the field of XAI considerably.