Open Access

The electrification of transportation requires the development of smart charging management systems for electric vehicles to optimize grid performance and enhance user satisfaction. However, existing methods often reduce multi-objective problems to single-objective formulations, limiting their ability to balance conflicting objectives and requiring iterative runs for diverse solutions. In this study, we propose a Multi-Objective Evolutionary Reinforcement Learning (MOEvoRL) framework to optimize electric vehicle charging strategies and discover multiple policies within a single training run. Our approach focuses on maximizing the batteries’ state of charge, increasing photovoltaic power consumption, reducing peak loads, and smoothing the overall load on the grid. Simultaneously, it adheres to essential grid constraints, such as load balancing and grid connection node limits, to ensure grid stability, efficiency, and real-world applicability. MOEvoRL utilizes the exploratory power of Evolutionary Algorithms and the sequential decision-making strengths of Reinforcement Learning. By employing neuroevolution, we optimize the weights and topologies of policy networks. Our approach employs the Non-dominated Sorting Genetic Algorithm II, Strength Pareto Evolutionary Algorithm 2, and a modified NeuroEvolution of Augmenting Topologies as optimizers and benchmarks their performance against the gradient-based Multi-Objective Deep Deterministic Policy Gradient (MODDPG) and a single-objective DDPG that simplifies multiple objectives into a single objective using a linear scalarization function. The results show that MOEvoRL approaches are superior to MODDPG in terms of generalization, robustness, constraint compliance, and multi-objective optimization capabilities. In contrast, DDPG exhibits poor and unstable performance. This positions MOEvoRL as a robust strategy for managing electric vehicle charging while optimizing local grid loads.

Robustness against internal or external disturbances is a key competence of Organic Computing Systems. Hereby, a rarely discussed aspect are physical disturbances, therefore, failures or breakdowns that affect a systems physical components. Before experiencing such a disturbance, physical components may show various measurable signs of deterioration that might be assessed through sensor data. If interpreted correctly, it would be possible to predict future physical disturbances and act appropriately in order to prevent them from possibly harming the overall system. As the actual structure of such data as well as the behaviour that disturbances produce might not be known a priori, it is of interest to equip Organic Computing Systems with the ability to learn to predict them autonomously. We utilize the Automated Machine Learning Framework TPOT for an online-learning-inspired methodology for learning to predict physical disturbances in an iterative manner. We evaluate our approach using a freely available dataset from the broader domain of Predictive Maintenance research and show that our approach is able to build predictors with reasonable prediction quality autonomously.

In sociotechnical settings, human operators are increasingly assisted by decision support systems. By employing such systems, important properties of sociotechnical systems, such as self-adaptation and self-optimization, are expected to improve further. To be accepted by and engage efficiently with operators, decision support systems need to be able to provide explanations regarding the reasoning behind specific decisions. In this article, we propose the use of learning classifier systems (LCSs), a family of rule-based machine learning methods, to facilitate and highlight techniques to improve transparent decision-making. Furthermore, we present a novel approach to assessing application-specific explainability needs for the design of LCS models. For this, we propose an application-independent template of seven questions. We demonstrate the approach’s use in an interview-based case study for a manufacturing scenario. We find that the answers received do yield useful insights for a well-designed LCS model and requirements for stakeholders to engage actively with an intelligent agent.

A large proportion of the energy consumed by private households is used for space heating and domestic hot water. In the context of the energy transition, the predominant aim is to reduce this consumption. In addition to implementing better energy standards in new buildings and refurbishing old buildings, intelligent energy management concepts can also contribute by operating heat generators according to demand based on an expected heat requirement. This requires forecasting models for heat demand to be as accurate and reliable as possible. In this paper, we present a case study of a newly built medium-sized living quarter in central Europe made up of 66 residential units from which we gathered consumption data for almost two years. Based on this data, we investigate the possibility of forecasting heat demand using a variety of time series models and offline and online machine learning (ML) techniques in a standard data science approach. We chose to analyze different modeling techniques as they can be used in different settings, where time series models require no additional data, offline ML needs a lot of data gathered up front, and online ML could be deployed from day one. A special focus lies on peak demand and outlier forecasting, as well as investigations into seasonal expert models. We also highlight the computational expense and explainability characteristics of the used models. We compare the used methods with naive models as well as each other, finding that time series models, as well as online ML, do not yield promising results. Accordingly, we will deploy one of the offline ML models in our real-world energy management system in the near future.