Open Access

Neele Kemper, Michael Heider, Dirk Pietruschka, Jörg Hähner

• The electrification of transportation requires the development of smart charging management systems for electric vehicles to optimize grid performance and enhance user satisfaction. However, existing methods often reduce multi-objective problems to single-objective formulations, limiting their ability to balance conflicting objectives and requiring iterative runs for diverse solutions. In this study, we propose a Multi-Objective Evolutionary Reinforcement Learning (MOEvoRL) framework to optimize electric vehicle charging strategies and discover multiple policies within a single training run. Our approach focuses on maximizing the batteries’ state of charge, increasing photovoltaic power consumption, reducing peak loads, and smoothing the overall load on the grid. Simultaneously, it adheres to essential grid constraints, such as load balancing and grid connection node limits, to ensure grid stability, efficiency, and real-world applicability. MOEvoRL utilizes the exploratory power ofThe electrification of transportation requires the development of smart charging management systems for electric vehicles to optimize grid performance and enhance user satisfaction. However, existing methods often reduce multi-objective problems to single-objective formulations, limiting their ability to balance conflicting objectives and requiring iterative runs for diverse solutions. In this study, we propose a Multi-Objective Evolutionary Reinforcement Learning (MOEvoRL) framework to optimize electric vehicle charging strategies and discover multiple policies within a single training run. Our approach focuses on maximizing the batteries’ state of charge, increasing photovoltaic power consumption, reducing peak loads, and smoothing the overall load on the grid. Simultaneously, it adheres to essential grid constraints, such as load balancing and grid connection node limits, to ensure grid stability, efficiency, and real-world applicability. MOEvoRL utilizes the exploratory power of Evolutionary Algorithms and the sequential decision-making strengths of Reinforcement Learning. By employing neuroevolution, we optimize the weights and topologies of policy networks. Our approach employs the Non-dominated Sorting Genetic Algorithm II, Strength Pareto Evolutionary Algorithm 2, and a modified NeuroEvolution of Augmenting Topologies as optimizers and benchmarks their performance against the gradient-based Multi-Objective Deep Deterministic Policy Gradient (MODDPG) and a single-objective DDPG that simplifies multiple objectives into a single objective using a linear scalarization function. The results show that MOEvoRL approaches are superior to MODDPG in terms of generalization, robustness, constraint compliance, and multi-objective optimization capabilities. In contrast, DDPG exhibits poor and unstable performance. This positions MOEvoRL as a robust strategy for managing electric vehicle charging while optimizing local grid loads.…