Open Access

Tobias Thummerer, Lars Mikelsons

• Using neural ordinary differential equations (ODEs) to model complex systems is still a challenging venture and failing for various reasons, often resulting in convergence to unsatisfactory local minima or unintended termination of the training process because of solver issues. To take a step back, the root of the problem lies in either the complexity of the optimization problem or the lack of data. The negative effects of both aspects can be reduced by incorporating a priori knowledge, a common strategy in Scientific Machine Learning. Especially when modeling physical systems, there is almost always more system knowledge available than is used for training. Examples range from knowledge of generic properties such as “the system is stable” to more quantifiable attributes such as “the system oscillates in a known frequency range”. To close the loop, such knowledge can be used to achieve faster convergence and better generalization. In this paper, we focus specifically on systemUsing neural ordinary differential equations (ODEs) to model complex systems is still a challenging venture and failing for various reasons, often resulting in convergence to unsatisfactory local minima or unintended termination of the training process because of solver issues. To take a step back, the root of the problem lies in either the complexity of the optimization problem or the lack of data. The negative effects of both aspects can be reduced by incorporating a priori knowledge, a common strategy in Scientific Machine Learning. Especially when modeling physical systems, there is almost always more system knowledge available than is used for training. Examples range from knowledge of generic properties such as “the system is stable” to more quantifiable attributes such as “the system oscillates in a known frequency range”. To close the loop, such knowledge can be used to achieve faster convergence and better generalization. In this paper, we focus specifically on system properties that can be expressed based on eigenvalues, such as (partly) stability, oscillation capability, frequencies, damping, and stiffness. We explain how such properties can be intuitively integrated into training neural ODEs, provide open-source software for this, and finally show in three academic examples that such eigen-informed neural ODEs are able to converge in fewer steps (median up to factor ), generalize better (median up to factor ) and are solvable more efficiently (median up to factor ) compared to pure neural ODEs, and can even reconstruct a given system on undersampled data.…