Open Access

Sara Garber, Yarema Okhrin

• Background While machine learning (ML) models are increasingly used to predict outcomes in health care, their practical effect on health care operations, such as bed capacity management, remains underexplored. There is a variety of traditionally used evaluation metrics to analyze ML models; however, decision makers in health care settings require a deeper understanding of their implications for resource management. Traditional performance measures often fail to provide this practical insight. Methods In this work, we conduct a simulation study to evaluate the impact of ML-driven length-of-stay (LOS) predictions on intensive care unit (ICU) bed capacity management. Two classification models differing in terms of explainability and interpretability, logistic regression (LR) and extreme gradient boosting (XGB), are applied to predict ICU-LOS. We use the HiRID dataset containing high-frequency data of more than 33,000 patients. The predictions of the ML models are integrated into aBackground While machine learning (ML) models are increasingly used to predict outcomes in health care, their practical effect on health care operations, such as bed capacity management, remains underexplored. There is a variety of traditionally used evaluation metrics to analyze ML models; however, decision makers in health care settings require a deeper understanding of their implications for resource management. Traditional performance measures often fail to provide this practical insight. Methods In this work, we conduct a simulation study to evaluate the impact of ML-driven length-of-stay (LOS) predictions on intensive care unit (ICU) bed capacity management. Two classification models differing in terms of explainability and interpretability, logistic regression (LR) and extreme gradient boosting (XGB), are applied to predict ICU-LOS. We use the HiRID dataset containing high-frequency data of more than 33,000 patients. The predictions of the ML models are integrated into a simulation framework that replicates real-world ICU bed management, allowing for the assessment of the practical implications of using these algorithms in a clinical setting. Results The application of both classification models results in improved capacity control regarding the key performance indicators in the simulation study, with XGB outperforming LR. While LR leads to slight overoccupancy in the ICU, slight underoccupancy can be observed when XGB is applied. Conclusion Our study bridges the gap between predictive accuracy and practical application by emphasizing the importance of evaluating ML models within the context of ICU capacity management. The simulation-based approach offers a more relevant assessment for health care practitioners, providing actionable insights that go beyond classical performance measures and directly address the needs of decision makers in clinical practice. Highlights • We apply multiple classification models for ICU-LOS prediction using time-series data. This approach enables an update of the initial prediction resulting in the possibility of efficiently managing intensive care capacities. • We present a simulation-based approach to evaluate ML algorithms and their impact on bed capacity management in real-world clinical settings. • Our work provides in-depth insights into the impact of using ML techniques as decision support systems in the ICU and can lead to increased acceptance in practice.…