Open Access

The integration of machine learning methodologies has become prevalent in the development of clinical prediction models, often suggesting superior performance compared to traditional statistical techniques. Within the scope of low-dimensional datasets, encompassing both classical and machine learning paradigms, we plan to undertake a comparison of variable selection methodologies through simulation-based analysis. The principal aim is the comparison of the variable selection strategies with respect to relative predictive accuracy and its variability, with a secondary aim the comparison of descriptive accuracy. We use six distinct statistical learning approaches across both data generation and model learning. The present manuscript is a protocol for the corresponding simulation study registration (Study registration Open Science Framework ID: k6c8f). We describe the planned steps through the Aims, Data, Estimands, Methods, and Performance framework for simulation study design and reporting.

The integration of machine learning methodologies has become prevalent in the development of clinical prediction models, often suggesting superior performance compared to traditional statistical techniques. Within the scope of low-dimensional datasets, encompassing both classical and machine learning paradigms, we plan to undertake a comparison of variable selection methodologies through simulation-based analysis. The principal aim is the comparison of the variable selection strategies with respect to relative predictive accuracy and its variability, with a secondary aim the comparison of descriptive accuracy. We use six distinct statistical learning approaches across both data generation and model learning. The present manuscript is a protocol for the corresponding simulation study. We describe the planned steps through the Aims, Data, Estimands, Methods, and Performance framework for simulation study design and reporting.

Disclosure: O. Abawi: None. G. Sommer: None. M. Groessl: None. U. Halbsguth: None. S.E. Hannema: None. C. de Bruin: None. E. Charmandari: None. E.L. van den Akker: None. A.B. Leichtle: None. C.E. Flueck: None. Introduction: Current treatment monitoring of children with congenital adrenal hyperplasia (CAH) relies on specialist’s interpretation of clinical and biochemical parameters, but remains dissatisfactory. Comprehensive 24h urine steroid profiling provides detailed insight into adrenal steroid pathways, but its merit in routine treatment monitoring of CAH is not yet established. Aim: To investigate whether 24h urine steroid profiling can predict treatment quality assessment in children with CAH using machine learning (ML). Methods: This prospective observational cohort study included children with genetically confirmed 21-hydroxylase deficiency. Children collected 24h urine at 2 outpatient clinic visits (mean 4.1 ± 0.7 months apart). Using gas chromatography-mass spectrometry, 40 adrenal steroids and metabolites from the classic, backdoor and 11-oxygenated pathways were analysed. Patients were classified as undertreated, optimally treated or overtreated by the pediatric endocrinologist based on detailed clinical and endocrinological evaluation including serum 17-hydroxyprogesterone and androstenedione. We used sparse partial least-squares discriminant analysis (sPLS-DA) to investigate optimal prediction of treatment quality assessment. This ML method computes components (combinations of all input variables) and selects the most discriminative parameters to classify samples (in our case optimally treated vs undertreated) by maximizing between-class variance. We computed area under the ROC-curve (AUC) of two sPLS-DA models: 1. using only 24h urine metabolites; 2. adding also clinical variables age, sex, pubertal status, CAH subtype (classic vs non-classic), medication (hydrocortisone [HC] vs prednisolone), daily HC-equivalent dose, Δbone age minus chronological age, ΔBMI-z, and Δheight-z. Results: We included 112 visits (68 [61%] optimally treated, 44 [39%] undertreated) of 59 patients: 27 (46%) girls, 46 (78%) classic CAH, 19 (32%) prepubertal. Mean age at first visit was 11.9 ± 4.0 years and mean BMI SDS 0.6 ± 1.1. SPLS-DA using 24h urine metabolites showed clear clustering of optimally treated patients on two components, while undertreated patients were more heterogenous (AUC 0.88; 95% CI 0.81-0.94). The model selected pregnanetriol and hydroxypregnanolon contributing to excluding undertreatment and 7 metabolites contributing to excluding optimal treatment: estradiol, cortison, tetrahydroaldosterone, androstenetriol, etiocholanolone, androstenediol, and α-dihydrocortison. Addition of clinical variables did not improve classification (AUC 0.90, 95% CI 0.84-0.96, P=0.59). Discussion: Using ML on 24h urine steroid profiling predicted treatment quality assessment in children with CAH even in absence of clinical data, suggesting that comprehensive 24h urine steroid profiling could improve treatment monitoring in children with CAH. Presentation: Thursday, June 15, 2023