ElasticNet

Using a single tuned regularisation value avoids nested cross-validation at every StackFeat iteration. The Elastic Net model showed the strongest overall performance across feature subsets, achieving an AUC of 0.821 with 5 to 30 features. Calibration was performed using a sigmoid regressor, which outperformed an isotonic regressor by producing a lower Brier score and a calibration plot closer to the ideal line. On COVID-19 miRNA data, ElasticNet had the highest mean AUC but used more than twice as many features as StackFeat-RL. The five-variable Elastic Net model achieved the same AUC as the full 30-variable version, showing that the most important predictive information was captured by a small set of variables. ElasticNetCV is treated as a strong embedded baseline but can depend on partitioning and retain large correlated feature groups. Regularisation techniques such as LASSO and Elastic Net are designed to identify informative features while penalising less useful predictors, which explains why simpler models can match complex ones. Many of the 30 variables may have been correlated or redundant, adding overlapping rather than independent predictive value, consistent with the cu…