Feature selection is a crucial step in statistical learning that aims to identify the most relevant variables from high-dimensional data to improve model interpretability, enhance generalization, and reduce computational complexity. Eliminating irrelevant or redundant features helps reduce the effects of the curse of dimensionality and contributes to preventing overfitting. Feature selection methods are typically categorized into filter, wrapper, and embedded approaches, each offering distinct trade-offs in terms of efficiency, scalability, and predictive performance. In this talk, we present a novel approach to feature ranking, referred to as greedy feature selection [3]. Greedy algorithms are traditionally used to iteratively select a reduced and appropriate set of examples based on error indicators, thus enabling the construction of surrogate and sparse models [1, 2]. In our approach, the greedy strategy is extended to operate in the feature space to select and rank the most predictive variables for classification tasks. This method falls into the category of wrapper feature selection techniques. Unlike classical methods such as Recursive Feature Elimination, our approach is fully model-dependent and target-specific, allowing any performance metric to be optimized during the iterative selection process. Specifically, given any classifier and evaluation metric, the algorithm selects the most informative feature at each step in a classifier-dependent manner. Within the framework of physics-informed machine learning models, we further demonstrate that this scheme can be employed to identify physics-informed features, opening the door to data-driven discovery of governing equations in scientific applications. We present results on a variety of datasets, including real and noisy data, across diverse domains such as medical diagnostics and space weather forecasting. References [1] Dutta, S., Farthing, M.W., Perracchione, E., Savant, G., Putti, M. A greedy non-intrusive reduced order model for shallow water equations. J. Comput. Phys. (2021) 439, 110378. [2] Wenzel, T., Santin, G., Haasdonk, B. A novel class of stabilized greedy kernel approximation algorithms: Convergence, stability and uniform point distribution. J. Approx. Theory (2021) 262,105508. [3] Camattari, F., Guastavino, S., Marchetti, F., Piana, M., Perracchione, E. Classifier-dependent feature selection via greedy methods, (2024) Statistics and Computing, 34(5), 151.