PSI - Issue 78

Marco Martino Rosso et al. / Procedia Structural Integrity 78 (2026) 1382–1388

1386

Last but not least, understanding the influence of input variables on classification outcomes is essential for optimizing post-earthquake survey forms, reducing assessment costs, and improving model transparency. Three complementary methods were employed herein to evaluate feature importance in the RF model. The RF intrinsic feature importance selection mechanism is based on assessing the Gini impurity index across all trees. With this method, it appears that building age, presence of curbs or chains, roof type, and masonry texture are the most important features driving the classification task. However, these findings are limited, since Strobl et al. (2007) [] underlined the limitations of this method, especially when dealing with mixed continuous and categorical data. The second feature importance method is denoted as permutation feature importance. It is a model-agnostic method, based on randomly shuffling one by one the columns of the dataset to break the input features-output class relationship, and tracking the classification errors during a hundred repetitions of this shuffling procedure. Also in this second case, it has been confirmed that building age, curbs/chains, and masonry texture are the most influential features, and among the seismic intensity measures, the vertical PGA appears to be the most critical seismic parameter. Although this second method is more robust to feature type rather than RF Gini-based intrinsic feature importance, it is worth analyzing another state-of-the-art method for interpretability of ML models, denoted as shapley additive explanations (SHAP) functions. Rooting their theoretical basis in game theory, this method is a rigorous model-agnostic explainability method that relies on attributing a local feature contribution to the final predictions, permitting the analyst to observe the average impact on RF of every feature affecting the prediction of both classes in the current binary classification scheme. Even in this case, interpretability results align with the other two analysed methods, i.e., confirming the greater importance of curbs/chains, roof type, masonry texture, and age on usability scores. These results are physically reasonable since, in a masonry building, the presence of curbs and chains ensures the masonry box behavior of the entire building. Moreover, SHAP highlights also the minority class contributions, i.e., the ones that affect the most an unusable score prediction. Indeed, if a building is old, with an ancient construction year, or if there are no chains/curbs, or if there are irregular masonry textures, the building will probably receive an unusable score. This is again consistent with the physics, especially with other Central Italy earthquake studies such as Fiorentino et al. (2018); Luzi et al. (2017), thus providing a useful engineering-based explanation tool attached to ML classifiers to assess the effectiveness of their possible real-world recall situations.

(b)

(a)

Fig. 2. RF feature importance: (a) with Gini impurity index; (b) with permutation.