PSI - Issue 62

Fabio Parisi et al. / Procedia Structural Integrity 62 (2024) 701–709 F. Parisi et al. / Structural Integrity Procedia -- (2024) _ – _ 5 EDPs of the different piers generated by using NLTHA. The features x which are used to train ML algorithms are: 1) the IM features reported in Table 2, 2) the vector of [ , , , .] which is univocally associated with each bridge realisation. As specified in the methodology in Section 2.3, a representative portion of is selected to be analysed and used in the training phase of the ML algorithms. The instances in are selected so that the target values (i.e., EDPs) cover with numerosity balance the entire range present in . 4. Feature importance and selection In line with the existing literature, the feature selection is a preliminary step for the training of ML algorithms on datasets presenting numerous features (Kazemi et al., 2023). It aims to reduce the “black box” effect of the models, improve the interpretability of the results, and reduce the computational effort [(Parr et al., 2018)]. In this work, this step is performed by: (i) Evaluating the feature importance with permutation importance (Altmann et al., 2010) and drop-column importance (Parr et al., 2020) strategies; (ii) hierarchical clustering to identify progressively less numerous groups of features; (iii) measuring the informative content of the feature groups by monitoring the loss of prediction performances along with the diminishing number of features. 4.1.1. Permutation importance The feature importance investigation is often conducted with the permutation importance strategy (Altmann et al., 2010). It relies on an already trained regressor whose performance decrease is measured when each feature is changed in its value, while the other features are not varied. However, permutation importance suffers from limitations in the case of multicollinear features because it can overestimate the importance of correlated predictor variables (Strobl et al., 2008). This limitation is shown in this work by adopting the drop-down column strategy (Parr et al., 2020) on ten-fold cross-validated models: three times, starting with all the features, ten-fold cross validated RF regressors were trained, the permutation importance was performed, and the most important feature was dropped to train the subsequent ten-fold cross-validated RF regressors. Then, three families of ten RFs were obtained: 0 , trained on all the features; 1 trained on all the features minus ( ) , and 2 trained on all the features minus ( ) and ( ) . Errore. L'origine riferimento non è stata trovata. , Errore. L'origine riferimento non è stata trovata. and Figure 3 show the absolute performance of the different models trained in predicting the displacements of the top of pier P3. In particular, Errore. L'origine riferimento non è stata trovata. reports a comparison among the predicted values of the displacements for 0 , 1 and 2 : the red line represents the bisector of the graph and the ideal condition for the EDP predicted to perfectly match the ground truth values. The more the points are distributed close to the red line, the more one model can interpret the test data. This comparison highlights that no evident model is outperforming the others, and, thus, more features (strongly important in the permutation importance investigation) do not imply better performance of the models. The performance highlighted in Errore. L'origine riferimento non è stata trovata. is expressed in terms of multiple summary metrics in Errore. L'origine riferimento non è stata trovata. , where the coefficient of determination, R 2 , , sum of the absolute errors ( ) and the maximum of the absolute errors ( ae = max( ) ) are reported. Even in this case, no models overcome the others in the reported metrics, supporting the thesis that decreasing the number of features does not generally imply the loss of prediction performance. Figure 3 reports the box plots of the scores ( ) obtained with the models trained with the ten-fold cross validation. It is evident that the means of the performance score (the vertical black line in each box plot) are comparable when decreasing the number of the features used to train the models, while we note a slightly smaller variance of the prediction with fewer features ( rf 2 in red). All the results reported in terms of feature importance highlight the poor power of the permutation importance strategy for the problem at hand and denote the presence of multicollinear features. This evidence, thus, confirms the need for further investigation to detect the informative features to consider in the training of the models. 4.1.2. Hierarchical clustering of features When features are collinear, permuting one feature has little effect on the model performance because it can get the same information from a correlated feature. We dealt with such data configuration by performing hierarchical 705