PSI - Issue 78

Marco Martino Rosso et al. / Procedia Structural Integrity 78 (2026) 1382–1388

1385

3. Results and Machine Learning Interpretability The grid-search k-fold cross-validation with 10 folds has been employed during the training phase for hyperparameter tuning of ML-selected models. The optimal hyperparameters have been identified in C=1 for Linear SVM, and for RF, after comparing almost six thousand combination of parameters, it was found that Gini impurity index has been appointed as splitting criterion, maximum depth could be set to 20, maximum leaf nodes is 100, minimum sample leaf is 4, minimum sample split is 10, and the number of decision tree estimators in the forest could be set to 200. The test set classification metrics for the binary building usability class classification problem are reported in Table 1.

Table 1. Test set results for Linear SVM and RF classifiers.

Model

Linear SVM

Random Forest

Class

AB

CE

AB

CE

Accuracy [%]

62.32 65.91

67.73 65.72

Balanced Accuracy [%]

Precision [%]

88.16 59.40 70.98

34.03 72.41 46.30

86.35 69.37 76.94

36.95 62.07 46.32

Recall [%]

F1-score [%] AUC-ROC [%]

71.29

74.33

Considering the obtained results, the balanced accuracy is practically the same for both RF and SVM models (about 66%); however, considering the minority class CE, it is possible to observe that SVM provides a higher recall value of 72% against 62%, demonstrating a better detection of minority class instances. Nevertheless, it is worth reminding that the simplicity of the SVM model makes it more prone to the overfitting issue, especially in imbalanced learning, where its focus is typically more related to the majority class only. Furthermore, focusing on F1-score, which is a harmonic mean of precision and recall, thus summarizing in a single index these two metrics, it is worth observing that RF achieves slightly better results with almost 37% against 34% of SVM, but this difference is negligible in practice. In conclusion, considering also ROC and precision-recall curves depicted in Figure 1, the RF can be appointed as the best classifier since its overall generalization capabilities and slightly higher classification performances, as demonstrated by its classification metrics.

Fig. 1. (a) RF and SVM ROC curve; (b) RF and SVM precision-recall curve.