PSI - Issue 64

Christoph Brenner et al. / Procedia Structural Integrity 64 (2024) 1240–1247 Christoph Brenner et al. / Structural Integrity Procedia 00 (2019) 000 – 000

1244

5

Evaluation metrics for generated trees include the misclassification rate, quantifying misclassified data points, and Gini impurity, assessing alignment between predicted and empirical probabilities. Both metrics are computed on training (in-sample) and test (out-of-sample) data. An implementation of OCTs in the Interpretable AI software package is used here (Interpretable AI LLC (2023)). 3. Results 3.1. Determination of suitable hyperparameters for OCTs in the case study Before the model with all six parameters is investigated, the hyperparameters of OCTs are examined first using a model with only one parameter. Gini impurity is used as the training criterion. The tree depth is varied from one to seven and it is investigated whether the specification of a minimum number of samples per leaf has an effect on the accuracy of the tree. Therefore, a threshold of 0.1% of the samples is chosen, which represents a good compromise between a sufficiently large branching for high accuracy and the avoidance of overfitting due to excessive splitting. Furthermore, the use of hyperplane splits with a sparsity of two (linear combinations with two parameters) is compared to axis-parallel splits. A k-fold cross-validation with k=3 is conducted for all investigated trees to ensure the consistency of the results. The resulting Gini impurities are shown in Fig. 3, the corresponding values are presented in Table 1.

Fig. 3. General trends of Gini impurities across different tree depths, types of splits, and minimum number of samples per leaf.

The left diagram in Fig. 3 illustrates the Gini impurity results for the OCTs when no minimum number of samples per leaf is imposed. Initially, with axis-parallel splits, the Gini impurity tends to increase as tree depth grows, both for the training and test dataset. Starting from a tree depth of four, a slight difference in Gini impurity with worse results for the test dataset is observed (see Table 1), indicating a tendency towards overfitting. OCTs using hyperplane splits with a sparsity of two exhibit marginally better results compared to axis-parallel splits, although this advantage diminishes with deeper trees. The divergence between training and test data can also be observed for OCTs with hyperplane splits from a depth of four onwards. The results depicted in the right diagram of Fig. 3, where a minimum of 0.1% samples per leaf is enforced, mirror those of the unrestricted OCTs. Although as can be observed in Table 1 outcomes for various tree depths are marginally inferior to those without constraints, the gap between the Gini impurities of training and test datasets is smaller, even for deeper trees. Similar to the previous setup, the results of OCTs with hyperplane splits are slightly better than the results with axis-parallel splits.