PSI - Issue 44

Agnese Natali et al. / Procedia Structural Integrity 44 (2023) 2020–2027

2024

Agnese Natali et al. / Structural Integrity Procedia 00 (2022) 000–000 5

ResNet50 in He et al. (2016) Xception in Chollet (2017)

62.83 64.94

65.35 67.09

Fig. 4: Qualitative results obtained by using our trained model having the Xception in Chollet (2017) feature extraction module that provided the best validation accuracy for the CODEBRIM dataset in Mundt et al. (2019).

3.3. Results: Training with METU dataset We also train the model using the METU dataset in Özgenel (2019), with the only difference of having 2 output classes (cracks and no-cracks) as opposed to 6 (1 background and 5 defects) when using the CODEBRIM dataset. For this purpose, we split the METU dataset into train/validation/test sets using 60/20/20 % data, respectively. Since this is a binary classification problem, the multi-target accuracy is same as the classification accuracy. The model trained using METU dataset is also evaluated on the CODEBRIM dataset. To this end we generate a simulated binary CODEBRIM dataset by relabeling the 6-class data into two classes, viz. background and defects. The latter contains all the defect classes combined into a single class. The accuracy of the trained model on the two datasets is presented in Table 2. We also present the qualitative result on an image from the CODEBRIM dataset in Fig. 6. The results indicate that the model trained with METU dataset has good accuracy for detecting cracks but does not generalize well for other defect classes. This is mainly because the model was never presented with other defect classes during training. Thus, use of multi-target datasets is more suitable for training a model that can recognize different infrastructure elements.

Table 2: Accuracy for the model trained on the METU dataset that provided the best validation accuracy. Dataset

Accuracy [%] validation set

test set 99.85 54.67

99.91 48.70

METU in Özgenel (2019) Simulated binary CODEBRIM

Fig. 5. Qualitative result for an image from the CODEBRIM dataset using the model trained on the METU dataset. Some parts of the background are wrongly detected as defects because of the absence of such patches in the training examples.