PSI - Issue 44

Agnese Natali et al. / Procedia Structural Integrity 44 (2023) 2020–2027 Agnese Natali et al./Structural Integrity Procedia 00 (2022) 000–000

2023

4

3.1. Model architecture and training We consider training a convolutional neural network (CNN) using the CODEBRIM dataset Mundt et al. (2019) with 4 state-of-the-art (SOTA) architectures, viz. VGG16, VGG19 in Simonyan and Zisserman (2015), ResNet50 in He et al. (2016) and Xception in Chollet (2017) as feature extraction modules, motivated by the results reported in Padalkar et al. (2020). The general architecture for training the CNN is shown in Fig. 4.

Fig. 3. Our model architecture can make use various state-of-the-art feature extraction modules (VGG16/VGG19/ResNet50/Xception). The input patches are resized according to the size needed by the SOTA feature extraction module. Size of the output equals the desired number of output classes. To train the classification model, we used the train/validation/test splits provided in Mundt et al. (2019), each containing 9214/621/637 patches, respectively. From these patches inputs were cropped as per the sizes needed by the feature extraction modules. For the training set, the cropping was performed from random locations in the patches, while for the validation and test sets the central areas were cropped. The models were then trained by using each of SOTA feature extraction modules, viz. VGG16, VGG19, ResNet50 and Xception. The training was performed using the Adam optimizer with a learning rate of 0.0001 for 50 epochs to minimize the binary crossentropy (BCE) loss defined below. � , � = − � ⋅ log � � + (1 − ) ⋅ log � 1 − �� (1) where, and are the true and predicted binary labels (0 or 1), respectively, for a given output. 3.2. Results: Training with CODEBRIM dataset In Table 1 we report the multi-target accuracies obtained after training the model with different SOTA feature extraction modules. Note that, for a given sample the multi-target accuracy denotes correct prediction of every defect class jointly. Thus, a low multi-target accuracy can translate to a high classification accuracy if every defect class is treated independently. For example, with the Xception module a 67% multi-target accuracy translates to ~91% classification accuracy treating each defect class independently. From Table 1 we observe that the Xception feature extraction module provides the best multi-target accuracies for both validation and test sets. For further analysis, we therefore fixed the feature extraction module to Xception in our model. In Fig. 5 we show the qualitative results obtained using the model trained with the Xception feature extraction module.

Table 1: Multi-target accuracy for the model trained with different feature extraction models. 'Bv-test' denotes the test accuracy using the same model that provided the best validation accuracy. Feature extraction module used in the model Multi-target accuracy [%] best validation bv-test VGG16 in Simonyan and Zisserman (2015) 59.09 57.75 VGG19 in Simonyan and Zisserman (2015) 55.68 54.75