PSI - Issue 64

Shirley J. Dyke et al. / Procedia Structural Integrity 64 (2024) 21–28 Author name / Structural Integrity Procedia 00 (2019) 000–000

23 3

Fig. 1. Automated image classification samples for buildings (first row) and bridges (second row).

Towards understanding and sorting the content of such real-world images, Yeum et al. originally developed a data schema and a novel deep learning-based method for the efficient classification of post-earthquake reconnaissance images (2016, 2017, 2018). Later, Park et al., applied a multi-output deep convolutional neural network (DCNN) to greatly expand the data schema and offer more classes and structure for the data and user. Some examples of the classified building images are shown in the first row of images in Figure. 1. This approach significantly improves the processing and analysis of large volumes of images by automating their classification into a complex, multi-level hierarchy, facilitating quicker and more informed post-disaster assessments. The deployment of this model within the ARIO platform exemplifies its practical application and potential to enhance disaster response and infrastructure resilience efforts. ARIO is a platform (see Figure 3) to automate the classification and analysis of a large volume of post-disaster visual data. This web-based tool is meant to organize reconnaissance images of reinforced concrete buildings exposed to seismic loads, enabling rapid sorting and sifting to find data that is useful for a given purpose, and generate comprehensive reports – one building at a time. The overall goal is to support data reuse and enhance the impact of the collected data. The tool was developed by first consulting with engineers that participate in these reconnaissance missions in order to define an appropriate data schema. Upon validation, the multi-class model achieved an impressive average of 90% recall and 79% precision. ARIO serves to efficiently organize and document visual data from damaged buildings, showcasing the potential of recent AI advances in handling complex real-world images and supporting disaster response efforts (Park et al., 2019, 2022). ARIO is a cloud-based data analytics system with advanced search capabilities akin to Netflix, allowing field engineers to search an extensive image repository efficiently. The capabilities embodied in ARIO are poised to revolutionize the way structural engineers learn from earthquake data by enabling rapid analysis and categorization of large image volumes with minimal manual effort. Through the use of deep convolutional neural networks and trained classifiers, it assists engineers in drawing insights from post-disaster data more rapidly (Dyke et al. 2020). Going beyond image classification, Yeum et al. also tackles and enables object detection for isolating damaged regions visible within post-disaster images (2018). Using convolutional neural networks (CNNs), they demonstrate the capability to automate detecting and localizing objects indicative of structural damage, such as spalling, in large volumes of image data collected from disaster-affected areas. By supporting post-event building reconnaissance and rapid data analysis in this manner, this work is enabling more precise and rapid identification of structural damage without manual analysis (2018, 2019). Clearly the classifiers and the data schema for buildings would not be suited to images of bridges. Furthermore, the structural components of bridges are typically visible, and images are collected on a regular basis during biannual inspections. Thus, Zhang et al. built on this approach to develop a way to use existing comprehensive inspection databases to develop classifiers for various purposes (2023a). For instance, seismic vulnerability of bridges is mainly dependent on the substructure of the bridge. Yet, the type of substructure is not recorded in inspection databases in