PSI - Issue 64

Shirley J. Dyke et al. / Procedia Structural Integrity 64 (2024) 21–28 Dyke et al / Structural Integrity Procedia 00 (2019) 000–000

22

2

1. Introduction Structural engineers are responsible for the design, monitoring, and performance of our buildings and bridges. As society faces new challenges, such as increased natural hazards events, congested and crowded cities, and demands to build taller buildings and longer bridges, engineers must draw on their vast expertise and wealth of knowledge. Although structural engineering is grounded in well-established analysis techniques, every building and bridge represents a new design – with new requirements and new analysis. The knowledge that goes into each design is based on hundreds of years of experience that is embedded in specifications and codes. The products of these processes include the structural drawings, the building information models, and the response and performance of our buildings and bridges when exposed to extreme loading. The use of AI to aid in structural engineering and monitoring has sparked considerable interest by researchers and practitioners alike. Machine learning knowledge, practice, and technologies represent a fast-advancing front that engineers must race to meet. This frontier is opening opportunities for powerful structural engineering applications that have the potential to aid practitioners and researchers seeking to improve and maintain our built environment. The opportunity is approaching to reduce the workload on the human engineer by providing reliable methods to leverage the wealth of past data and existing knowledge to train AI models. In turn, those models, when appropriately trained and paired with a human engineer, will reduce the need for the human engineer to perform the more repetitive and tedious tasks, and allow engineers to focus on the technical decisions that truly require their attention and judgement. Much of the work to date in applying machine learning and AI has been accomplished within our research institutions and universities. The IISL team at Purdue University has been working to adopt and adapt existing AI methods to the field of civil engineering and specifically to structural health monitoring (SHM). The goal of this body of work has been to give the human engineer a variety of tools and associated guidelines. The focus has been on developing practical, proven solutions to empower engineers and scientists in data collection, data organization, knowledge extraction, and predictive modeling. For instance, some examples of this automation include: establishing the ability to sort information-rich data by extracting the contents, developing the ability to automatically organize large volumes of visual data collected for scientific research, and mapping the indoor location in GPS-denied regions to structural drawings. This work has resulted in ARIO, the Automated Reconnaissance Image Organizer , which provides a comprehensive cloud-based data analytics service that serves to integrate the human engineer with both the data and applications that are most needed for their research, while alleviating the time and effort involved in moving data around, installing software, and provisioning computational resources. Furthermore, by creatively leveraging state bridge databases, we demonstrated a method whereby seismic vulnerability across a large region (an entire state) can rapidly be determined. By fusing the information extracted from an entire set of building images, we also developed an approach for assessing the post-event state of a building. And by pairing a human with a machine for data collection, we illustrated how bridge asset managers may be able to dramatically cut costs. This paper provides a comprehensive perspective of the techniques and platforms developed in the IISL to empower engineers to accelerate the assessment and monitoring of infrastructure systems. Over the years we have envisioned the development of a fully automated AI-based pipeline that seamlessly integrates image content extraction and image context incorporation, synthesizes this data, and navigates the decision-making landscape. This work has catalyzed various additional studies. These accomplishments are discussed herein, providing both a strong foundation and a breadth of experience to empower engineers to employ ML technologies toward a safe and resilient built environment. 2. Extracting and Understanding Image Contents Real-world data is often collected in a disordered manner, and the collections are filled with unstructured, uncontrolled images that are all jumbled together. This fact is especially true during disaster reconnaissance missions where teams are working against time to collect data and are preoccupied with the disaster situation. These are far from the organized datasets that are typically needed to feed into machine learning training algorithms, and the contents must be extracted from each image. As a result, a sophisticated image classification tool is required to effectively sort through this chaos. Such a tool must contend not only with the volume and complexity of the data, but also with the inherent biases—since image content is heavily influenced by the data collector’s interests, motivation and purpose. Moreover, the prevalence of damage images within these large datasets adds another layer of difficulty, necessitating technologies that can adapt to and learn from these imperfections to provide reliable results.