PSI - Issue 62

Fabio Severino et al. / Procedia Structural Integrity 62 (2024) 276–284 Severino et al. / Structural Integrity Procedia 00 (2019) 000–000

281

6

● Organic patina (biological patina - mosses, algae, and lichens) ● Missing material and holes

The annotated images are stored within a private blockchain, making them immutable and non-repudiable. In order to be considered suitable for this application, the training dataset must include at least tens of thousands of high-resolution images. It's important to note that although the total size of this dataset would make the development of the system on a traditional blockchain financially unfeasible, in our case this is not a problem because the framework uses the Hybrid DLT infrastructure, which has been specifically designed to support the storage of large data sets in a cost-efficient manner. The blockchain is publicly notarized, enabling external auditing of the traced processes. One or more organizations with the role of data contributors are responsible for storing the annotated images in the ledger. Examples of potential data contributors include universities, research groups, engineering boards, or infrastructure managers. One or more organizations are assigned the role of data certifier and are given access to the ledger of annotated images. The task of a data certifier is to analyze the annotated images and, for each one, record a compliance statement in the ledger. An image is considered compliant if it meets the expected format and if the certifier believes that all and only the displayed defects have been accurately annotated. Data certifiers can be third-party organizations, but organizations that have also served as data contributors can potentially obtain this role as well. In an earlier phase, all conflicts of interest between the involved organizations are recorded in the ledger, and the smart contract governing the process ensures that a data certifier cannot assess the compliance of annotated images authored by any organization with which it has a conflict of interest. The smart contract governing the process determines, based on all the collected compliance statements, which images will constitute the certified training dataset. Once a certified training dataset is established, the next phase of the life cycle is the training of an AI model. One or more organizations are assigned the role of model provider . The task of a model provider is to use the certified dataset to train an AI model and then store it in the ledger, along with all the metadata that makes the training process reproducible. This metadata includes the model type, the algorithms used, and all chosen parameters. Additionally, we require that the algorithm that will be used to run the model is also stored in the ledger, as part of the model metadata. For this specific use case, the model type is a neural network which, without loss of generality, we can assume to be a Convolutional Neural Network, paired with a suitable classifier and also providing a saliency map of the prediction. Similar to the training dataset, the models also undergo certification. One or more organizations are assigned the role of model certifier . The task of a model certifier is to analyze a model and then record a compliance statement in the ledger. A model is considered compliant if it can be exactly reproduced by retraining a new model on the certified dataset. In this case as well, the smart contract governing the process determines whether the model is certified, for example, by verifying that at least two model certifiers have recorded a compliance statement. Once a certified model is available, the next phase of the life cycle is the inferences production. The roles of service providers , and inference verifiers are assigned to organizations. The service provider is an organization responsible for making the AI system available to external users. Specifically, for the Structural Defects Recognition service, the provider supplies an application to surveyors. This application enables the surveyors to upload photos captured by drones during inspections and submit them for processing. In response, the service provider processes these submissions, using one of the certified models, and stores the inference in the blockchain leveraging a smart contract. The inference is then returned to the user enriched with a certificate (blockchain metadata) representing its data lineage. The blockchain metadata contains the information needed to make the inference computation process reproducible by a third party (e.g., the indication of the specific version of the certified model and the algorithms used, and the value for all the input parameters used). The task of an inference verifier is to analyze the inferences stored in the ledger by a service provider, and independently verify them using inference metadata. Note that this role can be assumed by anyone who has access to the model and has been provided with the input; most often, this role is taken by regulators and third-party auditors. Additionally, it is also possible to extend the external audit to both the model training and the training dataset collection processes. If discrepancies (data tampering, omissions, execution of operations not permitted by the processes) are found during the external audit, the blockchain-based tracking of the AI system life cycle ensures that