Date of Award
2024
Document Type
Thesis
Degree Name
Master of Science in Engineering (MSE)
Department
Electrical and Computer Engineering
Committee Chair
Avimanyu Sahoo
Committee Member
David Pan
Committee Member
Rahul Badani
Research Advisor
Avimanyu Sahoo
Subject(s)
Neural networks (Computer science), Computer vision, Artificial intelligence, System safety
Abstract
Visual inspections of safety-critical systems are crucial in reducing the risk of equipment failures, downtime, and loss of life. This nondestructive testing (NDT) method uses a portable borescope or camera along with other sensors directly or mounted on robotic platforms to inspect difficult-to-access areas with ease, minimum time, and cost. Although state-of-the-art visual inspection platforms are equipped with sensors from multiple modalities, the inspection tasks still require human subject matter experts to identify defects and analyze them. This jeopardizes human safety in a hazardous work environment in energy industries, as well as the extended time for inspection and human error. Moreover, defect identification becomes much more challenging, especially in large machinery and structures, such as aircraft engines, concrete bridges, and buildings, because of differences in material appearance, changing lighting, different surface markings, and the possible overlap of varying defect types. In order to automate the process and address the inherent challenge of defect classification, it is imperative to employ a resilient deep-learning approach that can accurately identify the defects. In this thesis, a hybrid deep learning method for multilable defect classification from visual data by using graph neural networks (GNN), convolutional neural networks (CNN), and feedforward neural networks (FFN) is presented. The first part of the thesis provides a comprehensive review of state-of-the-art GNNs for machine vision to derive motivation for the research. The literature review describes various graph-learning approaches and the challenges associated with generating graph- structured datasets from images. The primary focus of the review is the application of GNNs in machine vision and their mathematical formulations. In the second part, the proposed defect classification methodology is presented, which diverges from conventional deep learning approaches for multilabel defect classification by harnessing the combined strengths of CNN and GNN algorithms. The core idea is to exploit CNNs for their prowess in recognizing the visual characteristics of defects and GNNs for their ability to capture the relational structures of defects, facilitating more precise differentiation. This multilabel hybrid vision GNN algorithm is validated using the open-source CODEBRIM dataset, which contains multilabel images of large-scale concrete structural defects. The model’s performance in image classification is validated using the CIFAR-10 dataset, achieving 86 % accuracy during testing. Experimental results demonstrate that the hybrid architectures developed have fewer overall parameters and achieve a 16% improvement in accuracy compared to popular neural architectures for defect classification.
Recommended Citation
Hossen, MD Sazzad, "Multilabel defect classification using graph neural networks for autonomous visual inspections" (2024). Theses. 685.
https://louis.uah.edu/uah-theses/685