Automated Region-of-interest Localization and Classification for Visual Assessment

Abstract

Visual assessment is a process to understand the state of a structure based on evaluations originating from visual data. Low-cost, high-performance vision sensors are providing new avenues for overcoming the spatial and temporal limitation in current human-based visual assessment when used in conjunction with aerial sensing platforms. However, past implementations are limited in their ability to deal with a high volume of images while only a small fraction of them are important for actual inspection. Such difficulty induces an unwanted high rate of false-positive and negative errors, reducing the trustworthiness and efficiency of their implementation. To overcome this challenge, we develop and validate a novel automated image localization and classification technique to extract regions-of-interest (ROIs) on each of images, which contain the target region of the structure for visual evaluation (TRI). First, ROIs are extracted based on the geometric relationship between the collected images and the TRIs using structure-from-motion algorithm. Second, unwanted ROIs corrupted by occlusion and image blur are effectively filtered by a robust image classification technique, called convolutional neural network. Then, a damage detection technique is applied only on such highly relevant and localized ROI images. The capability of the technique is demonstrated using a full-scale highway sign structure for the case of crack detection on weld connections.