Enhancing Deep Learning-Based Damage Segmentation with Depth Hallucination

Abstract

Recent studies have suggested that the fusion of cross-modal information can enhance the performance of deep learning-based segmentation algorithms. In this context, this study evaluates the benefits of RGB-D fusion with regard to damage segmentation in reinforced concrete buildings. The fusion of depth data was observed to enhance the segmentation performance significantly. Additionally, a number of surrogate techniques based on modality hallucination and monocular depth estimation are exploited to eliminate the need for depth sensing at test time without foregoing the benefits of depth fusion. The proposed techniques require depth data only for network training, and at test time, depth features are simulated from the corresponding RGB frames, obliterating the need for real depth perception. The proposed methods are evaluated and are shown to increase the damage segmentation accuracy.