AI-Driven Railway Maintenance for Fault Identification Through Object Detection and Segmentation

Abstract

Railway inspection is an important task to ensure the safety and reliability of transportation systems. Regular inspection of key components such as sleepers, fasteners and tracks are essential to maintain the infrastructure and prevent accidents. This paper proposes a deep learning-based framework for automating railway inspection by combining object detection and fault segmentation. The experimentation was performed using the data comprising of track videos, which was further refined using image enhancement techniques where each frame of the video has been used for evaluating the models. The pipeline initiates with YOLO11 being employed in the first phase for detecting railway components due to its superior performance in limited data scenarios, with the second phase utilizing Mask R-CNN to detect and segment areas of damage and corrosion in the corresponding detected components. To benchmark our proposed pipeline, RTDETR was also used with Mask R-CNN and observed that YOLO11 combined with Mask R-CNN outperformed RTDETR in terms of accuracy and efficiency. This work highlights the potential of integrating advanced object detection and segmentation techniques to streamline railway maintenance by automating fault detection from images and video. The models were also evaluated on videos captured at varying speeds.