Automated Defect Detection using Stereo Vision Algorithm for Metal Surfaces

Abstract

Automated surface defect inspection on metallic components presents significant challenges due to the specular reflectivity and low surface texture of polished metal, which render conventional two-dimensional (2D) vision systems incapable of detecting geometrically invisible defects. This paper presents a stereo vision and image processing framework that integrates a custom GPU-accelerated classical stereo matching pipeline with a YOLOv8 deep learning detection model to perform simultaneous defect classification and physical depth measurement on metal surfaces in real time. The stereo pipeline is implemented entirely in PyTorch without pre-built stereo functions, comprising a fused matching cost function combining census transform, absolute difference, and gradient cost, followed by guided filter cost aggregation, Semi-Global Matching in four scan directions, subpixel-accurate Winner-Takes-All disparity selection, and a multi-stage refinement process. A YOLOv8 nano model is trained on the NEU Surface Defect Database (NEU-DET), covering six steel defect classes including crazing, inclusion, patches, pitted surface, rolled-in scale, and scratches. The integrated system maps each detected bounding box onto the computed disparity map and applies the depth formula Z = (f × B) / d to compute physical depth in millimetres per defect. The stereo pipeline achieves an average Bad-1.0 error of 7.57% across all 15 Middlebury MiddEval3 training scenes at quarter resolution. The YOLOv8 detector achieves 70.0% mean Average Precision (mAP) at 0.50 IoU threshold on the NEU-DET validation set. The integrated live system operates at 28 to 30 frames per second on a ZED stereo camera using an NVIDIA GeForce RTX 3070 GPU, providing simultaneous defect classification and depth measurement output. The proposed low-cost framework addresses the geometric measurement gap of existing 2D inspection systems, offering a practical and affordable solution for surface defect inspection in small and medium manufacturing environments.