| Title |
A Strategic Approach to Enhancing the Practical Applicability of Vision-based Detection and Classification Models for Construction Tools - Sensitivity Analysis of Model Performance Depending on Confidence Threshold - |
| Authors |
Han, Soeun ; Won, Jaeseung ; Koo, Choongwan |
| DOI |
https://dx.doi.org/10.6106/KJCEM.2025.26.1.102 |
| Keywords |
Object Detection and Classification; Practical Applicability; Confidence Threshold; F1-score; Recall; Precision |
| Abstract |
To prevent safety accidents and improve work efficiency at construction sites, it is essential to develop a model that can detect and classify various objects using vision technology, focusing on ‘small-sized construction tools’. This study aimed to improve the practical applicability of the object detection and classification model for four types of 'small-sized construction tools (bucket, cord reel, hammer, and tacker)' using the YOLOv8 algorithm. To achieve this, a sensitivity analysis was performed on 144 images captured directly from construction sites, enabling the development of utilization strategies for the proposed model and enhancing its effectiveness in real-world scenarios. The main findings of this study can be summarized as follows. First, the overall performance (mAP_0.5) of the proposed model was high at 94.8%. Second, the patterns of change in F1-score were systematically analyzed with varying confidence thresholds for each object. Based on this sensitivity analysis, three utilization strategies were proposed: (i) a standard strategy with recommended confidence threshold ranges for high F1-scores (Group(A): 0.7?0.8; Group(B): 0.2?0.3); (ii) a recall-oriented strategy with a lower confidence threshold to prevent safety accidents; and (iii) a precision-oriented strategy with a higher confidence threshold to facilitate the introduction of robotics and automation technologies for improved work efficiency. Future research is expected to enhance the scalability and reliability of the proposed model through living lab-based projects at construction sites. |