| Title |
Synthetic Video Generation Process Model for Enhancing the Activity Recognition Performance of Heavy Construction Equipment - Utilizing 3D Simulations in Unreal Engine Environment - |
| Authors |
Shin, Yejin ; Seo, Seungwon ; Koo, Choongwan |
| DOI |
https://dx.doi.org/10.6106/KJCEM.2025.26.1.074 |
| Keywords |
Synthetic Video Generation Process; Heavy Construction Equipment; Activity Recognition; F1 Score; Game Engine; 3D Simulations |
| Abstract |
There has been a growing interest in AI (Artificial Intelligence)-based smart management for heavy construction equipment, aiming at real-time monitoring of safety, productivity, and environmental impact. In addition, deep learning-based computer vision technologies have advanced to identify the activities of construction equipment through visual information from CCTV (Closed-Circuit Television) at construction sites. Ensuring the performance of such vision technologies requires a substantial amount of training video datasets collected from construction sites; however, there are limitations in gathering datasets across diverse scenarios due to the nature of construction environments. To address this challenge, this study aimed to develop a synthetic video generation process model to enhance the activity recognition performance of heavy construction equipment. The proposed process model can closely simulate real videos of construction equipment using 3D simulation in game engine. This study validated the stepwise performance improvement of the proposed process model using the 3D ResNet-18 model for excavator activity recognition. The performance of the final stage, measured by the weighted F1-score, showed a 90.89% performance, marking an approximate 25% improvement compared to the first stage (66.02%). This performance is very similar to the activity recognition performance for real videos (90.12%). The confusion matrix demonstrated that the recognition performance and patterns for both real and synthetic videos were considerably similar. The synthetic videos produced through the proposed process model can be utilized as training datasets and serve as a foundational model for simulating excavator operations. |