| Title |
A Study on GPT-based Tagging System with Optimized Keywords for Automatic Classification of Construction Site Images |
| Authors |
Sungil Son ; Ali Akbar ; Jungtaek Hong ; Soonwook Kwon |
| DOI |
https://dx.doi.org/10.6106/KJCEM.2025.26.5.112 |
| Keywords |
Construction Site; Image Multi-Label Classification; Prompt Engineering; Fine-Tuning |
| Abstract |
The construction industry generates an increasing amount of visual data through sources such as drones, CCTV, and wearable cameras; however, the practical utilization of this data is often limited by a lack of systematic metadata. This study proposes an automatic image tagging system based on a GPT model designed to classify construction site images using multiple keywords. Keywords for image classification were derived through a combination of prior research analysis and expert surveys. The most practical keywords were subsequently selected by optimizing their relevance through performance testing. For the quantitative performance evaluation of the image classification platform, experimental conditions were designed based on three prompt engineering configurations and fine-tuning. Prompt engineering was divided into three types: (1) no prompt application, (2) providing a basic prompt (keyword list), and (3) providing an advanced prompt (keyword list and image derivation case). Fine-tuning was conducted using a multi-keyword image dataset derived from the collected training data. The experimental results demonstrated that both prompt engineering and fine-tuning significantly improved the accuracy, precision, and F1-score. The combination of prompt engineering and fine-tuning produced the most accurate results in complex scenarios. This study highlights the potential of GPT-based models for construction image analysis and demonstrates their value in building a smart site management platform. Future research can expand applicability and scalability, such as extending image-based tagging results to visual tracking and quality evaluation in conjunction with Building Information Modeling (BIM). |