| Title |
Evaluating the Suitability of Large Language Models for Korean Urban Planning and Analyzing the Effectiveness of Legislation-Based RAG - Focused on the Korean Urban Planning Engineer Certification Examination |
| Authors |
조윤형(Cho, Yun Hyoung) ; 최해인(Choi, Hae in) ; 이제승(Lee, Jae Seung) |
| DOI |
https://doi.org/10.38195/judik.2026.08.27.4.5 |
| Keywords |
AI 도시계획; 대규모 언어모델; 도시계획; 도메인 적합성; 검색 증강 생성 AI Urbanism; Large Language Model; Urban Planning; Domain Suitability; Retrieval-Augmented Generation |
| Abstract |
This study examines the domain suitability of LLMs for Korean urban planning by constructing an evaluation dataset from the Korean Urban Planning Engineer certification examination. Five lightweight open-source LLMs and three commercial LLMs were evaluated, with additional tests on prompt engineering and retrieval ?augmented generation (RAG) for Urban Planning Laws and Regulations. The results show that performance does not simply depend on model size, while RAG yields more consistent gains than prompt engineering. Yet, open-source models with RAG only reached levels comparable to commercial LLMs without external knowledge, indicating limitations under on-premise deployment. The study provides a Korea-specific benchmark and a phased framework for integrating domain knowledge. |