| Title |
Real-to-Sim based Hierarchical Knowledge Transfer Learning for Semantic Segmentation in Off-road Environments |
| Authors |
류제호(Je-ho Ryu) ; 김용휘(Yong-hwi Kim) ; 이승주(SeungJoo Lee) ; 김민오(Mino Kim) ; 권효신(Hyo-shin Kwon) ; 조지혁(Jihyuk Cho) |
| Keywords |
Unmanned ground vehicle (UGV); Semantic segmentation; Real-to-sim; Hierarchical knowledge transfer; Domain generalization |
| Abstract |
Recently, research on autonomous driving technologies in off-road environments, particularly focusing on Unmanned Ground Vehicles (UGVs), has been actively conducted. While environmental perception via semantic segmentation is essential for autonomous driving in off-road environments lacking prior information, the inherent high irregularity of such environments requires a vast learning domain, making it difficult to secure corresponding training data. To address this, 'Sim-to-Real' methodologies utilizing virtual environment data have been predominantly adopted; however, limitations exist where model performance degrades during real-world application due to the domain gap between the virtual and actual off-road environments. Accordingly, this study proposes a novel methodology based on a 'Real-to-Sim' approach, contrary to conventional methods, that transfer-learns a perception model pre-trained on real-world data to virtual environment data. The proposed technique is based on a Hierarchical Knowledge Transfer structure that combines an anchor branch, which freezes parameters pre-trained in the real environment, and an adapter branch, which learns variations in the virtual environment. In particular, by introducing a gate module that controls the intervention of the adapter branch according to the domain characteristics of the input data, we implemented a mechanism that preserves knowledge during real-world input while selectively fusing features only during virtual environment input. Training was conducted utilizing the RELLIS-3D and NEGS-UGV datasets. Furthermore, through evaluations on separate, unseen real-world datasets, we confirmed that the proposed method demonstrates superior performance and generalization capabilities compared to existing approaches. |