| Title |
Behavior Tree-based Excavation Unit Task Planning Algorithm using Q-Learning |
| DOI |
https://doi.org/10.5573/ieie.2025.62.2.96 |
| Keywords |
Excavator; Automation; Excavation planning; Behavior tree; Reinforcement learning; Q-Learning |
| Abstract |
This study proposes the Q-Learning behavior tree (QL-BT) algorithm, which combines BT and reinforcement learning (RL) to improve automation and environmental adaptability in excavation tasks. QL-BT is designed to maintain the structural advantages of BT, including modularity and scalability, while utilizing reinforcement learning to adapt to dynamic environmental changes in real time and optimize task priorities. The proposed algorithm employs the proximal policy optimization (PPO) deep reinforcement learning technique for efficient learning and execution of unit tasks and integrates Q-Learning to dynamically optimize task flows, addressing the inherent limitations of traditional BTs. Experimental results demonstrate that QL-BT outperforms existing algorithms in terms of learning-based adaptability, task efficiency, task quality, and environmental stability. Furthermore, it proves capable of implementing flexible and efficient task planning even in complex and rapidly changing environments. This research provides a new approach to autonomous task planning systems and establishes a foundation for further applications in multi-agent systems and various industrial scenarios, contributing to advancements in automation across diverse fields. |