Improving Control Performance of Tilt-Rotor VTOL UAV with Model-Based Reward and Multi-Agent Reinforcement Learning

Ugur M., YEŞİLDİREK A.

Aerospace, cilt.12, sa.9, 2025 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 12 Sayı: 9
  • Basım Tarihi: 2025
  • Doi Numarası: 10.3390/aerospace12090814
  • Dergi Adı: Aerospace
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Anahtar Kelimeler: dynamic reference-based reward mechanism, multi-agent RL, reinforcement learning, soft actor-critic algorithm, tilt-rotor UAV
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

Tilt-rotor Vertical Takeoff and Landing Unmanned Aerial Vehicles (TR-VTOL UAVs) combine fixed-wing and rotary-wing configurations, offering optimized flight planning but presenting challenges due to their complex dynamics and uncertainties. This study investigates a multi-agent reinforcement learning (RL) control system utilizing Soft Actor-Critic (SAC) modules, which are designed to independently control each input with a tailored reward mechanism. By implementing a novel reward structure based on a dynamic reference response region, the multi-agent design improves learning efficiency by minimizing data redundancy. Compared to other control methods such as Actor-Critic Neural Networks (AC NN), Proximal Policy Optimization (PPO), Nonsingular Terminal Sliding Mode Control (NTSMC), and PID controllers, the proposed system shows at least a 30% improvement in transient performance metrics—including RMSE, rise time, settling time, and maximum overshoot—under both no wind and constant 20 m/s wind conditions, representing an extreme scenario to evaluate controller robustness. This approach has also reduced training time by 80% compared to single-agent systems, lowering energy consumption and environmental impact.