Three-Dimensional Coordination Control of Multi-UAV for Partially Observable Multi-Target Tracking

Vincentius Charles Maynad, Yurid Eka Nugraha, Abdullah Alkaff

Abstract


This research deals with multi-UAV systems to track partially observable multi-targets in noisy three-dimensional environments, which are commonly encountered in defense and surveillance systems. It is a far extension from previous research which focused mainly on two-dimensional, fully observable, and/or perfect measurement settings. The targets are modeled as linear time-invariant systems with Gaussian noise and the pursuers UAV are represented in a standard six-degree-of-freedom model. Necessary equations to describe the relationship between observations regarding the target and the pursuers states are derived and represented as the Gauss-Markov model. Partially observable targets require the pursuers to maintain belief values for target positions. In the presence of a noisy environment, an extended Kalman filter is used to estimate and update those beliefs. A Decentralized Multi-Agent Reinforcement Learning (MARL) algorithm known as soft Double Q-Learning is proposed to learn the coordination control among the pursuers. The algorithm is enriched with an entropy regulation to train a certain stochastic policy and enable interactions among pursuers to foster cooperative behavior. The enrichment encourages the algorithm to explore wider and unknown search areas which is important for multi-target tracking systems. The algorithm was trained before it was deployed to complete several scenarios. The experiments using various sensor capabilities showed that the proposed algorithm had higher success rates compared to the baseline algorithm. A description of the many distinctions between two-dimensional and three-dimensional settings is also provided.

Keywords


Coordination Control; Extended Kalman Filter; Multi-Agent Reinforcement Learning; Multi-Target Tracking; Multi-UAV System.

Full Text:

PDF

References


K. Guo, X. Li, and L. Xie, “Simultaneous cooperative relative localization and distributed formation control for multiple UAVs,” Sci. China Inf. Sci., vol. 63, no. 1, pp. 2019–2021, 2020, doi: 10.1007/s11432-018-9603-y.

Y. Cao and Y. Sun, “Necessary and sufficient conditions for consensus of third-order discrete-time multi-agent systems in directed networks,” J. Appl. Math. Comput., vol. 57, no. 1–2, pp. 199–210, 2018, doi: 10.1007/s12190-017-1101-8.

W. Zhou, J. Li, Z. Liu, and L. Shen, “Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning,” Chinese J. Aeronaut., vol. 35, no. 7, pp. 100–112, 2022, doi: 10.1016/j.cja.2021.09.008.

Z. Ma and J. Chen, “Multi-UAV Urban Logistics Task Allocation Method Based on MCTS,” Drones, vol. 7, no. 11, 2023, doi: 10.3390/drones7110679.

M. Zhang and C. Pan, “Hierarchical Optimization Scheduling Algorithm for Logistics Transport Vehicles Based on Multi-Agent Reinforcement Learning,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 3, pp. 3108–3117, 2023, doi: 10.1109/TITS.2023.3337334.

H. R. Lee and T. Lee, “Multi-agent reinforcement learning algorithm to solve a partially-observable multi-agent problem in disaster response,” Eur. J. Oper. Res., vol. 291, no. 1, pp. 296–308, 2021, doi: 10.1016/j.ejor.2020.09.018.

M. A. Blais and M. A. Akhloufi, “Drone Swarm Coordination Using Reinforcement Learning for Efficient Wildfires Fighting,” SN Comput. Sci., vol. 5, no. 3, 2024, doi: 10.1007/s42979-024-02650-6.

T. A. Karaguzel, V. Retamal, N. Cambier, and E. Ferrante, “From Shadows to Light: A Swarm Robotics Approach with Onboard Control for Seeking Dynamic Sources in Constrained Environments,” IEEE Robot. Autom. Lett., vol. 9, no. 1, pp. 127–134, 2024, doi: 10.1109/LRA.2023.3331897.

R. K. Lee, C. A. Kitts, M. A. Neumann, and R. T. McDonald, “Multiple UAV Adaptive Navigation for Three-Dimensional Scalar Fields,” IEEE Access, vol. 9, pp. 122626–122654, 2021, doi: 10.1109/ACCESS.2021.3107854.

M. Lee et al., “A Study on the Advancement of Intelligent Military Drones: Focusing on Reconnaissance Operations,” IEEE Access, vol. 12, pp. 55964–55975, 2024, doi: 10.1109/ACCESS.2024.3390035.

Y. Liu, J. Liu, Z. He, Z. Li, Q. Zhang, and Z. Ding, “A Survey of Multi-Agent Systems on Distributed Formation Control,” Unmanned Syst., pp. 1–14, Mar. 2023, doi: 10.1142/S2301385024500274.

Y. Liu, J. Hu, and Y. Li, “Quantized Formation Control of Heterogeneous Nonlinear Multi-Agent Systems with Switching Topology,” J. Syst. Sci. Complex., vol. 36, no. 6, pp. 2382–2397, 2023, doi: 10.1007/s11424-023-2387-2.

J. Liao et al., “UAV swarm formation reconfiguration control based on variable-stepsize MPC-APCMPIO algorithm,” Sci. China Inf. Sci., vol. 66, no. 11, p. 212207, 2023, doi: 10.1007/s11432-022-3735-5.

Z. Pang, Y. Fu, H. Guo, and J. Sun, “Analysis of Stealthy False Data Injection Attacks Against Networked Control Systems: Three Case Studies,” J. Syst. Sci. Complex., vol. 36, no. 4, pp. 1407–1422, 2023, doi: 10.1007/s11424-022-2120-6.

Z. Zhao, J. Chen, B. Xin, L. Li, K. Jiao, and Y. Zheng, “Learning Scalable Task Assignment with Imperative-Priori Conflict Resolution in Multi-UAV Adversarial Swarm Defense Problem,” J. Syst. Sci. Complex., vol. 37, no. 1, pp. 369–388, 2024, doi: 10.1007/s11424-024-4029-8.

N. Li, Z. Su, H. Ling, M. Karatas, and Y. Zheng, “Optimization of Air Defense System Deployment Against Reconnaissance Drone Swarms,” Complex Syst. Model. Simul., vol. 3, no. 2, pp. 102–117, 2023, doi: 10.23919/CSMS.2023.0003.

G. Wu, T. Xu, Y. Sun, and J. Zhang, “Review of multiple unmanned surface vessels collaborative search and hunting based on swarm intelligence,” Int. J. Adv. Robot. Syst., vol. 19, no. 2, pp. 1–20, 2022, doi: 10.1177/17298806221091885.

S. Ishtiaq, X. Wang, S. Hassan, A. Mohammad, A. A. Alahmadi, and N. Ullah, “Three-Dimensional Multi-Target Tracking Using Dual-Orthogonal Baseline Interferometric Radar,” Sensors, vol. 22, no. 19. 2022, doi: 10.3390/s22197549.

X. Qu, W. Gan, D. Song, and L. Zhou, “Pursuit-evasion game strategy of USV based on deep reinforcement learning in complex multi-obstacle environment,” Ocean Eng., vol. 273, p. 114016, 2023, doi: 10.1016/j.oceaneng.2023.114016.

A. A. Firmansyah and A. Alkaff, “Enhancing Autonomous Ground Vehicle Positioning and Navigation through Cascaded Multi-Sensor Integration,” 2023 Int. Conf. Adv. Mechatronics, Intell. Manuf. Ind. Autom. ICAMIMIA 2023 - Proc., pp. 436–441, 2023, doi: 10.1109/ICAMIMIA60881.2023.10427562.

T. Zhang, L. Yu, Z.-L. Zhou, and L. Wang, “Decision-making for air combat maneuvering based on hybrid algorithm,” Xi Tong Gong Cheng Yu Dian Zi Ji Shu/Systems Eng. Electron., vol. 35, pp. 1445–1450, Jul. 2013, doi: 10.3969/j.issn.1001-506X.2013.07.15.

J. M. Gongora-Torres, C. Vargas-Rosales, A. Aragón-Zavala, and R. Villalpando-Hernandez, “Elevation Angle Characterization for LEO Satellites: First and Second Order Statistics,” Applied Sciences, vol. 13, no. 7. 2023, doi: 10.3390/app13074405.

R. G. Raju and S. K. Kashyap, “3D Localisation of Target using Elevation Angle Algorithm with the use of Ground Radars,” Def. Sci. J., vol. 70, pp. 260–271, 2020.

K. E. Hoque and H. Aljamaan, “Impact of hyperparameter tuning on machine learning models in stock price forecasting,” IEEE Access, vol. 9, pp. 163815–163830, 2021, doi: 10.1109/ACCESS.2021.3134138.

N. Sutarna, C. Tjahyadi, P. Oktivasari, M. Dwiyaniti, and T. Tohazen, “Hyperparameter Tuning Impact on Deep Learning Bi-LSTM for Photovoltaic Power Forecasting,” J. Robot. Control (JRC), vol. 5, no. 3, 2024, doi: 10.18196/jrc.v5i3.21120.

D. C. E. Saputra, A. Ma’arif, and K. Sunat, “Optimizing Predictive Performance: Hyperparameter Tuning in Stacked Multi-Kernel Support Vector Machine Random Forest Models for Diabetes Identification,” J. Robot. Control, vol. 4, no. 6, pp. 896–904, 2023, doi: 10.18196/jrc.v4i6.20898.

M. N. A. Al-Hamadani, M. A. Fadhel, L. Alzubaidi, and H. Balazs, “Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review,” Sensors, vol. 24, no. 8, p. 2461, 2024, doi: 10.3390/s24082461.

S. Mukhopadhyay and B. Jain, “Multi-agent markov decision processes with limited agent communication,” IEEE Int. Symp. Intell. Control - Proc., pp. 7–12, 2001, doi: 10.1109/isic.2001.971476.

S. Huang, H. Zhang, and Z. Huang, “Multi-UAV Collision Avoidance using Multi-Agent Reinforcement Learning with Counterfactual Credit Assignment,” arXiv preprint arXiv:2204.08594, 2022.

M. Kayaalp, F. Ghadieh, and A. H. Sayed, “Policy Evaluation in Decentralized POMDPs With Belief Sharing,” IEEE Open J. Control Syst., vol. 2, pp. 125–145, 2023, doi: 10.1109/ojcsys.2023.3277760.

A. K. Shakya, G. Pillai, and S. Chakrabarty, “Reinforcement learning algorithms: A brief survey,” Expert Syst. Appl., vol. 231, p. 120495, 2023, doi: 10.1016/j.eswa.2023.120495.

M. Yuan, J. Shan, and K. Mi, “Deep Reinforcement Learning Based Game-Theoretic Decision-Making for Autonomous Vehicles,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 818–825, 2022, doi: 10.1109/LRA.2021.3134249.

S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” J. F. Robot., vol. 37, no. 3, pp. 362–386, 2020, doi: 10.1002/rob.21918.

Ó. Pérez-Gil et al., “Deep reinforcement learning based control for Autonomous Vehicles in CARLA,” Multimed. Tools Appl., vol. 81, no. 3, pp. 3553–3576, 2022, doi: 10.1007/s11042-021-11437-3.

S. Han et al., “A Multi-Agent Reinforcement Learning Approach for Safe and Efficient Behavior Planning of Connected Autonomous Vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 5, pp. 3654–3670, 2023, doi: 10.1109/TITS.2023.3336670.

B. Memarian and T. Doleck, “A scoping review of reinforcement learning in education,” Comput. Educ. Open, vol. 6, p. 100175, 2024, doi: 10.1016/j.caeo.2024.100175.

B. Fahad Mon, A. Wasfi, M. Hayajneh, A. Slim, and N. Abu Ali, “Reinforcement Learning in Education: A Literature Review,” Informatics, vol. 10, no. 3, pp. 1–22, 2023, doi: 10.3390/informatics10030074.

A. Charpentier, R. Élie, and C. Remlinger, “Reinforcement Learning in Economics and Finance,” Comput. Econ., vol. 62, no. 1, pp. 425–462, 2023, doi: 10.1007/s10614-021-10119-4.

F. Soleymani and E. Paquet, “Deep graph convolutional reinforcement learning for financial portfolio management – DeepPocket,” Expert Syst. Appl., vol. 182, 2021, doi: 10.1016/j.eswa.2021.115127.

M. Liu, F. R. Yu, Y. Teng, V. C. M. Leung, and M. Song, “Performance optimization for blockchain-enabled industrial internet of things (iiot) systems: A deep reinforcement learning approach,” IEEE Trans. Ind. Informatics, vol. 15, no. 6, pp. 3559–3570, 2019, doi: 10.1109/TII.2019.2897805.

X. He, K. Wang, H. Huang, T. Miyazaki, Y. Wang, and S. Guo, “Green Resource Allocation Based on Deep Reinforcement Learning in Content-Centric IoT,” IEEE Trans. Emerg. Top. Comput., vol. 8, no. 3, pp. 781–796, 2020, doi: 10.1109/TETC.2018.2805718.

Y. Dai, D. Xu, S. Maharjan, Z. Chen, Q. He, and Y. Zhang, “Blockchain and Deep Reinforcement Learning Empowered Intelligent 5G beyond,” IEEE Netw., vol. 33, no. 3, pp. 10–17, 2019, doi: 10.1109/MNET.2019.1800376.

C. H. Liu, Q. Lin, and S. Wen, “Blockchain-enabled data collection and sharing for industrial iot with deep reinforcement learning,” IEEE Trans. Ind. Informatics, vol. 15, no. 6, pp. 3516–3526, 2019, doi: 10.1109/TII.2018.2890203.

R. Nian, J. Liu, and B. Huang, “A review On reinforcement learning: Introduction and applications in industrial process control,” Comput. Chem. Eng., vol. 139, p. 106886, 2020, doi: 10.1016/j.compchemeng.2020.106886.

X. Fang, J. Wang, G. Song, Y. Han, Q. Zhao, and Z. Cao, “Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling,” Energies, vol. 13, no. 1. 2020, doi: 10.3390/en13010123.

A. Subramanian, S. Chitlangia, and V. Baths, “Reinforcement learning and its connections with neuroscience and psychology,” Neural Networks, vol. 145, pp. 271–287, 2022, doi: 10.1016/j.neunet.2021.10.003.

F. Ye, Y. Li, X. Wang, N. Nedjah, P. Zhang, and H. Shi, “Parameters tuning of multi-model database based on deep reinforcement learning,” J. Intell. Inf. Syst., vol. 61, no. 1, pp. 167–190, 2023, doi: 10.1007/s10844-022-00762-0.

J. García and D. Shafie, “Teaching a humanoid robot to walk faster through Safe Reinforcement Learning,” Eng. Appl. Artif. Intell., vol. 88, p. 103360, 2020, doi: 10.1016/j.engappai.2019.103360.

T. Kobayashi and T. Sugino, “Reinforcement learning for quadrupedal locomotion with design of continual–hierarchical curriculum,” Eng. Appl. Artif. Intell., vol. 95, no. August, p. 103869, 2020, doi: 10.1016/j.engappai.2020.103869.

J. Ibarz, J. Tan, C. Finn, M. Kalakrishnan, P. Pastor, and S. Levine, “How to train your robot with deep reinforcement learning: lessons we have learned,” Int. J. Rob. Res., vol. 40, no. 4–5, pp. 698–721, Jan. 2021, doi: 10.1177/0278364920987859.

M. D. Al-Masrur Khan et al., “A systematic review on reinforcement learning-based robotics within the last decade,” IEEE Access, vol. 8, pp. 176598–176623, 2020, doi: 10.1109/ACCESS.2020.3027152.

K. Zhu and T. Zhang, “Deep reinforcement learning based mobile robot navigation: A review,” Tsinghua Sci. Technol., vol. 26, no. 5, pp. 674–691, 2021, doi: 10.26599/TST.2021.9010012.

Y. Zhu, Z. Wang, C. Chen, and D. Dong, “Rule-Based Reinforcement Learning for Efficient Robot Navigation With Space Reduction,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 2, pp. 846–857, 2022, doi: 10.1109/TMECH.2021.3072675.

X. Wang et al., “Deep Reinforcement Learning: A Survey,” IEEE Trans. Neural Networks Learn. Syst., vol. 35, no. 4, pp. 5064–5078, 2024, doi: 10.1109/TNNLS.2022.3207346.

L. Canese et al., “Multi-Agent Reinforcement Learning: A Review of Challenges and Applications,” Applied Sciences, vol. 11, no. 11. 2021, doi: 10.3390/app11114948.

J. Castellini, F. A. Oliehoek, R. Savani, and S. Whiteson, “Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning,” Auton. Agent. Multi. Agent. Syst., vol. 35, no. 2, p. 25, 2021, doi: 10.1007/s10458-021-09506-w.

B. Liu, Y. Xie, L. Feng, and P. Fu, “Correcting biased value estimation in mixing value-based multi-agent reinforcement learning by multiple choice learning,” Eng. Appl. Artif. Intell., vol. 116, p. 105329, 2022, doi: 10.1016/j.engappai.2022.105329.

Y. Deng, Z. Wang, X. Chen, and Y. Zhang, “Boosting Multi-agent Reinforcement Learning via Contextual Prompting,” J. Mach. Learn. Res., vol. 24, no. 399, pp. 1–34, 2023.

J.-Y. Lee, A. Rahman, S. Huang, A. D. Smith, and S. Katipamula, “On-policy learning-based deep reinforcement learning assessment for building control efficiency and stability,” Sci. Technol. Built Environ., vol. 28, no. 9, pp. 1150–1165, Sep. 2022, doi: 10.1080/23744731.2022.2094729.

N. Wang, Z. Li, X. Liang, Y. Hou, and A. Yang, “A Review of Deep Reinforcement Learning Methods and Military Application Research,” Math. Probl. Eng., vol. 2023, p. 7678382, 2023, doi: 10.1155/2023/7678382.

X. Zhuang, D. Li, Y. Wang, X. Liu, and H. Li, “Optimization of high-speed fixed-wing UAV penetration strategy based on deep reinforcement learning,” Aerosp. Sci. Technol., vol. 148, p. 109089, 2024, doi: 10.1016/j.ast.2024.109089.

J. Fan, D. Dou, and Y. Ji, “Impact-Angle Constraint Guidance and Control Strategies Based on Deep Reinforcement Learning,” Aerospace, vol. 10, no. 11, 2023, doi: 10.3390/aerospace10110954.

A. Garg and S. S. Jha, “Deep deterministic policy gradient based multi-UAV control for moving convoy tracking,” Eng. Appl. Artif. Intell., vol. 126, p. 107099, 2023, doi: 10.1016/j.engappai.2023.107099.

J. Kim, D. Jang, and H. J. Kim, “Distributed Multi-agent Target Search and Tracking With Gaussian Process and Reinforcement Learning,” Int. J. Control. Autom. Syst., vol. 21, no. 9, pp. 3057–3067, 2023, doi: 10.1007/s12555-022-0555-0.

A. Puente-Castro, D. Rivero, E. Pedrosa, A. Pereira, N. Lau, and E. Fernandez-Blanco, “Q-Learning based system for Path Planning with Unmanned Aerial Vehicles swarms in obstacle environments[Formula presented],” Expert Syst. Appl., vol. 235, p. 121240, 2024, doi: 10.1016/j.eswa.2023.121240.

Z. Qu, X. Zhao, H. Xu, H. Tang, J. Wang, and B. Li, “An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking,” Sensors, vol. 22, no. 18, 2022, doi: 10.3390/s22186972.

Q. Long, Z. Zhou, A. Gupta, F. Fang, Y. Wu, and X. Wang, "Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning," arXiv preprint arXiv:2003.10423, 2020.

X. Wang and X. Fang, “A multi-agent reinforcement learning algorithm with the action preference selection strategy for massive target cooperative search mission planning,” Expert Syst. Appl., vol. 231, p. 120643, 2023, doi: 10.1016/j.eswa.2023.120643.

X. Li, J. Ren, and Y. Li, “Multi-mode filter target tracking method for mobile robot using multi-agent reinforcement learning,” Eng. Appl. Artif. Intell., vol. 127, p. 107398, 2024, doi: 10.1016/j.engappai.2023.107398.

S. Li, Y. Jia, F. Yang, Q. Qin, H. Gao, and Y. Zhou, “Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning,” IEEE Access, vol. 10, pp. 91385–91396, 2022, doi: 10.1109/ACCESS.2022.3199070.

R. Su, Z. Gong, D. Zhang, C. Li, Y. Chen, and R. Venkatesan, “An Adaptive Asynchronous Wake-Up Scheme for Underwater Acoustic Sensor Networks Using Deep Reinforcement Learning,” IEEE Trans. Veh. Technol., vol. 70, no. 2, pp. 1851–1865, 2021, doi: 10.1109/TVT.2021.3055065.

L. Yue, R. Yang, J. Zuo, M. Yan, X. Zhao, and M. Lv, “Factored Multi-Agent Soft Actor-Critic for Cooperative Multi-Target Tracking of UAV Swarms,” Drones, vol. 7, no. 3, p. 150, 2023, doi: 10.3390/drones7030150.

K. Su and F. Qian, “Multi-UAV Cooperative Searching and Tracking for Moving Targets Based on Multi-Agent Reinforcement Learning,” Appl. Sci., vol. 13, no. 21, p. 11905, 2023, doi: 10.3390/app132111905.

C. D. Hsu, H. Jeong, G. J. Pappas, and P. Chaudhari, “Scalable Reinforcement Learning Policies for Multi-Agent Control,” IEEE Int. Conf. Intell. Robot. Syst., pp. 4785–4791, 2021, doi: 10.1109/IROS51168.2021.9636344.

X. Zhang, W. Yue, and W. Tang, “Research on Scheme Design and Decision of Multiple Unmanned Aerial Vehicle Cooperation Anti-Submarine Based on Knowledge-Driven Soft Actor-Critic,” Appl. Sci., vol. 13, no. 20, p. 11527, 2023, doi: 10.3390/app132011527.

Z. Fan, Y. Xu, Y. Kang, and D. Luo, “Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning,” Machines, vol. 10, no. 11, pp. 1–18, 2022, doi: 10.3390/machines10111033.

Y. Cao, Y.-X. Kou, Z.-W. Li, and A. Xu, “Autonomous Maneuver Decision of UCAV Air Combat Based on Double Deep Q Network Algorithm and Stochastic Game Theory,” Int. J. Aerosp. Eng., vol. 2023, p. 3657814, 2023, doi: 10.1155/2023/3657814.

J. H. Bae, H. Jung, S. Kim, S. Kim, and Y.-D. Kim, “Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment,” IEEE Access, vol. 11, pp. 26427–26440, 2023, doi: 10.1109/ACCESS.2023.3257849.

J. Xianyong, M. Hou, G. Wu, Z. Ma, and Z. Tao, “Research on Maneuvering Decision Algorithm Based on Improved Deep Deterministic Policy Gradient,” IEEE Access, vol. 10, pp. 92426–92445, 2022, doi: 10.1109/ACCESS.2022.3202918.

D. Hu, R. Yang, J. Zuo, Z. Zhang, J. Wu, and Y. Wang, “Application of Deep Reinforcement Learning in Maneuver Planning of Beyond-Visual-Range Air Combat,” IEEE Access, vol. 9, pp. 32282–32297, 2021, doi: 10.1109/ACCESS.2021.3060426.

S. Li et al., “Multi-UAV Cooperative Air Combat Decision-Making Based on Multi-Agent Double-Soft Actor-Critic,” Aerospace, vol. 10, no. 7. 2023, doi: 10.3390/aerospace10070574.

W. Kong, D. Zhou, Z. Yang, Y. Zhao, and K. Zhang, “UAV Autonomous Aerial Combat Maneuver Strategy Generation with Observation Error Based on State-Adversarial Deep Deterministic Policy Gradient and Inverse Reinforcement Learning,” Electronics, vol. 9, no. 7. 2020, doi: 10.3390/electronics9071121.

A. White and A. Karimoddini, “Event-based diagnosis of flight maneuvers of a fixed-wing aircraft,” Reliab. Eng. Syst. Saf., vol. 193, p. 106609, 2020, doi: 10.1016/j.ress.2019.106609.

H. Jeong, B. Schlotfeldt, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas, “Learning Q-network for Active Information Acquisition,” IEEE Int. Conf. Intell. Robot. Syst., pp. 6822–6827, 2019, doi: 10.1109/IROS40897.2019.8968173.

J. Hu, L. Wang, T. Hu, C. Guo, and Y. Wang, “Autonomous Maneuver Decision Making of Dual-UAV Cooperative Air Combat Based on Deep Reinforcement Learning,” Electronics, vol. 11, no. 3. 2022, doi: 10.3390/electronics11030467.

Y. Li, J. Shi, W. Jiang, W. Zhang, and Y. Lyu, “Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm,” Def. Technol., vol. 18, no. 9, pp. 1697–1714, 2022, doi: 10.1016/j.dt.2021.09.014.

N. Sadeghzadeh-Nokhodberiz, A. Can, R. Stolkin, and A. Montazeri, “Dynamics-Based Modified Fast Simultaneous Localization and Mapping for Unmanned Aerial Vehicles with Joint Inertial Sensor Bias and Drift Estimation,” IEEE Access, vol. 9, pp. 120247–120260, 2021, doi: 10.1109/ACCESS.2021.3106864.

M. Zaheer, S. Kottur, S. Ravanbhakhsh, B. Póczos, R. Salakhutdinov, and A. J. Smola, “Deep sets,” Adv. Neural Inf. Process. Syst., pp. 3392–3402, 2017.

B. D. Ziebart, A. Maas, J. A. Bagnell, and A. K. Dey, “Maximum Entropy Inverse Reinforcement Learning,” Proc. 23rd AAAI Conf. Artif. Intell. AAAI 2008, pp. 1433–1438, 2008.

B. Jang, M. Kim, G. Harerimana, and J. W. Kim, “Q-Learning Algorithms: A Comprehensive Classification and Applications,” IEEE Access, vol. 7, pp. 133653–133667, 2019, doi: 10.1109/ACCESS.2019.2941229.

H. Jiang, G. Li, J. Xie, and J. Yang, “Action Candidate Driven Clipped Double Q-Learning for Discrete and Continuous Action Tasks,” IEEE Trans. Neural Networks Learn. Syst., vol. 35, no. 4, pp. 5269–5279, 2024, doi: 10.1109/TNNLS.2022.3203024.




DOI: https://doi.org/10.18196/jrc.v5i5.22560

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Vincentius Charles Maynad, Yurid Eka Nugraha, Abdullah Alkaff

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik