Hybrid Path Planning for Wheeled Mobile Robot Based on RRT-star Algorithm and Reinforcement Learning Method
DOI:
https://doi.org/10.18196/jrc.v6i4.27678Keywords:
Wheeled Mobile Robots, Reinforcement Learning, Rrtstar, Path PlanningAbstract
In the field of wheeled mobile robots (WMRs), path planning is a critical concern. WMRs employ advanced algorithms to find out the feasible path from a starting point to a specific destination. The paper proposes efficient and optimal path planning for WMRs, integrating collision avoidance strategies and smoothed techniques to determine the best route during navigation. The proposed hybrid path planning consists of improved RRTstar algorithm and reinforcement learning method. Therefore, the RRT* algorithm employs random sampling in conjunction with a reinforcement learning model to purposefully guide the sampling process towards areas that demonstrate an increased likelihood of successful navigation completion. The proposed RRTstar-RL algorithm generates significantly shorter trajectories compared to the traditional RRT and RRTstar methods. Specifically, the path length with the proposed algorithm is 11.323 meters, while the lengths for RRT and RRTstar are 15.74 and 14.40 meters, respectively. Moreover, the optimization of computation time, especially when using pre-trained data, greatly speeds up the path-finding calculation process. In particular, the time needed to generate the optimal path with the RRTstar-RL algorithm is 2.02 times faster than that of RRTstar and 1.6 times faster than RRT. Finally, the proposed RRTstar-RL algorithm has been successfully verified for feasibility and effectively meets numerous objectives established during simulations and validation experiments.
References
V. T. Nguyen, D. N. Duong, and T. V. Dang, “Optimal Two-Wheeled Self-Balancing Mobile Robot Strategy of Navigation using Adaptive Fuzzy controller-based KD-SegNet,” Intelligent Service Robotics, pp. 1-25, 2025, doi: 10.1007/s11370-025-00606-0.
T. T. Le, K. K. Phung Cong, and T. V. Dang, “An Improved Coverage Path planning for Service robots based on Backtracking method,” MM Science Journal, vol. 10, pp. 7464-7468, 2024, doi: 10.17973/MMSJ.2024_10_2024063.
A. Abadi et al., “Robust Tracking Control of Wheeled Mobile Robot Based on Differential Flatness and Sliding Active Disturbance Rejection Control: Simulations and Experiments,” Sensor, vol. 24, no. 2849, 2024, doi: 10.3390/s24092849.
M. Y. Silaa, A. Bencherif, and O. Barambones, “Indirect Adaptive Control Using Neural Network and Discrete Extended Kalman Filter for Wheeled Mobile Robot,” Actuators, vol. 13, no. 51, 2024, doi: 10.3390/act13020051.
J. Lin et al., “Combined Localization Method for Multimodal Wheel-Track Robots in Sheltered Space,” IEEE Access, vol. 12, pp. 47271-47282, 2024, doi: 10.1109/ACCESS.2024.3364068.
B. Kazed and A. Guessoum, “A Lyapunov based posture controller for a differential drive mobile robot,” IAES International Journal of Robotics and Automation (IJRA), vol. 13, no. 1, pp. 1-10, 2024, doi: 10.11591/ijra.v13i1.
S. Sachan and P. M. Pathak, “Addressing unpredictable movements of dynamic obstacles with deep reinforcement learning to ensure safe navigation for omni-wheeled mobile robot,” Proc. I. Mech. E. Part C: J. Mechanical Engineering Science, vol. 239, no. 4, pp. 1267-1293, 2024, doi: 10.1177/09544062241281115.
S. Yang et. al., “A RISE-based asymptotic prescribed performance trajectory tracking control of two-wheeled self-balancing mobile robot,” Nonlinear Dyn., vol. 112, pp. 15327-15348, 2024, doi: 10.1007/s11071-024-09569-w.
H. Xue, S. Lu, and C. Zhang, “An Adaptive Control Based on Improved Gray Wolf Algorithm for Mobile Robots,” Applied Sciences, vol. 14, no. 7092, 2024, doi: 10.3390/app14167092.
C. Li and Z. Li, “Dynamic Modeling and Disturbance-Observer-Enhanced Control for Mecanum-Wheeled Vehicles Under Load and Noise Disturbance,” Mathematics, vol. 13, no. 789, 2025, doi: 10.3390/math13050789.
T. V. Dang and N. T. Bui, “Multi-Scale Fully Convolutional Network-Based Semantic Segmentation for Mobile Robot Navigation,” Electronics, vol. 12, no. 3, p. 533, 2023, doi: 10.3390/electronics12030533.
T. V. Dang and N. T. Bui, “Obstacle Avoidance Strategy for Mobile Robot based on Monocular Camera,” Electronics, vol. 12, no. 8, p. 1932, 2023, doi: 10.3390/electronics12081932.
H. T. Linh and T. V. Dang, “An ultra-fast Semantic Segmentation Model for AMR's Path Planning,” Journal of Robotics and Control, vol. 4, no. 3, pp. 424-430, 2023, doi: 10.18196/jrc.v4i3.18758.
T. V. Dang and N. T. Bui, “Design the Abnormal Object Detection System Using Template Matching and Subtract Background Algorithm,” Proceedings of the 3rd Annual International Conference on Material, Machines and Methods for Sustainable Development, pp. 87-95, 2024, doi: 10.1007/978-3-031-57460-3_10.
T. V. Dang, N. N. Bui, and N. T. Bui, “Binary-SegNet: Efficient Convolutional Architecture for Semantic Segmentation Based on Monocular Camera,” From Smart City to Smart Factory for Sustainable Future: Conceptual Framework, Scenarios, and Multidiscipline Perspectives, pp 275-285, 2024, doi: 10.1007/978-3-031-65656-9_28.
V. T. Nguyen, N. N. Bui, D. M. C. Tran, P. X. Tan, and T. V. Dang, “FDE-Net: Lightweight Depth Estimation for Monocular Cameras,” The 13th International Symposium on Information and Communication Technology (SOICT 2024), pp. 3-13, 2025, doi: 10.1007/978-981-96-4282-3_1.
T. V. Dang, V. D. Ngo, M. Q. Ngo, and N. T. Bui, “OD-CT3D: Object Detection Model based on CentreTrack 3D for Mobile Robot Global Path Planning,” 5th International Conference on Intelligent Systems & Networks, 2025.
Y. Zheng et al., “Adaptive fuzzy sliding mode control of uncertain nonholonomic wheeled mobile robot with external disturbance and actuator saturation,” Information Sciences, vol. 663, no. 120303, 2024.
Y. Wu and W. Yu, “Asymptotic tracking control of uncertain nonholonomic wheeled mobile robot with actuator saturation and external disturbances,” Neural Computing and Applications, vol. 32, no. 2, 2020, doi: 10.1007/s00521-019-04373-9.
A. D. Nguyen, T. D. Vu, Q. A. Vu, and T. V. Dang, “Research on Modeling and Object Tracking for Robot Arm based on Deep Reinforcement Learning,” MM Science Journal, vol. 6, pp. 8459-8463, 2025, doi: 10.17973/MMSJ.2025_06_2025059.
B. Shi et al., “An intelligence enhancement method for USV navigation visual measurement based on variable gradient soft-threshold correction,” Measurement, vol. 242, no. 116201, 2025, doi: 10.1016/j.measurement.2024.116201.
T. V. Dang, “Optimization Hybrid Path Planning based on A-star Algorithm combining with DWA,” MM Science Journal, vol. 10, pp. 7551-7555, 2024, doi: 10.17973/MMSJ.2024_10_2024077.
T. V. Dang and D. S. Nguyen, “Optimal Navigation Based on Improved A* Algorithm for Mobile Robot,” Intelligent Systems and Networks, pp. 574-580, 2023, doi: 10.1007/978-981-99-4725-6_68.
T. P. Nguyen, H. Nguyen, and N. Q. T. Ha, “Towards sustainable scheduling of a multi-automated guided vehicle system for collision avoidance,” Computers and Electrical Engineering, vol. 120, no. 109824, 2024, doi: 10.1016/j.compeleceng.2024.109824.
C. C. Huang, C. H. Huang, and J. S. Shaw, “Development of an AMR Applying Cartographer Combined with Visual Odometry for Navigation,” Journal of Applied Science and Engineering, vol. 28, no 1, pp. 35-40, 2024, doi: 10.6180/jase.202501_28(1).0004.
Z. Lin, L. Lu, Y. Yuan, and H. Zhao, “A novel robotic path planning method in grid map context based on D* lite algorithm and deep learning,” J. Circuits Syst. Comput., vol. 33, no. 4, p. 2450057, 2023, doi: 10.1142/S0218126624500579.
L. Liu et al., “Global dynamic path planning fusion algorithm combining jump-A* Algorithm and dynamic window approach,” IEEE Access, vol. 9, pp. 19632-19638, 2021, doi: 10.1109/ACCESS.2021.3052865.
Z. Xunyu, T. Jun, H. Huosheng, and P. Xiafu, “Hybrid path planning based on safe A* Algorithm and adaptive window approach for mobile robot in large-scale dynamic environment,” J. Intell. Rob. Syst., vol. 99, no. 2, pp. 65-77, 2020, doi: 10.1007/s10846-019-01112-z.
C. C. Hsu, Y. J. Chen, M. C. Lu, and L. S. An, “Hybrid path planning incorporating global and local search for Mobile robot,” Conference Towards Autonomous Robotic Systems, vol. 7429, 2012, doi: 10.1007/978-3-642-32527-4_50.
M. Imran and F. Kunwar, “A Hybrid path planning technique developed by integrating global and local path planner,” 2016 International Conference on Intelligent Systems Engineering (ICISE), pp. 118-122, 2016, doi: 10.1109/INTELSE.2016.7475172.
L. S. Liu et al., “Path planning for smart car based on Dijkstra algorithm and dynamic window approach,” Wirel. Commun. Mob. Comput., vol. 4, pp. 1-12, 2021, doi: 10.1088/10.1155/2021/8881684.
R. Song, Y. Liu, and R. Bucknall, “Smoothed A∗ algorithm for practical unmanned surface vehicle path planning,” Appl. Ocean Res., vol. 83, no. 6, pp. 9-20, 2019, doi: 10.1016/j.apor.2018.12.001.
L. Zhao, G. Li, and H. Zhang, “Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance,” IEEE Open Journal of Intelligent Transportation Systems, vol. 5, pp. 422-432, 2024, doi: 10.1109/OJITS.2024.3424587.
Y. Zhu, J. Zhu, and P. Zhang, “Local obstacle avoidance control for multi-axle and multi-steering-mode wheeled robot based on window-zone division strategy,” Robotics and Autonomous Systems, vol. 183, no. 104843, 2025, doi: 10.1016/j.robot.2024.104843.
L. Xiang, X. Li, H. Liu, and P. Li, “Parameter Fuzzy Self-Adaptive Dynamic Window Approach for Local Path Planning of Wheeled Robot,” IEEE Open Journal of Intelligent Transportation Systems, vol. 3, pp. 1-6, 2021, doi: 10.1109/OJITS.2021.3137931.
M. Kobayashi and N. Motoi, “Local Path Planning: Dynamic Window Approach with Virtual Manipulators Considering Dynamic Obstacles,” IEEE Access, vol. 10, pp. 17018-17029, 2022, doi: 10.1109/ACCESS.2022.3150036.
M. Kobayashi, H. Zuski, T. Nakamura, and N. Motoi, “Local Path Planning: Dynamic Window Approach With Q-Learning Considering Congestion Environments for Mobile Robot,” IEEE Access, vol. 11, pp. 96733-96742, 2023, doi: 10.1109/ACCESS.2023.3311023.
M. Kramer and T. Bertram, “Improving Local Trajectory Optimization by Enhanced Initialization and Global Guidance,” IEEE Access, vol. 10, pp. 29633-29645, 2022, doi: 10.1109/ACCESS.2022.3159233.
T. V. Dang, “Research and design of a path planning using an improved RRT* algorithm for an autonomous mobile robot,” MM Science Journal, vol. 10, pp. 6712-6716, 2023, doi: 10.17973/MMSJ.2023_10_2023051.
T. V. Dang, “Autonomous mobile robot path planning based on enhanced A* algorithm integrating with time elastic band,” MM Science Journal, vol. 10, pp. 6717-6722, 2023, doi: 10.17973/MMSJ.2023_10_2023052.
T. V. Dang, D. S. Nguyen, and N. T. Bui, “Hybrid Path Planning for Mobile Robot based on Improved A* Fusion Dynamic Window Approach,” Proceedings of the International Conference on Intelligent Systems and Networks, pp. 82-88, 2024, doi: 10.1007/978-981-97-5504-2_10.
Y. Sun et al., “Local Path Planning for Mobile Robots Based on Fuzzy Dynamic Window Algorithm,” Sensor, vol. 23, no. 8260, 2023, doi.: 10.3390/ s23198260
Y. Chen, G. Bai, Y. Zhan, X. Hu, and J. Liu, “The USV Based on Improved ACO-APF Hybrid Algorithm with Adaptive Early-Warning,” IEEE Access, vol. 9, pp. 40728-40742, 2021, doi: 10.1109/ACCESS.2021.3062375.
Y. Ji, L. Ni, C. Zhao, C. Lei, and Y. Du, “TriPField: A 3D Potential Field Model and Its Applications to Local Path Planning of Autonomous Vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 3, 2023, doi: 10.1109/TITS.2022.3231259.
D. Li, W. Yin, W. E. Wong, M. Jian, and M. Chau, “Quality-Oriented Hybrid Path Planning Based on A∗ and Q-Learning for Unmanned Aerial Vehicle,” IEEE Access, vol. 10, pp. 7664-7674, 2021, doi: 10.1109/ACCESS.2021.3139534.
C. Wang, X. Yang, and H. Li, “Improved Q-Learning Applied to Dynamic Obstacle Avoidance and Path Planning,” IEEE Access, vol. 10, pp. 92879-92888, 2022, doi: 10.1109/ACCESS.2022.3203072.
Y. Liu, C. Wang, and H. Wu, “Mobile Robot Path Planning Based on Kinematically Constrained A-Star Algorithm and DWA Fusion Algorithm,” Mathematics, vol. 11, no. 4552, 2023, doi: 10.3390/math11214552.
Y. Feng, W. Zhang, and J. Zhu, “Application of an Improved A* Algorithm for the Path Analysis of Urban Multi-Type Transportation Systems,” Appl. Sci., vol. 13, no. 13090, 2023, doi: 10.3390/app132413090.
T. V. Dang and P. X. Tan, “Hybrid Mobile Robot Path Planning Using Safe JBS-A*B Algorithm and Improved DWA Based on Monocular Camera,” Journal of Intelligent & Robotic Systems, vol. 110, no. 151, pp. 1-21, 2024, doi: 10.1007/s10846-024-02179-z.
H. Esmaiel, G. Zhao, Z. A. Qasem, J. Qi, and H. Sun, “Double-Layer RRT* Objective Bias Anytime Motion Planning Algorithm,” Robotics, vol. 13, no. 41, 2024, doi: 10.3390/robotics13030041.
H. Wang, X. Zhou, and J. Li, “Improved RRT* Algorithm for Disinfecting Robot Path Planning,” Sensors, vol. 24, no. 1520, 2024, doi: 10.3390/s24051520.
Q. Zhang, Y. Liu, and J. Qin, “An Informed-Bi-Quick RRT* Algorithm Based on Offlne Sampling: Motion Planning Considering Multiple Constraints for a Dual-Arm Cooperative System,” Actuators, vol. 13, no. 75, 2024, doi: 10.3390/act13020075.
H. Han, J. Wang, L. Kuang, X. Han, and H. Xue, “Improved Robot Path Planning Method Based on Deep Reinforcement Learning,” Sensor, vol. 23, no. 5622, 2023, doi: 10.3390/s23125622.
C. H. Nguyen, Q. A. Vu, K. K. Phung Cong, T. V. Dang, “Optimal obstacle avoidance strategy using deep reinforcement learning based on stereo camera,” MM Science Journal, vol. 10, pp. 7556-7561, 2024, doi: 10.17973/MMSJ.2024_10_2024078.
C. Copurkaya, E. Meriç, F. P. Akbulut, and C. Catal, “A multi-pretraining U-Net architecture for semantic segmentation,” Signal Image and Video Processing, vol. 19, no. 8, 2025, doi: 10.1007/s11760-025-04125-4.
K. Rezaee et al., “Hand gestures classification of sEMG signals based on BiLSTM-metaheuristic optimization and hybrid U-Net-MobileNetV2 encoder architecture,” Scientific Reports, vol. 14, no. 1, 2024, doi: 10.1038/s41598-024-82676-1.
Q. Zhu et al., “A study on expression recognition based on improved mobilenetV2 network,” Scientific Reports, vol. 14, no. 1, 2024, doi: 10.1038/s41598-024-58736-x.
Z. Li, “Enhancing Tea Leaf Disease Identification with Lightweight MobileNetV2,” Computers, Materials & Continua, vol. 80, no. 1, pp. 679-694, 2024, doi: 10.32604/cmc.2024.051526.
T. V. Dang, D. M. C. Tran, and P. X. Tan, “IRDC-Net: Lightweight Semantic Segmentation Network Based on Monocular Camera for Mobile Robot Navigation,” Sensors, vol. 23, no. 15, p. 6907, 2023. doi: 10.3390/s23156907
T. V. Dang, X. T. Phan, and N. N. Bui, “KD-SegNet: Efficient Semantic Segmentation Network with Knowledge Distillation Based on Monocular Camera,” Computers, Materials & Continua, vol. 82, no. 2, pp. 2001-2026, doi: 10.32604/cmc.2025.060605.
M. Zhang, S. Liu, Q. Zhou, and X. Han, “A novel path planning scheme based on Fast-IBi-RRT* algorithm for industrial robots,” Applied Intelligence, vol. 55, no. 11, 2025, doi: 10.1007/s10489-025-06694-w.
S. Lei et al., “Research on improved RRT path planning algorithm based on multi-strategy fusion,” Scientific Reports, vol. 15, no. 1, 2025, doi: 10.1038/s41598-025-92675-5.
V. T. Nguyen, C. D. Do, T. V. Dang, T. L. Bui, and P. X. Tan, “A Comprehensive RGB-D Dataset for 6D Pose Estimation for Industrial Robots Pick and Place: Creation and Real-World Validation,” Results in Engineering, vol. 24, no. 103459, 2024, doi: 10.1016/j.rineng.2024.103459.
T. V. Dang, D. M. C. Tran, N. N. Bui, and P. X. Tan, “ELDE-Net: Efficient Light-weight Depth Estimation Network for Deep Reinforcement Learning-based Mobile Robot Path Planning,” Computers, Materials & Continua, 2025.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Hoang-Long Pham, Nhu-Nghia Bui, Thai-Viet Dang
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
