Robust Optimal Tracking Control for Wheel Mobile Robot via Reinforcement Learning
DOI:
https://doi.org/10.18196/jrc.v6i3.26659Keywords:
Wheel Mobile Robot, Reinforcement Learning (RL), Robust Control, Unknown DynamicsAbstract
This paper aims to address robust optimal tracking control for a wheel mobile robot (WMR) with unknown dynamics. Firstly, the WMR system is considered a nonholonomic system with nonlinearity and input disturbance. Traditional optimal methods typically require solving the Hamilton- Jacobi- Bellman (HJB) equation or Algebraic Riccati equation (ARE), which are related to minimizing a cost function. However, these methods become increasingly difficult or even impossible to implement for high nonlinear systems such as the WMR in practical applications. To overcome this challenge, a Reinforcement Learning (RL) algorithm is designed to learn the solution of the HJB equation by using the input-output system data collected from the WMR during the data collection process. Consequently, the WMR can achieve optimal trajectory tracking without knowledge of the dynamic system. Finally, a simulation built in MATLAB software is given to show the effectiveness of the robust controller for WMR under the influence of uncertainties and input disturbance.
References
A. S. Lafmejani, H. Farivarnejad, and S. Berman, "Adaptation of gradient-based navigation control for holonomic robots to nonholonomic robots," IEEE Robotics and Automation Letters, vol. 6, no. 1, pp. 191-198, 2020, doi: 10.1109/LRA.2020.3037855.
G. Peng et al., "Pose estimation based on wheel speed anomaly detection in monocular visual-inertial SLAM," IEEE Sensors Journal, vol. 21, no. 10, pp. 11692-11703, 2020, doi: 10.1109/JSEN.2020.3011945.
H.-W. Chae, J.-H. Choi, and J.-B. Song, "Robust and autonomous stereo visual-inertial navigation for non-holonomic mobile robots," IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 9613-9623, 2020, doi: 10.1109/TVT.2020.3004163.
D. Huang et al., "Disturbance observer-based robust control for trajectory tracking of wheeled mobile robots," Neurocomputing, vol. 198, pp. 74-79, 2016.
B. S. Park et al., "Adaptive neural sliding mode control of nonholonomic wheeled mobile robots with model uncertainty," IEEE Transactions on Control Systems Technology, vol. 17, no. 1, pp. 207-214, 2008, doi: 10.1109/TCST.2008.922584.
Z. Chen et al., "Adaptive-neural-network-based trajectory tracking control for a nonholonomic wheeled mobile robot with velocity constraints," IEEE Transactions on Industrial Electronics, vol. 68, no. 6, pp. 5057-5067, 2020, doi: 10.1109/TIE.2020.2989711.
W. Xiao et al., "A novel adaptive robust control for trajectory tracking of mobile robot with uncertainties," Journal of Vibration and Control, vol. 30, no. 5-6, pp. 1313-1325, 2024, doi: 10.1177/10775463231161847.
N. T. Binh et al., "An adaptive backstepping trajectory tracking control of a tractor trailer wheeled mobile robot," International Journal of Control, Automation and Systems, vol. 17, pp. 465-473, 2019, doi: 10.1007/s12555-017-0711-0.
S. T. Dang et al., "Adaptive backstepping hierarchical sliding mode control for 3-wheeled mobile robots based on RBF neural networks," Electronics, vol. 12, no. 11, p. 2345, 2023.
J. Zhai and Z. Song, "Adaptive sliding mode trajectory tracking control for wheeled mobile robots," International Journal of Control, vol. 92, no. 10, pp. 2255-2262, 2019, doi: 10.1080/00207179.2018.1436194.
K. Nath et al., "Event-triggered sliding-mode control of two wheeled mobile robot: an experimental validation," IEEE Journal of Emerging and Selected Topics in Industrial Electronics, vol. 2, no. 3, pp. 218-226, 2021, doi: 10.1109/JESTIE.2021.3087965.
Z. B. Moudoud, H. Aissaoui, and M. Diany, "Extended state observer-based finite-time adaptive sliding mode control for wheeled mobile robot," Journal of Control and Decision, vol. 9, no. 4, pp. 465-476, 2022, doi: 10.1080/23307706.2021.2024458.
H. Xie et al., "Finite-time tracking control for nonholonomic wheeled mobile robot using adaptive fast nonsingular terminal sliding mode," Nonlinear Dynamics, vol. 110, no. 2, pp. 1437-1453, 2022, doi: 10.1007/s11071-022-07682-2.
S. Peng and W. Shi, "Adaptive fuzzy output feedback control of a nonholonomic wheeled mobile robot," IEEE Access, vol. 6, pp. 43414-43424, 2018, doi: 10.1109/ACCESS.2018.2862163.
P. Bozek et al., "Neural network control of a wheeled mobile robot based on optimal trajectories," International Journal of Advanced Robotic Systems, vol. 17, no. 2p. 1729881420916077, 2020, doi: 10.1177/1729881420916.
H. Huang et al,. "Robust neural network–based tracking control and stabilization of a wheeled mobile robot with input saturation," International Journal of Robust and Nonlinear Control, vol. 29, no. 2, pp. 375-392, 2019, doi: 10.1002/rnc.4396.
G. Wang et al., "Neural network-based adaptive motion control for a mobile robot with unknown longitudinal slipping," Chinese Journal of Mechanical Engineering, vol. 32, no. 1, p. 61, 2019, doi: 10.1007/s40314-017-0538-6.
N. S. Ahmad, "Robust H∞-fuzzy logic control for enhanced tracking performance of a wheeled mobile robot in the presence of uncertain nonlinear perturbations," Sensors, vol. 20, no. 13, p. 3673, 2020, doi: 10.3390/s20133673.
A. Al-Tamimi, F. L. Lewis, and M. Abu-Khalaf, "Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 38, no. 4, pp. 943-949, 2008, doi: 10.1109/TSMCB.2008.926614.
C. Mu et al., "Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation," Neurocomputing, vol. 260, pp. 432-442, 2017, doi: 10.1016/j.neucom.2017.04.043.
B. Kiumarsi et al., "Optimal tracking control of unknown discrete-time linear systems using input-output measured data," IEEE transactions on cybernetics, vol. 45, no. 12, pp. 2770-2779, 2015, doi: 10.1109/TCYB.2014.2384016.
D. Vrabie et al., "Adaptive optimal control for continuous-time linear systems based on policy iteration," Automatica, vol. 45, no. 2, pp. 477-484, 2009, doi: 10.1016/j.automatica.2008.08.017.
D. Vrabie and F. Lewis, "Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems," Neural Networks, vol. 22, no. 3, pp. 237-246, 2009, doi: 10.1016/j.neunet.2009.03.008.
H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, "Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems," Automatica, vol. 50, no. 1, pp. 193-202, 2014.
Q. Song, H. Ge, J. Caverlee, and X. Hu, “Tensor completion algorithms in big data analytics,” arXiv, vol. 13, no. 1, 2017.
G. Xiao et al., "Data‐driven optimal tracking control for a class of affine non‐linear continuous‐time systems with completely unknown dynamics," IET Control Theory & Applications, vol. 10, no. 6, pp. 700-710, 2016.
Y. Zhu, D. Zhao, and X. Li, "Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics," IET Control Theory & Applications, vol. 10, no. 12, pp. 1339-1347, 2016.
H. Modares, F. L. Lewis, and Z. P. Jiang, "${H} _ {{infty}} $ tracking control of completely unknown continuous-time systems via off-policy reinforcement learning," IEEE transactions on neural networks and learning systems, vol. 26, no. 10, pp. 2550-2562, 2015, doi: 10.1109/TNNLS.2015.2441749.
A. Azzabi and K. Nouri, "Design of a robust tracking controller for a nonholonomic mobile robot based on sliding mode with adaptive gain," International journal of advanced robotic systems, vol. 18, no. 1, 2021.
L. Xin et al., "Robust adaptive tracking control of wheeled mobile robot," Robotics and Autonomous Systems, vol. 78, pp. 36-48, 2016, doi: 10.1016/j.robot.2016.01.002.
O. Tutsoy, D. E. Barkana, and H. Tugal, "Design of a completely model free adaptive control in the presence of parametric, non-parametric uncertainties and random control signal delay," ISA transactions, vol. 76, pp. 67-77, 2018, doi: 10.1016/j.isatra.2018.03.002.
M. Szeremeta and M. Szuster, "Neural tracking control of a four-wheeled mobile robot with mecanum wheels," Applied Sciences, vol. 12, no. 11, p. 5322, 2022.
L. Li et al., "Trajectory tracking control for a wheel mobile robot on rough and uneven ground," Mechatronics, vol. 83, p. 102741, 2022, doi: 10.1016/j.mechatronics.2022.102741.
A. Andreev and O. Peregudova, "On the trajectory tracking control of a wheeled mobile robot based on a dynamic model with slip," 2020 15th International Conference on Stability and Oscillations of Nonlinear Control Systems (Pyatnitskiy's Conference)(STAB), 2020, doi: 10.1109/STAB49150.2020.9140714.
H. Cen and B. K. Singh, "Nonholonomic wheeled mobile robot trajectory tracking control based on improved sliding mode variable structure," Wireless Communications and Mobile Computing, vol. 2021, no. 1, p. 2974839, 2021, doi: 10.1155/2021/2974839.
L. Ding, S. Li, Y. -J. Liu, H. Gao, C. Chen, and Z. Deng, "Adaptive Neural Network-Based Tracking Control for Full-State Constrained Wheeled Mobile Robotic System," in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 47, no. 8, pp. 2410-2419, Aug. 2017, doi: 10.1109/TSMC.2017.2677472.
X. Gao et al., "A hybrid tracking control strategy for nonholonomic wheeled mobile robot incorporating deep reinforcement learning approach," IEEE Access, vol. 9, pp. 15592-15602, 2021, doi: 10.1109/ACCESS.2021.3053396.
N. Hassan and A. Saleem, "Analysis of trajectory tracking control algorithms for wheeled mobile robots," 2021 IEEE Industrial Electronics and Applications Conference (IEACon), 2021, doi: 10.1109/IEACon51066.2021.9654675.
S.-H. Tsai et al., "A sensor fusion based nonholonomic wheeled mobile robot for tracking control," Sensors, vol. 20, no. 24, p. 7055, 2020, doi: 10.3390/s20247055.
H. Xie et al., "Robust tracking control of a differential drive wheeled mobile robot using fast nonsingular terminal sliding mode," Computers & Electrical Engineering, vol. 96, p. 107488, 2021.
H. R. Shafei and M. Bahrami, "Trajectory tracking control of a wheeled mobile robot in the presence of matched uncertainties via a composite control approach," Asian Journal of Control, vol. 23, no. 6, pp. 2805-2823, 2021, doi: 10.1002/asjc.2418.
L. Zhao et al., "Double-loop tracking control for a wheeled mobile robot with unmodeled dynamics along right angle roads," ISA transactions, vol. 136, pp. 525-534, 2023.
N. Hassan and A. Saleem, "Neural network-based adaptive controller for trajectory tracking of wheeled mobile robots," IEEE Access, vol. 10, pp. 13582-13597, 2022, doi: 10.1109/ACCESS.2022.3146970.
Z. Shao and J. Zhang, "Vision-based adaptive trajectory tracking control of wheeled mobile robot with unknown translational external parameters," IEEE/ASME Transactions on Mechatronics, vol. 29, no. 1, pp. 358-365, 2023, doi: 10.1109/TMECH.2023.3278027.
X. Yue et al., "Path tracking control of skid-steered mobile robot on the slope based on fuzzy system and model predictive control," International Journal of Control, Automation and Systems, vol. 20, no. 4, pp. 1365-1376, 2022, doi: 10.1007/s12555-021-0203-0.
C. Shen et al., "Trajectory tracking control for wheeled mobile robot subject to generalized torque constraints," Transactions of the Institute of Measurement and Control, vol. 45, no. 7, pp. 1258-1270, 2023, doi: 10.1177/01423312221127478.
X. Gao, L. Yan, and C. Gerada,"Modeling and analysis in trajectory tracking control for wheeled mobile robots with wheel skidding and slipping: Disturbance rejection perspective," Actuators, vol. 10, no. 9, 2021, doi 10.3390/act10090222.
D. Wang et al., "Sliding mode observer-based model predictive tracking control for Mecanum-wheeled mobile robot," ISA transactions, vol. 151, pp. 51-61, 2024.
K. Nath, M. K. Bera, and S. Jagannathan, "Concurrent learning-based neuroadaptive robust tracking control of wheeled mobile robot: An event-triggered design," IEEE Transactions on Artificial Intelligence, vol. 4, no. 6, pp. 1514-1525, 2022, doi: 10.1109/TAI.2022.3207133.
Q. Geng et al., "A dynamic controller design for trajectory tracking control of wheeled mobile robot under stochastic denial of service attacks," IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 69, no. 8, pp. 3560-3564, 2022, doi: 10.1109/TCSII.2022.3168304.
H. Xie, J. Zheng, R. Chai, and H. T. Nguyen, “Robust tracking control of a differential drive wheeled mobile robot using fast nonsingular terminal sliding mode,” Computers & Electrical Engineering, vol. 96, p. 107488, 2021.
H. Yang et al., "Trajectory tracking for a wheeled mobile robot with an omnidirectional wheel on uneven ground," IET control theory & applications, vol. 14, no. 7, pp. 921-929, 2020, doi: 10.1049/iet-cta.2019.1074.
Y. Wu and Y. Wang, "Asymptotic tracking control of uncertain nonholonomic wheeled mobile robot with actuator saturation and external disturbances," Neural Computing and Applications, vol. 32, no. 12, pp. 8735-8745, 2020, doi: 10.1007/s00521-019-04373-9
M. Cui et al., "Adaptive control for simultaneous tracking and stabilization of wheeled mobile robot with uncertainties," Journal of Intelligent & Robotic Systems, vol. 108, no. 3, p. 46, 2023, doi: 10.1007/s10846-023-01908-0.
J. Bai et al., "Trajectory tracking control for wheeled mobile robots with kinematic parameter uncertainty," International Journal of Control, Automation and Systems, vol. 20, no. 5, pp. 1632-1639, 2022, doi: 10.1007/s12555-021-0212-z.
B. Qin et al., "Enhanced extended state observer based prescribed time tracking control of wheeled mobile robot with slipping and skidding," International Journal of Robust and Nonlinear Control, vol. 34, no. 11, pp. 7314-7331, 2024, doi: 10.1002/rnc.7347.
J. Zhang et al., "Finite-time global trajectory tracking control for uncertain wheeled mobile robots," IEEE Access, vol. 8, pp. 187808-187813, 2020, doi: 10.1109/ACCESS.2020.3030633.
L. Zhao, J. Jin, and J. Gong, "Robust zeroing neural network for fixed-time kinematic control of wheeled mobile robot in noise-polluted environment," Mathematics and Computers in Simulation, vol. 185, pp. 289-307, 2021, doi: 10.1016/j.matcom.2020.12.030.
W. Yuan et al., "Differential flatness-based adaptive robust tracking control for wheeled mobile robots with slippage disturbances," ISA transactions, vol. 144, pp. 482-489, 2024, doi: 10.1016/j.isatra.2023.11.008.
F. Wang et al., "Adaptive visually servoed tracking control for wheeled mobile robot with uncertain model parameters in complex environment," Complexity, vol. 2020, no. 1, p. 8836468, 2020, doi: 10.1155/2020/8836468.
L. Li et al., "Trajectory tracking control for wheeled mobile robots based on nonlinear disturbance observer with extended Kalman filter," Journal of the Franklin Institute, vol. 357, no. 13, pp. 8491-8507, 2020, doi: 10.1016/j.jfranklin.2020.04.043.
J. Bai et al., "Trajectory tracking control for wheeled mobile robots subject to longitudinal slippage," Asian Journal of Control, 2025, doi: 10.1002/asjc.3608.
T. Ding et al., "Trajectory tracking of redundantly actuated mobile robot by MPC velocity control under steering strategy constraint," Mechatronics, vol. 84, p. 102779, 2022, doi: 10.1016/j.mechatronics.2022.102779.
H. Ye and S. Wang, "Trajectory tracking control for nonholonomic wheeled mobile robots with external disturbances and parameter uncertainties," International Journal of Control, Automation and Systems, vol. 18, no. 12, pp. 3015-3022, 2020, doi: 10.1007/s12555-019-0643-y.
H. Zhang et al., "Nonsingular recursive-structure sliding mode control for high-order nonlinear systems and an application in a wheeled mobile robot," ISA transactions, vol. 130, pp. 553-564, 2022, doi: 10.1016/j.isatra.2022.04.021.
I. Matraji, K. Al-Wahedi, and A. Al-Durra, "Higher-order super-twisting control for trajectory tracking control of skid-steered mobile robot," IEEE Access, vol. 8, pp. 124712-124721, 2020, doi: 10.1109/ACCESS.2020.3007784.
C.-G. Yun et al., "Trajectory tracking control of a three-wheeled omnidirectional mobile robot using disturbance estimation compensator by RBF neural network," Journal of the Brazilian Society of Mechanical Sciences and Engineering, vol. 45, no. 8, p. 432, 2023, doi: 10.1007/s40430-023-04340-5.
S. Yang et al., "A RISE-based asymptotic prescribed performance trajectory tracking control of two-wheeled self-balancing mobile robot," Nonlinear Dynamics, vol. 112, no. 17, pp. 15327-15348, 2024, doi: 10.1007/s11071-024-09569-w.
Z. Sun et al., "Trajectory-tracking control of Mecanum-wheeled omnidirectional mobile robots using adaptive integral terminal sliding mode," Computers & Electrical Engineering, vol. 96, p. 107500, 2021, doi: 10.1016/j.compeleceng.2021.107500.
S.-L. Dai et al., "Adaptive image-based moving-target tracking control of wheeled mobile robots with visibility maintenance and obstacle avoidance," IEEE Transactions on Control Systems Technology, 32, no. 2, pp. 488-501, 2023, doi: 10.1109/TCST.2023.3331553.
J.-J. Zhang et al., "Trajectory tracking control of nonholonomic wheeled mobile robots using model predictive control subjected to Lyapunov-based input constraints," International Journal of Control, Automation and Systems, vol. 20, no. 5, pp. 1640-1651, 2022, doi: 10.1007/s12555-019-0814-x.
P. Guo et al., "Adaptive trajectory tracking of wheeled mobile robot based on fixed-time convergence with uncalibrated camera parameters," ISA transactions, vol. 99, pp. 1-8, 2020, doi: 10.1016/j.isatra.2019.09.021.
K. Liu et al., "Adaptive sliding mode based disturbance attenuation tracking control for wheeled mobile robots," International Journal of Control, Automation and Systems, vol. 18, no. 5, pp. 1288-1298, 2020, doi: 10.1007/s12555-019-0262-7.
Z. Han et al., "Adaptive tracking control of two-wheeled mobile robots under denial-of-service attacks, " ISA transactions, vol. 141, pp. 365-376, 2023, doi: 10.1016/j.isatra.2023.06.022.
M. Cui, H. Liu, X. Wang, and W. Liu, “Adaptive control for simultaneous tracking and stabilization of wheeled mobile robot with uncertainties,” Journal of Intelligent & Robotic Systems, vol. 108, no. 3, p. 46, 2023.
R. Deng et al., "A trajectory tracking control algorithm of nonholonomic wheeled mobile robot," 2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM), 2021, doi: 10.1109/ICARM52023.2021.9536154.
B. Moudoud, H. Aissaoui, and M. Diany, "Fuzzy adaptive sliding mode controller for electrically driven wheeled mobile robot for trajectory tracking task," Journal of Control and Decision, vol. 9, no. 1, pp. 71-79, 2022, doi: 10.1080/23307706.2021.1912665.
J. Bai et al., "Trajectory tracking controller design for wheeled Mobile robot with velocity and torque constraints," International Journal of Systems Science, vol. 55, no. 14, pp. 2825-2837, 2024, doi: 10.1080/00207721.2024.2354844.
H. Pang et al., "Adaptive sliding mode attitude control of two-wheel mobile robot with an integrated learning-based RBFNN approach," Neural Computing and Applications, vol. 34, no. 17, pp. 14959-14969, 2022, doi: 10.1007/s00521-022-07304-3.
X. Zou, T. Zhao, and S. Dian, "Finite-time adaptive interval type-2 fuzzy tracking control for Mecanum-wheel mobile robots," International Journal of Fuzzy Systems, vol. 24, no. 3, pp. 1570-1585, 2022, doi: 10.1007/s40815-021-01211-w.
T. D. Tran, T. T. Nguyen, V. T. Duong, H. H. Nguyen, and T. T. Nguyen, “Parameter-adaptive event-triggered sliding mode control for a mobile robot,” Robotics, vol. 11, no. 4, p. 78, 2022.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ngoc Trung Dang, Duc Trinh Duong, Duc Trinh Duong
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
