Key Factors that Negatively Affect Performance of Imitation Learning for Autonomous Driving

Estiko Rijanto, Nelson Changgraini, Roni Permana Saputra, Zainal Abidin

Abstract


Conditional imitation learning (CIL) has proven superior to other autonomous driving (AD) algorithms. However, its performance evaluation through physical implementations is still limited. This work contributes a systematic evaluation to identify key factors potentially improving its performance. It modified convolutional neural network parameter values, such as reducing the number of filter channels and neuron units, and implemented the model into a vision-based autonomous vehicle (AV). The AV has front-wheel steering with an Ackermann mechanism since it is commonly used by passenger cars. Using the Inertia Measurement Unit, we measured the vehicle’s location and yaw angle along the experimental route. The AV had to move autonomously through new road sectors in the morning, afternoon, and night. First, an overall performance evaluation was carried out. The results showed a 99% success rate from 648 evaluation experiments under different conditions in which the 1% failure rate happened at new intersections. Then, a turning performance evaluation was conducted to identify key factors leading to failure at new intersections. They include fast speed, dazzling light reflection, late navigation command change instant, and the untrained turning driving pattern. The AV never failed while driving on the trained routes. It had a 100% success rate when driving slower, even under various lighting conditions and at various driving patterns, including untrained intersections. Although this study is limited to identifying key factors at three constant speeds, the results become the foundation for future research to improve CIL performance for AD, including by incorporating multimodal fusion and multi-route networks.

Keywords


Autonomous Driving; Convolutional Neural Networks; Front-Wheel Steering; Imitation Learning.

Full Text:

PDF

References


Ó. Pérez-Gil et al., “Deep reinforcement learning based control for Autonomous Vehicles in CARLA,” Multimed. Tools Appl., vol. 81, no. 3, pp. 3553–3576, 2022.

J. Laconte, A. Kasmi, R. Aufrère, M. Vaidis, and R. Chapuis, “A Survey of Localization Methods for Autonomous Vehicles in Highway Scenarios,” Sensors (Basel), vol. 22, no. 1, p. 247, 2021.

L. Fridman et al., “MIT Advanced Vehicle Technology Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction With Automation,” IEEE Access, vol. 7, pp. 102021–102038, 2019.

Y.-B. Chang, C. Tsai, C.-H. Lin, and P. Chen, “Real-Time Semantic Segmentation with Dual Encoder and Self-Attention Mechanism for Autonomous Driving,” Sensors (Basel), vol. 21, no. 23, p. 8072, 2021.

E. Horváth, C. Pozna, and M. Unger, “Real‐time lidar‐based urban road and sidewalk detection for autonomous vehicles,” Sensors, vol. 22, no. 1, 2022.

C. Sun, X. Zhang, Q. Zhou, and Y. Tian, “A Model Predictive Controller With Switched Tracking Error for Autonomous Vehicle Path Tracking,” IEEE Access, vol. 7, pp. 53103–53114, 2019.

S. Teng et al., “Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 6, pp. 3692–3711, 2023.

M. Chghaf, S. Rodriguez, and A. El Ouardi, “Camera, LiDAR and Multi-modal SLAM Systems for Autonomous Ground Vehicles: A Survey,” J. Intell. Robot Syst., vol. 105, no. 1, p. 2, 2022.

C. Liu, H. Liu, L. Han, and C. Xiang, “New Integrated Multi-Algorithm Fusion Localization and Trajectory Tracking Framework of Autonomous Vehicles under Extreme Conditions with Non-Gaussian Noises,” International Journal of Automotive Technology, vol. 24, no. 1, pp. 259–272, 2023.

Q. Song, K. Tan, P. Runeson, and S. Persson, “Critical scenario identification for realistic testing of autonomous driving systems,” Software Quality Journal, vol. 31, no. 2, pp. 441–469, 2023.

C. Gómez-Huélamo et al., “360o real-time and power-efficient 3D DAMOT for autonomous driving applications,” Multimed. Tools Appl., vol. 81, no. 19, pp. 26915–26940, 2022.

F. Tener and J. Lanir, “Investigating intervention road scenarios for teleoperation of autonomous vehicles,” Multimed. Tools Appl., pp. 1-17, 2023.

E. O. Appiah and S. Mensah, “Object detection in adverse weather condition for autonomous vehicles,” Multimed. Tools Appl., pp. 1-27, 2023.

Md. M. Rana and K. Hossain, “Connected and Autonomous Vehicles and Infrastructures: A Literature Review,” International Journal of Pavement Research and Technology, vol. 16, no. 2, pp. 264–284, 2023.

C. Chen, A. Seff, A. Kornhauser, and J. Xiao, “DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving,” Proceedings of the IEEE international conference on computer vision, pp. 2722-2730, 2015.

D. A. Pomerleau, “ALVINN: An Autonomous Land Vehicle in a Neural Network,” in Advances in Neural Information Processing Systems, vol. 1, 1988.

D. A. Pomerleau, “Efficient Training of Artificial Neural Networks for Autonomous Navigation,” Neural Comput., vol. 3, no. 1, pp. 88–97, 1991.

M. Al-Qizwini, I. Barjasteh, H. Al-Qassab, and H. Radha, “Deep learning algorithm for autonomous driving using GoogLeNet,” in 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 89–96, 2017.

Q. Liu, S. Song, H. Hu, T. Huang, C. Li, and Q. Zhu, “Extended model predictive control scheme for smooth path following of autonomous vehicles,” Frontiers of Mechanical Engineering, vol. 17, no. 1, p. 4, 2022.

U. Muller, J. Ben, E. Cosatto, B. Fleep, and Y. LeCun, “Autonomous off-road vehicle control using end-to-end learning,” Courant Institute/CBLL, Arlington, VA, USA, Tech. Rep. DARPA-IPTO Final technical Report, vol. 458, 2004.

U. Muller, J. Ben, E. Cosatto, B. Flepp, and Y. Cun, “Off-Road Obstacle Avoidance through End-to-End Learning,” in Advances in Neural Information Processing Systems, vol. 18, 2005.

M. Bojarski et al., “End to End Learning for Self-Driving Cars,” arXiv:1604.07316, 2016.

M. Bojarski et al., “Explaining how a deep neural network trained with end-to-end learning steers a car,” arXiv preprint arXiv:1704.07911, 2017.

L. A. Curiel-Ramirez et al., “End-to-End Automated Guided Modular Vehicle,” Applied Sciences, vol. 10, no. 12, p. 4400, 2020.

Y. Pan et al., “Agile Autonomous Driving using End-to-End Deep Imitation Learning,” arXiv:1709.07174, 2017.

S. Nozari, A. Krayani, P. Marin-Plaza, L. Marcenaro, D. M. Gomez, and C. Regazzoni, “Active Inference Integrated With Imitation Learning for Autonomous Driving,” IEEE Access, vol. 10, pp. 49738–49756, 2022.

Y. Pan et al., “Imitation learning for agile autonomous driving,” Int J Rob Res, vol. 39, no. 2–3, pp. 286–302, 2020.

S. Teng, L. Chen, Y. Ai, Y. Zhou, Z. Xuanyuan, and X. Hu, “Hierarchical Interpretable Imitation Learning for End-to-End Autonomous Driving,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 1, pp. 673–683, 2023.

H. Tian, C. Wei, C. Jiang, Z. Li, and J. Hu, “Personalized Lane Change Planning and Control By Imitation Learning From Drivers,” IEEE Transactions on Industrial Electronics, vol. 70, no. 4, pp. 3995–4006, 2023.

J. Ying and Y. Feng, “Full Vehicle Trajectory Planning Model for Urban Traffic Control Based on Imitation Learning,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2676, no. 7, pp. 186–198, 2022.

F. Codevilla, M. Muller, A. Lopez, V. Koltun, and A. Dosovitskiy, “End-to-End Driving Via Conditional Imitation Learning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 4693–4700, 2018.

A. Sauer, N. Savinov, and A. Geiger, “Conditional Affordance Learning for Driving in Urban Environments,” in Proceedings of The 2nd Conference on Robot Learning, vol. 87, pp. 237–252, 2018.

A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An Open Urban Driving Simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, vol. 78, pp. 1–16, 2017.

D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl, “Learning by Cheating,” in Proceedings of the Conference on Robot Learning, vol. 100, pp. 66–75, 2020.

Y. Xiao, F. Codevilla, A. Gurram, O. Urfalioglu, and A. M. Lopez, “Multimodal End-to-End Autonomous Driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 537–547, 2022.

P. S. Chib and P. Singh, “Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey,” IEEE Transactions on Intelligent Vehicles, pp. 1–18, 2023.

D. Coelho and M. Oliveira, “A Review of End-to-End Autonomous Driving in Urban Environments,” IEEE Access, vol. 10, pp. 75296–75311, 2022.

A. Tampuu, T. Matiisen, M. Semikin, D. Fishman, and N. Muhammad, “A Survey of End-to-End Driving: Architectures and Training Methods,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 4, pp. 1364–1384, 2022.

A. Amini, I. Gilitschenski, J. Phillips, J. Moseyko, R. Banerjee, S. Karaman, and D. Rus, “Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation,” IEEE Robot Autom. Lett., vol. 5, no. 2, pp. 1143–1150, 2020.

X. Wang, Z. Ning, S. Guo, and L. Wang, “Imitation Learning Enabled Task Scheduling for Online Vehicular Edge Computing,” IEEE Trans. Mob. Comput., vol. 21, no. 2, pp. 598–611, 2022.

S. Kuutti, R. Bowden, Y. Jin, P. Barber, and S. Fallah, “A Survey of Deep Learning Applications to Autonomous Vehicle Control,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 2, pp. 712–733, 2021.

W. Wang, L. Wang, C. Zhang, C. Liu, and L. Sun, “Social Interactions for Autonomous Driving: A Review and Perspectives,” Foundations and Trends® in Robotics, vol. 10, no. 3–4, pp. 198–376, 2022.

E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, “A Survey of Autonomous Driving: Common Practices and Emerging Technologies,” IEEE Access, vol. 8, pp. 58443–58469, 2020.

U. M. Gidado, H. Chiroma, N. Aljojo, S. Abubakar, S. I. Popoola, and M. A. Al-Garadi, “A survey on deep learning for steering angle prediction in autonomous vehicles,” IEEE Access, vol. 8, pp. 163797–163817, 2020.

A. O. Ly and M. Akhloufi, “Learning to Drive by Imitation: An Overview of Deep Behavior Cloning Methods,” IEEE Transactions on Intelligent Vehicles, vol. 6, no. 2, pp. 195–209, 2021.

É. Zablocki, H. Ben-Younes, P. Pérez, and M. Cord, “Explainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges,” Int. J. Comput. Vis., vol. 130, no. 10, pp. 2425–2452, 2022.

L. Arras, A. Osman, and W. Samek, “CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations,” Information Fusion, vol. 81, pp. 14–40, 2022.

M. Borg et al., “Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry,” Journal of Automotive Software Engineering, vol. 1, no. 1, p. 1, 2019.

Z. Zhang, R. Tian, R. Sherony, J. Domeyer, and Z. Ding, “Attention-Based Interrelation Modeling for Explainable Automated Driving,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 2, pp. 1564–1573, Feb. 2023.

P. P. Angelov, E. A. Soares, R. Jiang, N. I. Arnold, and P. M. Atkinson, “Explainable artificial intelligence: an analytical review,” WIREs Data Mining and Knowledge Discovery, vol. 11, no. 5, 2021.

H. Mankodiya, D. Jadav, R. Gupta, S. Tanwar, W.-C. Hong, and R. Sharma, “OD-XAI: Explainable AI-Based Semantic Object Detection for Autonomous Vehicles,” Applied Sciences, vol. 12, no. 11, p. 5310, 2022.

M. P. S. Lorente, E. M. Lopez, L. A. Florez, A. L. Espino, J. A. I. Martínez, and A. S. de Miguel, “Explaining Deep Learning-Based Driver Models,” Applied Sciences, vol. 11, no. 8, p. 3321, 2021.

J. Kim, A. Rohrbach, Z. Akata, S. Moon, T. Misu, Y. Chen, T. Darrell, and J. Canny, “Toward explainable and advisable model for self‐driving cars,” Applied AI Letters, vol. 2, no. 4, 2021.

X. Bai, X. Wang, X. Liu, Q. Liu, J. Song, N. Sebe, and B. Kim, “Explainable deep learning for efficient and robust pattern recognition: A survey of recent developments,” Pattern Recognit., vol. 120, p. 108102, 2021.

P. Angelov and E. Soares, “Towards explainable deep neural networks (xDNN),” Neural Networks, vol. 130, pp. 185–194, 2020.

K. Muhammad, A. Ullah, J. Lloret, J. Del Ser, and V. H. C. de Albuquerque, “Deep learning for safe autonomous driving: Current challenges and future directions,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4316–4336, 2020.

V. Belle and I. Papantonis, “Principles and practice of explainable machine learning,” Front Big Data, p. 39, 2021.

A. Pereira and C. Thomas, “Challenges of machine learning applied to safety-critical cyber-physical systems,” Mach. Learn. Knowl. Extr., vol. 2, no. 4, pp. 579–602, 2020.

Z. Huang, C. Lv, Y. Xing, and J. Wu, “Multi-Modal Sensor Fusion-Based Deep Neural Network for End-to-End Autonomous Driving With Scene Understanding,” IEEE Sens. J., vol. 21, no. 10, pp. 11781–11790, 2021.

L. Chen, X. Hu, B. Tang, and Y. Cheng, “Conditional DQN-Based Motion Planning With Fuzzy Logic for Autonomous Driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 4, pp. 2966–2977, 2022.

J. Chen, S. E. Li, and M. Tomizuka, “Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5068–5078, 2022.

M. Ahmed, A. Abobakr, C. P. Lim, and S. Nahavandi, “Policy-Based Reinforcement Learning for Training Autonomous Driving Agents in Urban Areas With Affordance Learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 8, pp. 12562–12571, 2022.

C. Huang, R. Zhang, M. Ouyang, P. Wei, J. Lin, J. Su, and L. Lin, “Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 12, pp. 5379–5391, 2021.

L. Le Mero, D. Yi, M. Dianati, and A. Mouzakitis, “A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 14128–14147, 2022.

J. Hawke et al., “Urban Driving with Conditional Imitation Learning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 251–257, 2020.

M. Abe. Vehicle Handling Dynamics: Theory and Application. Butterworth-Heinemann. 2015.

M. Podpora, G. P. Korbas, and A. Kawala-Janik, “YUV vs RGB-Choosing a Color Space for Human-Machine Interaction.,” in FedCSIS (Position Papers), pp. 29–34, 2014.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

H. M. Eraqi, M. N. Moustafa, and J. Honer, “Dynamic Conditional Imitation Learning for Autonomous Driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 22988–23001, 2022.

M. Cao, R. Wang, N. Chen, and J. Wang, “A Learning-Based Vehicle Trajectory-Tracking Approach for Autonomous Vehicles With LiDAR Failure Under Various Lighting Conditions,” IEEE/ASME Transactions on Mechatronics, vol. 27, no. 2, pp. 1011–1022, 2022.

R. Fan et al., “Learning Collision-Free Space Detection From Stereo Images: Homography Matrix Brings Better Data Augmentation,” IEEE/ASME Transactions on Mechatronics, vol. 27, no. 1, pp. 225–233, 2022.

K. Bayoudh, R. Knani, F. Hamdaoui, and A. Mtibaa, “A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets,” Vis. Comput., vol. 38, no. 8, pp. 2939–2970, 2022.

J. Kaur and W. Singh, “A systematic review of object detection from images using deep learning,” Multimed. Tools Appl., vol. 83, no. 4, pp. 12253–12338, 2024.




DOI: https://doi.org/10.18196/jrc.v5i1.20371

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Estiko Rijanto, Nelson Changgraini, Roni Permana Saputra, Zainal Abidin

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik