An Ultra Fast Semantic Segmentation Model for AMR’s Path Planning

Hoai-Linh Tran, Thai-Viet Dang

Abstract


Computer vision plays a significant role in mobile robot navigation due to the abundance of information extracted from digital images. On the basis of the captured images, mobile robots determine their location and proceed to the desired destination. Obstacle avoidance still requires a complex sensor system with a high computational efficiency requirement due to the complexity of the environment. This research provides a real-time solution to the issue of extracting corridor scenes from a single image. Using an ultra-fast semantic segmentation model to reduce the number of training parameters and the cost of computation. In addition, the mean Intersection over Union (mIoU) is 89%, and the high accuracy is 95%. To demonstrate the viability of the prosed method, the simulation results are contrasted to those of contemporary techniques. Finally, the authors employ the segmented image to construct the frontal view of the mobile robot in order to determine the available free areas for mobile robot path planning tasks.

Keywords


Artificial Intelligence; Computer Vision; Sematic Segmentation; Mobile Robot; Path Planning.

Full Text:

PDF

References


T. V. Dang and N. T. Bui, “Multi-scale Fully Convolutional Network based Semantic Segmentation for Mobile Robot Navigation,” Electronics, vol. 12, no. 3, p. 533, 2022.

V. Pagire and S. Mate, “Autonomous Vehicle using Computer Vision and LiDAR”, I-manager’s Journal on Embedded Systems, vol. 9, no. 2, pp. 7-14, 2021.

Q. Luo, H. Wang, Y. Zheng, and J. He, “Research on path planning of mobile robot based on improved ant colony algorithm,” Neural Computing and Applications, vol. 32, pp. 1555-1566, 2020.

T. V. Dang and N. T. Bui, “Obstacle Avoidance Strategy for Mobile Robot based on Monocular Camera,” Electronics, vol. 12, no. 8, p. 1932, 2023.

A. Gianibelli, I. Carlucho, M. D. Paula, and G. G. Acosta, “An obstacle avoidance system for mobile robotics based on the virtual force field method,” 2018 IEEE Biennial Congress of Argentina (ARGENCON), pp. 1-8, 2018, doi: 10.1109/ARGENCON.2018.8646065.

P.G. Luan and N.T. Thinh, “Real-Time Hybrid Navigation System-Based Path Planning and Obstacle Avoidance for Mobile Robots,” Appl. Sci., vol. 10, no. 10, p. 3355, 2020.

I. Maulana, A. Rusdinar, and R.A. Priramadhi, “Lidar applications for Mapping and Robot Navigation on Closed Environment,” J. Meas. Electron. Commun. Syst., vol. 4, no. 1, pp. 20-26, 2018.

X. Pan, L. Gao, A. Marinoni, B. Zhang, F. Yang, and P. Gamba, “Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network,” Remote Sens., vol. 10, no. 5, p. 743, 2018.

W. Xu, L. Yang, and S. Cao, “A Review of Semantic Segmentation Based on Context Information,” 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 494-498, 2018, doi: 10.1109/ITOEC.2018.8740714.

C. Peng, Y. Li, L. Jiao, Y. Chen, and R. Shang, “Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 2612-2626, Aug. 2019, doi: 10.1109/JSTARS.2019.2906387.

Y. Wang, Z. Sun, and W. Zhao, “Encoder- and Decoder-Based Networks Using Multiscale Feature Fusion and Nonlocal Block for Remote Sensing Image Semantic Segmentation,” in IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 7, pp. 1159-1163, July 2021, doi: 10.1109/LGRS.2020.2998680.

M. Pastorino, G. Moser, S. B. Serpico, and J. Zerubia, “Semantic Segmentation of Remote-Sensing Images Through Fully Convolutional Neural Networks and Hierarchical Probabilistic Graphical Models,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-16, 2022, doi: 10.1109/TGRS.2022.3141996.

M. Oršic’ and S. Šegvic, “Efficient semantic segmentation with pyramidal fusion,” Pattern Recognition, vol. 110, p. 107611, 2021.

A. Gupta, A. Anpalagan, L. Guan, and A. S. Khwaja, “Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues,” Array, vol. 10, p. 100057, 2021.

R. Leonard, N. Brilly, and R. Rusman, “Vision-based vanishing point detection of autonomous navigation of mobile robot for outdoor applications,” J. Mechatron. Electr. Power Veh. Technol., vol. 12, pp. 117-125, 2021.

P. Wan, S. Gao, L. Li, B. Sun, and S. Cheng, “Obstacle Avoidance Path Planning Design for Autonomous Driving Vehicles Based on an Improved Artificial Potential Field Algorithm,” Energy, vol. 12, p. 2342, 2019.

S. Feraco, S. Luciani, A. Bonfitto, N. Amati, and A. Tonoli, “A local trajectory planning and control method for autonomous vehicles based on the RRT algorithm,” 2020 AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE), pp. 1-6, 2020, doi: 10.23919/AEITAUTOMOTIVE50086.2020.9307439.

D. Qin, “Path Planning Algorithm Based on Visual Image Feature Extraction for Mobile Robots,” Mob. Inf. Syst., vol. 2022, 2022.

Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” Advances in Neural Information Processing Systems, vol. 19 pp. 153-160, 2007.

D. Erhan, P.A. Manzagol, Y. Bengio, S. Bengio, and P. Vincent, “The difficulty of training deep architectures and the effect of unsupervised pretraining,” Proceedings of the 12th IC on Artificial Intelligence and Statistics (AISTATS’09), vol. 5, pp. 153-160, 2009.

N. A. Othman, M. U. Salur, M. Karakose and I. Aydin, “An Embedded Real-Time Object Detection and Measurement of its Size,” 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), pp. 1-4, 2018, doi: 10.1109/IDAP.2018.8620812.

Y. Xiao, Z.Q. Tian, J.C. Yu, Y.S. Zhang, S. Liu, S. Du, and X. Lan, “A review of object detection based on deep learning,” Multimedia Tools and Applications, vol. 79, pp. 23729-23791, 2020.

A. Ammar, A. Koubaa, M. Ahmed, and A. Saad, “Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3,” Preprints.org, arXiv: 2019100195, 2019.

D. Michael, U.S. Patent No. 5,872,870. Washington, DC: U.S. Patent and Trademark Office. 1999.

Z. Ren, F. Fang, N. Yan, and Y. Wu, “State of the art in defect detection based on machine vision,” International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 9, no. 2, pp. 661-691, 2022.

Y. Li et al., “A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving,” in IEEE Access, vol. 8, pp. 194228-194239, 2020, doi: 10.1109/ACCESS.2020.3033289.

S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image Segmentation Using Deep Learning: A Survey,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3523-3542, 2022, doi: 10.1109/TPAMI.2021.3059968.

B. Li, Y. Shi, Z. Qi, and Z. Chen, “A Survey on Semantic Segmentation,” 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 1233-1240, 2018, doi: 10.1109/ICDMW.2018.00176.

ISPRS Working group III/4. Isprs 2D Semantic Labeling Contest. Available online: http://www2.isprs.org/commissions/comm3/wg4/ (accessed on 1 May 2023).

S.J. Fusic, K. Hariharan, R. Sitharthan, and S. Karthikeyan, “Scene terrain classification for autonomous vehicle navigation based on semantic segmentation method,” Trans. Inst. Meas. Control., vol. 44, pp. 2574-2589, 2022.

W. Sun and R. Wang, “Fully Convolutional Networks for Semantic Segmentation of Very High Resolution Remotely Sensed Images Combined With DSM,” in IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 3, pp. 474-478, March 2018, doi: 10.1109/LGRS.2018.2795531.

W. Xiaolei, H. Zirong, S. Shouhai, H. Mei, X. Le, and Z. Xiang, “A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved UNet,” Scientific Report, vol. 13, no. 1, p. 7600, 2023.

L. Rusli, B. Nurhalim, and R. Rusyadi, “Vision-based vanishing point detection of autonomous navigation of mobile robot for outdoor applications,” Journal of Mechatronics, Electrical Power, and Vehicular Technology, vol. 12, pp. 117-125, 2021.

T. V. Dang, “Smart Attendance System based on Improved Facial Recognition,” Journal of Robotics and Control, vol. 4, no.1, pp. 46-53, 2023.

T. V. Dang, “Smart home Management System with Face Recognition based on ArcFace model in Deep Convolutional Neural Network,” Journal of Robotics and Control, vol. 3, no. 6, pp. 754-761, 2022.

B. Michalski and M. P-. Wójcik, “Comparison of LeNet-5, AlexNet and GoogLeNet models in handwriting recognition,” Journal of Computer Sciences Institute, vol. 23, pp. 145-151, 2022.

E. Suherman, B. Rahman, D. Hindarto, and H. Santoso, “Implementation of ResNet-50 on End-to-End Object Detection (DETR) on Objects,” Sinkron: jurnal dan penelitian teknik informatika, vol. 8, no. 2, pp. 1085-1096, 2023

A. A. Süzen, B. Duman, and B. Şen, “Benchmark Analysis of Jetson TX2, Jetson Nano and Raspberry PI using Deep-CNN,” 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1-5, 2020, doi: 10.1109/HORA49412.2020.9152915.

Y. Nan, J. Ju, Q. Hua, H. Zhang, and B. Wang, “A-MobileNet: An approach of facial expression recognition,” Alexandria Engineering Journal, vol. 61, no. 6, pp. 4435-4444, 2021.

Z. Hong, Z. Zhi-hui, L. Ying, C. Xiao-feng, L. Xiaonan, Z. Jie, S. Kwong, Jun, and Z. Jun, “Local Binary Pattern-Based Adaptive Differential Evolution for Multimodal Optimization Problems”, IEEE Transaction on Cybernetics, vol. 50, no. 7, pp. 3343-3357, 2020.

S. Biswas, S. Kundu, and S. Das, “Inducing Niching Behavior in Differential Evolution Through Local Information Sharing,” in IEEE Transactions on Evolutionary Computation, vol. 19, no. 2, pp. 246-263, 2015, doi: 10.1109/TEVC.2014.2313659.

F. Caraffini, A. V. Kononova, D. Corne, “Infeasibility and structural bias in differential evolution,” Infomation Science, vol. 496, pp. 161–179, 2019.

Z. -J. Wang et al., “Dual-Strategy Differential Evolution With Affinity Propagation Clustering for Multimodal Optimization Problems,” in IEEE Transactions on Evolutionary Computation, vol. 22, no. 6, pp. 894-908, Dec. 2018, doi: 10.1109/TEVC.2017.2769108.

X. Li, D. Chang, T. Tian and J. Cao, “Large-Margin Regularized Softmax Cross-Entropy Loss,” in IEEE Access, vol. 7, pp. 19572-19578, 2019, doi: 10.1109/ACCESS.2019.2897692.

T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley Interscience, 2006.

J. Brownlee, Probability for Machine Learning: Discover How To Harness Uncertainty With Python. Machine Learning Mystery, 2019.

Q. Tang, M. Lécuyer, “DP-Adam: Correcting DP Bias in Adam's Second Moment Estimation,” Preprints.org, arXiv:2304.11208, pp. 1-6, 2023.

J.R. Sashank, K. Satyen, K. Sanjiv, “On the Convergence of Adam and Beyond,” Preprints.org, arXiv:1904.09237v1, pp. 1-23, 2018.

G. Chen, C. K. Qu, and P. Gong, “Anomalous diffusion dynamics of learning in deep neural networks,” Neural Networks, vol. 149, pp. 18-28, 2022.

D. P. Kingma and J. B. Adam, “A method for stochastic optimization,” Preprints.org, arXiv:1412.6980v9, 2014.

W. Liu, Y. Wen, Z. Yu, and M. Yang, “Large-Margin Softmax Loss for Convolutional Neural Networks,” Proceedings of the 33rd International Conference on Machine Learning, vol. 48, pp. 507-516, 2016.

P. Reverdy and N. E. Leonard, “Parameter Estimation in Softmax Decision-Making Models With Linear Objective Functions,” in IEEE Transactions on Automation Science and Engineering, vol. 13, no. 1, pp. 54-67, Jan. 2016, doi: 10.1109/TASE.2015.2499244.

I. Goodfellow, Y. Bengio, and A. Courville, “Softmax Units for Multinoulli Output Distributions,” Deep Learning, pp. 180–184, 2016.

B. Gao and L. Pavel, “On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning,” Preprints.org, arXiv:1704.00805, pp. 1-10, 2017.

T. Lindeberg, “A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time,” Biological Cybernetics, vol. 117, pp. 21–59, 2023.

A. C. Bergstrom, D. Conran, and D. W. Messinger, “Gaussian Blur and Relative Edge Response,” Preprints.org, arXiv:2301.00856v1, pp. 1-12, 2023.

R. Legin, A. Adam, Y. Hezaveh, and L. P. Levasseur, “Beyond Gaussian Noise: A Generalized Approach to Likelihood Analysis with Non-Gaussian Noise,” The Astrophysical Journal Letters, vol. 949, no. 2, 2023.

I. M. Bright, M. L. R. Meister, N. A. Cruzado, Z. Tiganj, E. A. Buffalo, and M.W. Howard, “A temporal record of the past with a spectrum of time constants in the monkey entorhinal cortex,” Proceedings of the National Academy of Sciences, vol. 117, no. 33, pp. 20274–20283, 2020.

C.B. Maxime, G. Floria, C. Yanjun, M. Bhairav, P. Liam, “Duckietown Environments for OpenAI Gym,” 2018. https://github.com/duckietown/gym-duckietown (accessed on 1 May 2023).

Z. Lorincz, Imitation Learning in The Duckietown environment., PhD thesis, Budapest University of Technology and Economics, Faculty of Electrical Engineering and Informatics, 2020.




DOI: https://doi.org/10.18196/jrc.v4i3.18758

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Hoai-Linh Tran, Thai-Viet Dang

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik