An Ultra Fast Semantic Segmentation Model for AMR’s Path Planning
DOI:
https://doi.org/10.18196/jrc.v4i3.18758Keywords:
Artificial Intelligence, Computer Vision, Sematic Segmentation, Mobile Robot, Path Planning.Abstract
Computer vision plays a significant role in mobile robot navigation due to the abundance of information extracted from digital images. On the basis of the captured images, mobile robots determine their location and proceed to the desired destination. Obstacle avoidance still requires a complex sensor system with a high computational efficiency requirement due to the complexity of the environment. This research provides a real-time solution to the issue of extracting corridor scenes from a single image. Using an ultra-fast semantic segmentation model to reduce the number of training parameters and the cost of computation. In addition, the mean Intersection over Union (mIoU) is 89%, and the high accuracy is 95%. To demonstrate the viability of the prosed method, the simulation results are contrasted to those of contemporary techniques. Finally, the authors employ the segmented image to construct the frontal view of the mobile robot in order to determine the available free areas for mobile robot path planning tasks.References
T. V. Dang and N. T. Bui, “Multi-scale Fully Convolutional Network based Semantic Segmentation for Mobile Robot Navigation,” Electronics, vol. 12, no. 3, p. 533, 2022.
V. Pagire and S. Mate, “Autonomous Vehicle using Computer Vision and LiDAR”, I-manager’s Journal on Embedded Systems, vol. 9, no. 2, pp. 7-14, 2021.
Q. Luo, H. Wang, Y. Zheng, and J. He, “Research on path planning of mobile robot based on improved ant colony algorithm,” Neural Computing and Applications, vol. 32, pp. 1555-1566, 2020.
T. V. Dang and N. T. Bui, “Obstacle Avoidance Strategy for Mobile Robot based on Monocular Camera,” Electronics, vol. 12, no. 8, p. 1932, 2023.
A. Gianibelli, I. Carlucho, M. D. Paula, and G. G. Acosta, “An obstacle avoidance system for mobile robotics based on the virtual force field method,” 2018 IEEE Biennial Congress of Argentina (ARGENCON), pp. 1-8, 2018, doi: 10.1109/ARGENCON.2018.8646065.
P.G. Luan and N.T. Thinh, “Real-Time Hybrid Navigation System-Based Path Planning and Obstacle Avoidance for Mobile Robots,” Appl. Sci., vol. 10, no. 10, p. 3355, 2020.
I. Maulana, A. Rusdinar, and R.A. Priramadhi, “Lidar applications for Mapping and Robot Navigation on Closed Environment,” J. Meas. Electron. Commun. Syst., vol. 4, no. 1, pp. 20-26, 2018.
X. Pan, L. Gao, A. Marinoni, B. Zhang, F. Yang, and P. Gamba, “Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network,” Remote Sens., vol. 10, no. 5, p. 743, 2018.
W. Xu, L. Yang, and S. Cao, “A Review of Semantic Segmentation Based on Context Information,” 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 494-498, 2018, doi: 10.1109/ITOEC.2018.8740714.
C. Peng, Y. Li, L. Jiao, Y. Chen, and R. Shang, “Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 2612-2626, Aug. 2019, doi: 10.1109/JSTARS.2019.2906387.
Y. Wang, Z. Sun, and W. Zhao, “Encoder- and Decoder-Based Networks Using Multiscale Feature Fusion and Nonlocal Block for Remote Sensing Image Semantic Segmentation,” in IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 7, pp. 1159-1163, July 2021, doi: 10.1109/LGRS.2020.2998680.
M. Pastorino, G. Moser, S. B. Serpico, and J. Zerubia, “Semantic Segmentation of Remote-Sensing Images Through Fully Convolutional Neural Networks and Hierarchical Probabilistic Graphical Models,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-16, 2022, doi: 10.1109/TGRS.2022.3141996.
M. Oršic’ and S. Šegvic, “Efficient semantic segmentation with pyramidal fusion,” Pattern Recognition, vol. 110, p. 107611, 2021.
A. Gupta, A. Anpalagan, L. Guan, and A. S. Khwaja, “Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues,” Array, vol. 10, p. 100057, 2021.
R. Leonard, N. Brilly, and R. Rusman, “Vision-based vanishing point detection of autonomous navigation of mobile robot for outdoor applications,” J. Mechatron. Electr. Power Veh. Technol., vol. 12, pp. 117-125, 2021.
P. Wan, S. Gao, L. Li, B. Sun, and S. Cheng, “Obstacle Avoidance Path Planning Design for Autonomous Driving Vehicles Based on an Improved Artificial Potential Field Algorithm,” Energy, vol. 12, p. 2342, 2019.
S. Feraco, S. Luciani, A. Bonfitto, N. Amati, and A. Tonoli, “A local trajectory planning and control method for autonomous vehicles based on the RRT algorithm,” 2020 AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE), pp. 1-6, 2020, doi: 10.23919/AEITAUTOMOTIVE50086.2020.9307439.
D. Qin, “Path Planning Algorithm Based on Visual Image Feature Extraction for Mobile Robots,” Mob. Inf. Syst., vol. 2022, 2022.
Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” Advances in Neural Information Processing Systems, vol. 19 pp. 153-160, 2007.
D. Erhan, P.A. Manzagol, Y. Bengio, S. Bengio, and P. Vincent, “The difficulty of training deep architectures and the effect of unsupervised pretraining,” Proceedings of the 12th IC on Artificial Intelligence and Statistics (AISTATS’09), vol. 5, pp. 153-160, 2009.
N. A. Othman, M. U. Salur, M. Karakose and I. Aydin, “An Embedded Real-Time Object Detection and Measurement of its Size,” 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), pp. 1-4, 2018, doi: 10.1109/IDAP.2018.8620812.
Y. Xiao, Z.Q. Tian, J.C. Yu, Y.S. Zhang, S. Liu, S. Du, and X. Lan, “A review of object detection based on deep learning,” Multimedia Tools and Applications, vol. 79, pp. 23729-23791, 2020.
A. Ammar, A. Koubaa, M. Ahmed, and A. Saad, “Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3,” Preprints.org, arXiv: 2019100195, 2019.
D. Michael, U.S. Patent No. 5,872,870. Washington, DC: U.S. Patent and Trademark Office. 1999.
Z. Ren, F. Fang, N. Yan, and Y. Wu, “State of the art in defect detection based on machine vision,” International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 9, no. 2, pp. 661-691, 2022.
Y. Li et al., “A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving,” in IEEE Access, vol. 8, pp. 194228-194239, 2020, doi: 10.1109/ACCESS.2020.3033289.
S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image Segmentation Using Deep Learning: A Survey,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3523-3542, 2022, doi: 10.1109/TPAMI.2021.3059968.
B. Li, Y. Shi, Z. Qi, and Z. Chen, “A Survey on Semantic Segmentation,” 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 1233-1240, 2018, doi: 10.1109/ICDMW.2018.00176.
ISPRS Working group III/4. Isprs 2D Semantic Labeling Contest. Available online: http://www2.isprs.org/commissions/comm3/wg4/ (accessed on 1 May 2023).
S.J. Fusic, K. Hariharan, R. Sitharthan, and S. Karthikeyan, “Scene terrain classification for autonomous vehicle navigation based on semantic segmentation method,” Trans. Inst. Meas. Control., vol. 44, pp. 2574-2589, 2022.
W. Sun and R. Wang, “Fully Convolutional Networks for Semantic Segmentation of Very High Resolution Remotely Sensed Images Combined With DSM,” in IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 3, pp. 474-478, March 2018, doi: 10.1109/LGRS.2018.2795531.
W. Xiaolei, H. Zirong, S. Shouhai, H. Mei, X. Le, and Z. Xiang, “A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved UNet,” Scientific Report, vol. 13, no. 1, p. 7600, 2023.
L. Rusli, B. Nurhalim, and R. Rusyadi, “Vision-based vanishing point detection of autonomous navigation of mobile robot for outdoor applications,” Journal of Mechatronics, Electrical Power, and Vehicular Technology, vol. 12, pp. 117-125, 2021.
T. V. Dang, “Smart Attendance System based on Improved Facial Recognition,” Journal of Robotics and Control, vol. 4, no.1, pp. 46-53, 2023.
T. V. Dang, “Smart home Management System with Face Recognition based on ArcFace model in Deep Convolutional Neural Network,” Journal of Robotics and Control, vol. 3, no. 6, pp. 754-761, 2022.
B. Michalski and M. P-. Wójcik, “Comparison of LeNet-5, AlexNet and GoogLeNet models in handwriting recognition,” Journal of Computer Sciences Institute, vol. 23, pp. 145-151, 2022.
E. Suherman, B. Rahman, D. Hindarto, and H. Santoso, “Implementation of ResNet-50 on End-to-End Object Detection (DETR) on Objects,” Sinkron: jurnal dan penelitian teknik informatika, vol. 8, no. 2, pp. 1085-1096, 2023
A. A. Süzen, B. Duman, and B. Şen, “Benchmark Analysis of Jetson TX2, Jetson Nano and Raspberry PI using Deep-CNN,” 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1-5, 2020, doi: 10.1109/HORA49412.2020.9152915.
Y. Nan, J. Ju, Q. Hua, H. Zhang, and B. Wang, “A-MobileNet: An approach of facial expression recognition,” Alexandria Engineering Journal, vol. 61, no. 6, pp. 4435-4444, 2021.
Z. Hong, Z. Zhi-hui, L. Ying, C. Xiao-feng, L. Xiaonan, Z. Jie, S. Kwong, Jun, and Z. Jun, “Local Binary Pattern-Based Adaptive Differential Evolution for Multimodal Optimization Problems”, IEEE Transaction on Cybernetics, vol. 50, no. 7, pp. 3343-3357, 2020.
S. Biswas, S. Kundu, and S. Das, “Inducing Niching Behavior in Differential Evolution Through Local Information Sharing,” in IEEE Transactions on Evolutionary Computation, vol. 19, no. 2, pp. 246-263, 2015, doi: 10.1109/TEVC.2014.2313659.
F. Caraffini, A. V. Kononova, D. Corne, “Infeasibility and structural bias in differential evolution,” Infomation Science, vol. 496, pp. 161–179, 2019.
Z. -J. Wang et al., “Dual-Strategy Differential Evolution With Affinity Propagation Clustering for Multimodal Optimization Problems,” in IEEE Transactions on Evolutionary Computation, vol. 22, no. 6, pp. 894-908, Dec. 2018, doi: 10.1109/TEVC.2017.2769108.
X. Li, D. Chang, T. Tian and J. Cao, “Large-Margin Regularized Softmax Cross-Entropy Loss,” in IEEE Access, vol. 7, pp. 19572-19578, 2019, doi: 10.1109/ACCESS.2019.2897692.
T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley Interscience, 2006.
J. Brownlee, Probability for Machine Learning: Discover How To Harness Uncertainty With Python. Machine Learning Mystery, 2019.
Q. Tang, M. Lécuyer, “DP-Adam: Correcting DP Bias in Adam's Second Moment Estimation,” Preprints.org, arXiv:2304.11208, pp. 1-6, 2023.
J.R. Sashank, K. Satyen, K. Sanjiv, “On the Convergence of Adam and Beyond,” Preprints.org, arXiv:1904.09237v1, pp. 1-23, 2018.
G. Chen, C. K. Qu, and P. Gong, “Anomalous diffusion dynamics of learning in deep neural networks,” Neural Networks, vol. 149, pp. 18-28, 2022.
D. P. Kingma and J. B. Adam, “A method for stochastic optimization,” Preprints.org, arXiv:1412.6980v9, 2014.
W. Liu, Y. Wen, Z. Yu, and M. Yang, “Large-Margin Softmax Loss for Convolutional Neural Networks,” Proceedings of the 33rd International Conference on Machine Learning, vol. 48, pp. 507-516, 2016.
P. Reverdy and N. E. Leonard, “Parameter Estimation in Softmax Decision-Making Models With Linear Objective Functions,” in IEEE Transactions on Automation Science and Engineering, vol. 13, no. 1, pp. 54-67, Jan. 2016, doi: 10.1109/TASE.2015.2499244.
I. Goodfellow, Y. Bengio, and A. Courville, “Softmax Units for Multinoulli Output Distributions,” Deep Learning, pp. 180–184, 2016.
B. Gao and L. Pavel, “On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning,” Preprints.org, arXiv:1704.00805, pp. 1-10, 2017.
T. Lindeberg, “A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time,” Biological Cybernetics, vol. 117, pp. 21–59, 2023.
A. C. Bergstrom, D. Conran, and D. W. Messinger, “Gaussian Blur and Relative Edge Response,” Preprints.org, arXiv:2301.00856v1, pp. 1-12, 2023.
R. Legin, A. Adam, Y. Hezaveh, and L. P. Levasseur, “Beyond Gaussian Noise: A Generalized Approach to Likelihood Analysis with Non-Gaussian Noise,” The Astrophysical Journal Letters, vol. 949, no. 2, 2023.
I. M. Bright, M. L. R. Meister, N. A. Cruzado, Z. Tiganj, E. A. Buffalo, and M.W. Howard, “A temporal record of the past with a spectrum of time constants in the monkey entorhinal cortex,” Proceedings of the National Academy of Sciences, vol. 117, no. 33, pp. 20274–20283, 2020.
C.B. Maxime, G. Floria, C. Yanjun, M. Bhairav, P. Liam, “Duckietown Environments for OpenAI Gym,” 2018. https://github.com/duckietown/gym-duckietown (accessed on 1 May 2023).
Z. Lorincz, Imitation Learning in The Duckietown environment., PhD thesis, Budapest University of Technology and Economics, Faculty of Electrical Engineering and Informatics, 2020.
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
