Autonomous Movement Control of Coaxial Mobile Robot based on Aspect Ratio of Human Face for Public Relation Activity Using Stereo Thermal Camera

Muhammad Ilhamdi Rusydi, Aulia Novira, Takayuki Nakagome, Joseph Muguro, Rio Nakajima, Waweru Njeri, Kojiro Matsushita, Minoru Sasaki

Abstract


In recent years, robots that recognize people around them and provide guidance, information, and monitoring have been attracting attention. The mainstream of conventional human recognition technology is the method using a camera or laser range finder. However, it is difficult to recognize with a camera due to fluctuations in lighting 1), and it is often affected by the recognition environment such as misrecognition 2) with a person's leg and a chair's leg with a laser range finder. Therefore, we propose a human recognition method using a thermal camera that can visualize human heat. This study aims to realize human-following autonomous movement based on human recognition. In addition, the distance from the robot to the person is measured with a stereo thermal camera that uses two thermal cameras. A coaxial two-wheeled robot that is compact and capable of super-credit turning is used as a mobile robot. Finally, we conduct an autonomous movement experiment of a coaxial mobile robot based on human recognition by combining these. We performed human-following experiments on a coaxial two-wheeled robot based on human recognition using a stereo thermal camera and confirmed that it moves appropriately to the location where the recognized person is in multiple use cases (scenarios). However, the accuracy of distance measurement by stereo vision is inferior to that of laser measurement. It is necessary to improve it in the case of movement that requires more accuracy.

Keywords


coaxial mobile robot; human following robot; recognition; stereo thermal camera;

Full Text:

PDF

References


A. Khan, B. Rinner, and A. Cavallaro, “Cooperative robots to observe moving targets: Review,” IEEE Trans. Cybern., vol. 48, no. 1, pp. 187–198, 2018, doi: 10.1109/TCYB.2016.2628161.

K. P. Sinaga and M. S. Yang, “Unsupervised K-means clustering algorithm,” IEEE Access, vol. 8, pp. 80716–80727, 2020, doi: 10.1109/ACCESS.2020.2988796.

A. J. Abougarair and M. M. Edardar, “Adaptive Neural Networks Based Robust Output Feedback Control for Nonlinear System,” Wseas Trans. Comput. Res., vol. 9, no. 1, pp. 125–136, 2021, doi: 10.37394/232018.2021.9.15.

A. Ouda and A. Mohamed, “Autonomous Fuzzy Heading Control for a Multi-Wheeled Combat Vehicle,” Int. J. Robot. Control Syst., vol. 1, no. 1, pp. 90–101, 2021, doi: 10.31763/ijrcs.v1i1.286.

M. Tahmasebi, M. Gohari, and A. Emami, “An Autonomous Pesticide Sprayer Robot with a Color-based Vision System,” Int. J. Robot. Control Syst., vol. 2, no. 1, pp. 115–123, 2022, doi: 10.31763/ijrcs.v2i1.480.

J. Smids, S. Nyholm, and H. Berkers, “Robots in the Workplace: a Threat to—or Opportunity for—Meaningful Work?,” Philos. Technol., vol. 33, no. 3, pp. 503–522, 2020, doi: 10.1007/s13347-019-00377-4.

R. Tomari, Y. Kobayashi, and Y. Kuno, “Analysis of socially acceptable smart wheelchair navigation based on head cue information,” Procedia Comput. Sci., vol. 42, pp. 198–205, 2014, doi: 10.1016/j.procs.2014.11.052.

G. D’Andréa, L. Bordenave, F. Nguyen, Y. Tao, V. Paleri, S. Temam, A. Moya-Plana, and P. Gorphe, “A prospective longitudinal study of quality of life in robotic-assisted salvage surgery for oropharyngeal cancer,” Eur. J. Surg. Oncol., vol. 48, no. 6, pp. 1243-1250, 2022, doi: 10.1016/j.ejso.2022.01.017.

C. M. Song, H. S. Bang, H. G. Kim, H. J. Park, and K. Tae, “Health-related quality of life after transoral robotic thyroidectomy in papillary thyroid carcinoma,” Surg. (United States), vol. 170, no. 1, pp. 99–105, 2021, doi: 10.1016/j.surg.2021.02.042.

R. R. Galin and R. V Meshcheryakov, “Human-robot interaction efficiency and human-robot collaboration,” in Robotics: Industry 4.0 Issues & New Intelligent Control Paradigms, pp. 55–63, 2020.

A. Dzedzickis, J. Subačiūtė-Žemaitienė, E. Šutinys, U. Samukaitė-Bubnienė, and V. Bučinskas, “Advanced Applications of Industrial Robotics: New Trends and Possibilities,” Applied Sciences, vol. 12, no. 1. 2022, doi: 10.3390/app12010135.

J. Zhang, R. Liu, K. Yin, Z. Wang, M. Gui, and S. Y. Chen, “Intelligent Collaborative Localization among Air-Ground Robots for Industrial Environment Perception,” IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9673-9681, 2018, doi: 10.1109/TIE.2018.2880727.

W. Sheng, A. Thobbi, and Y. Gu, “An Integrated Framework for Human-Robot Collaborative Manipulation,” IEEE Trans. Cybern., vol. 45, no. 10, pp. 2030–2041, 2015, doi: 10.1109/TCYB.2014.2363664.

H. Xing, L. Shi, K. Tang, S. Guo, X. Hou, Y. Liu, H. Liu, and Y. Hu, “Robust RGB-D camera and IMU fusion-based cooperative and relative close-range localization for multiple turtle-inspired amphibious spherical robots,” J. Bionic Eng., vol. 16, no. 3, pp. 442–454, 2019.

C. F. Juang, M. G. Lai, and W. T. Zeng, “Evolutionary Fuzzy Control and Navigation for Two Wheeled Robots Cooperatively Carrying an Object in Unknown Environments,” IEEE Trans. Cybern., vol. 45, no. 9, pp. 1731–1743, 2015, doi: 10.1109/TCYB.2014.2359966.

J. W. Wang, Y. Guo, M. Fahad, and B. Bingham, “Dynamic Plume Tracking by Cooperative Robots,” IEEE/ASME Trans. Mechatronics, vol. 24, no. 2, pp. 609–620, 2019, doi: 10.1109/TMECH.2019.2892292.

S.-J. Chung and J.-J. E. Slotine, “Cooperative Robot Control and Concurrent Synchronization of Lagrangian Systems,” IEEE Trans. Robot., vol. 25, no. 3, pp. 686–700, 2009, doi: 10.1109/tro.2008.2014125.

I. Hassani, I. Ergui, and C. Rekik, “Turning Point and Free Segments Strategies for Navigation of Wheeled Mobile Robot,” Int. J. Robot. Control Syst., vol. 2, no. 1, pp. 172–186, 2022, doi: 10.31763/ijrcs.v2i1.586.

Y. Zou, C. Wen, M. Shan, and M. Guan, “An adaptive control strategy for indoor leader-following of wheeled mobile robot,” J. Franklin Inst., vol. 357, no. 4, pp. 2131–2148, 2020, doi: 10.1016/j.jfranklin.2019.11.054.

D. Gu and K. S. Chen, “Design and performance evaluation of wiimote-based two-dimensional indoor localization systems for indoor mobile robot control,” Meas. J. Int. Meas. Confed., vol. 66, pp. 95–108, 2015, doi: 10.1016/j.measurement.2015.01.009.

Y. Nakamori, Y. Hiroi, and A. Ito, “Multiple player detection and tracking method using a laser range finder for a robot that plays with human,” ROBOMECH J., vol. 5, no. 1, p. 25, 2018, doi: 10.1186/s40648-018-0122-x.

P. T. Nguyen, S.-W. Yan, J.-F. Liao, and C.-H. Kuo, “Autonomous Mobile Robot Navigation in Sparse LiDAR Feature Environments,” Applied Sciences, vol. 11, no. 13. 2021, doi: 10.3390/app11135963.

H. Liu, J. Luo, P. Wu, S. Xie, and H. Li, “People detection and tracking using RGB-D cameras for mobile robots,” Int. J. Adv. Robot. Syst., vol. 13, no. 5, 2016, doi: 10.1177/1729881416657746.

G. Du, S. Long, F. Li, and X. Huang, “Active Collision Avoidance for Human-Robot Interaction With UKF, Expert System, and Artificial Potential Field Method,” Frontiers in Robotics and AI, vol. 5. 2018.

B. K. Patle, G. Babu L, A. Pandey, D. R. K. Parhi, and A. Jagadeesh, “A review: On path planning strategies for navigation of mobile robot,” Def. Technol., vol. 15, no. 4, pp. 582–606, 2019, doi: 10.1016/j.dt.2019.04.011.

U. Côtéallard et al., “A Convolutional Neural Network for robotic arm guidance using sEMG based frequency-features,” in IEEE International Conference on Intelligent Robots and Systems. 2016, pp. 2464–2470, doi: 10.1109/IROS.2016.7759384.

W. Guan, S. Chen, S. Wen, Z. Tan, H. Song, and W. Hou, “High-Accuracy Robot Indoor Localization Scheme Based on Robot Operating System Using Visible Light Positioning,” IEEE Photonics J., vol. 12, no. 2, 2020, doi: 10.1109/JPHOT.2020.2981485.

T. Kanda, D. F. Glas, M. Shiomi, and N. Hagita, “Abstracting peoples trajectories for social robots to proactively approach customers,” IEEE Trans. Robot., vol. 25, no. 6, pp. 1382–1396, 2009, doi: 10.1109/TRO.2009.2032969.

S. Saunderson and G. Nejat, “Investigating Strategies for Robot Persuasion in Social Human-Robot Interaction,” IEEE Trans. Cybern., vol. 52, no. 1, pp. 641–653, 2022, doi: 10.1109/TCYB.2020.2987463.

G. Song, K. Yin, Y. Zhou, and X. Cheng, “A surveillance robot with hopping capabilities for home security,” IEEE Trans. Consum. Electron., vol. 55, no. 4, pp. 2034–2039, 2009, doi: 10.1109/TCE.2009.5373766.

J. N. K. Liu, M. Wang, and B. Feng, “iBotGuard: An internet-based intelligent robot security system using invariant face recognition against intruder,” IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 35, no. 1, pp. 97–105, 2005, doi: 10.1109/TSMCC.2004.840051.

M. Javaid, A. Haleem, A. Vaish, R. Vaishya, and K. P. Iyengar, “Robotics Applications in COVID-19: A Review,” J. Ind. Integr. Manag., vol. 5, no. 4, pp. 441–451, 2020, doi: 10.1142/S2424862220300033.

S. Wang, K. Wang, R. Tang, J. Qiao, H. Liu, and Z. G. Hou, “Design of a Low-Cost Miniature Robot to Assist the COVID-19 Nasopharyngeal Swab Sampling,” IEEE Trans. Med. Robot. Bionics, vol. 3, no. 1, pp. 289–293, 2021, doi: 10.1109/TMRB.2020.3036461.

S. H. Alsamhi and B. Lee, “Blockchain-Empowered Multi-Robot Collaboration to Fight COVID-19 and Future Pandemics,” IEEE Access, vol. 9, pp. 44173–44197, 2021, doi: 10.1109/ACCESS.2020.3032450.

S. K. von Bueren, A. Burkart, A. Hueni, U. Rascher, M. P. Tuohy, and I. J. Yule, “Deploying four optical UAV-based sensors over grassland: challenges and limitations,” Biogeosciences, vol. 12, no. 1, pp. 163–175, 2015, doi: 10.5194/bg-12-163-2015.

M. O. A. Aqel, M. H. Marhaban, M. I. Saripan, and N. B. Ismail, “Review of visual odometry: types, approaches, challenges, and applications,” SpringerPlus, vol. 5, no. 1, p. 1897, 2016, doi: 10.1186/s40064-016-3573-7.

A. Leigh, J. Pineau, N. Olmedo, and H. Zhang, “Person tracking and following with 2D laser scanners,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), 2015, pp. 726–733, doi: 10.1109/ICRA.2015.7139259.

E. Dorronzoro Zubiete et al., “Evaluation of a Home Biomonitoring Autonomous Mobile Robot.,” Comput. Intell. Neurosci., vol. 2016, p. 9845816, 2016, doi: 10.1155/2016/9845816.

Y. Hosoda, K. Yamamoto, R. Ichinose, S. Egawa, J. Tamamoto, and T. Tsubouchi, “Collision Avoidance Control of Human-Symbiotic Robot,” Trans. JAPAN Soc. Mech. Eng. Ser. C, vol. 77, no. 775, pp. 1051–1061, 2011, doi: 10.1299/kikaic.77.1051.

R. Ali, P. Yunfeng, A. Ali, H. Ali, N. Akhter, J. Ahmed, and A. Jalil, “Passive Autofocusing System for a Thermal Camera,” IEEE Access, vol. 8, no. 1, pp. 130014–130022, 2020, doi: 10.1109/ACCESS.2020.3006356.

T. E. Salem, D. Ibitayo, and B. R. Geil, “Validation of infrared camera thermal measurements on high-voltage power electronic components,” IEEE Trans. Instrum. Meas., vol. 56, no. 5, pp. 1973–1978, 2007, doi: 10.1109/TIM.2007.903590.

E. Benli, Y. Motai, and J. Rogers, “Human Behavior-Based Target Tracking with an Omni-Directional Thermal Camera,” IEEE Trans. Cogn. Dev. Syst., vol. 11, no. 1, pp. 36–50, 2019, doi: 10.1109/TCDS.2017.2726356.

A. K. Kashyap, D. R. Parhi, and A. Pandey, “Analysis of Hybrid Technique for Motion Planning of Humanoid NAO,” Int. J. Robot. Control Syst., vol. 1, no. 1, pp. 75–83, 2021, doi: 10.31763/ijrcs.v1i1.285.

J. Wang, Q. Xue, L. Li, B. Liu, L. Huang, and Y. Chen, “Dynamic analysis of simple pendulum model under variable damping,” Alexandria Eng. J., vol. 61, no. 12, pp. 10563–10575, 2022, doi: 10.1016/j.aej.2022.03.064.

M. Sasaki, E. Kunii, T. Uda, K. Matsushita, J. K. Muguro, bin M. S. A. Suhaimi, and W. Njeri, “Construction of an Environmental Map including Road Surface Classification Based on a Coaxial Two-Wheeled Robot,” J. Sustain. Res. Eng., vol. 5, no. 3, pp. 159 – 169, 2020.

M. Khaled, A. Mohammed, M. S. Ibraheem, and R. Ali, “Balancing a Two Wheeled Robot,” 2009.

J. Chen and X. Bai, “Thermal face segmentation based on circular shortest path,” Infrared Phys. Technol., vol. 97, pp. 391–400, 2019, doi: 10.1016/j.infrared.2019.01.021.

Z. H. Xie, L. J. Liu, X. Y. Wang, and C. Yang, “An iterative method with enhanced Laplacian-scaled thresholding for noise-robust compressive sensing magnetic resonance image reconstruction,” IEEE Access, vol. 8, pp. 177021–177040, 2020, doi: 10.1109/ACCESS.2020.3027313.

E. H. Houssein, M. M. Emam, and A. A. Ali, “An efficient multilevel thresholding segmentation method for thermography breast cancer imaging based on improved chimp optimization algorithm,” Expert Syst. Appl., vol. 185, p. 115651, 2021, doi: 10.1016/j.eswa.2021.115651.

H. El Khoukhi, Y. Filali, A. Yahyaouy, M. A. Sabri, and A. Aarab, “A hardware implementation of OTSU thresholding method for skin cancer image segmentation,” 2019 Int. Conf. Wirel. Technol. Embed. Intell. Syst. WITS 2019, pp. 1–5, 2019, doi: 10.1109/WITS.2019.8723815.

S. Umirzakova and T. K. Whangbo, “Detailed feature extraction network-based fine-grained face segmentation,” Knowledge-Based Syst., p. 109036, 2022, doi: 10.1016/j.knosys.2022.109036.

M. S. Yang and K. P. Sinaga, “A feature-reduction multi-view k-means clustering algorithm,” IEEE Access, vol. 7, pp. 114472–114486, 2019, doi: 10.1109/ACCESS.2019.2934179.

K. P. Sinaga, I. Hussain, and M. S. Yang, “Entropy K-Means Clustering with Feature Reduction under Unknown Number of Clusters,” IEEE Access, vol. 9, pp. 67736–67751, 2021, doi: 10.1109/ACCESS.2021.3077622.

A. Elngar, A. El, R. Naeem, A. Essa, and Z. Shaaban, “The Viola-Jones Face Detection Algorithm Analysis: A Survey,” Journal of Cybersecurity and Information Management (JCIM), vol. 6, pp. 85–95, Apr. 2021, doi: 10.5281/zenodo.4898039.

T. Paul, U. A. Shammi, M. U. Ahmed, R. Rahman, S. Kobashi, and M. A. R. Ahad, “A Study on Face Detection Using Viola-Jones Algorithm in Various Backgrounds, Angles and Distances,” Int. J. Biomed. Soft Comput. Hum. Sci. Off. J. Biomed. Fuzzy Syst. Assoc., vol. 23, no. 1, pp. 27–36, 2018, doi: 10.24466/ijbschs.23.1_27.




DOI: https://doi.org/10.18196/jrc.v3i3.14750

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Muhammad Ilhamdi Rusydi, Minoru Sasaki

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik