Autonomous Movement Control of Coaxial Mobile Robot based on Aspect Ratio of Human Face for Public Relation Activity Using Stereo Thermal Camera

Authors

  • Muhammad Ilhamdi Rusydi Universitas Andalas https://orcid.org/0000-0001-7529-1584
  • Aulia Novira Universitas Andalas
  • Takayuki Nakagome Aisin Software Co. Ltd.
  • Joseph Muguro Dedan Kimathi University of Technology
  • Rio Nakajima Gifu University
  • Waweru Njeri Dedan Kimathi University of Technology
  • Kojiro Matsushita Gifu University
  • Minoru Sasaki Gifu University

DOI:

https://doi.org/10.18196/jrc.v3i3.14750

Keywords:

coaxial mobile robot, human following robot, recognition, stereo thermal camera,

Abstract

In recent years, robots that recognize people around them and provide guidance, information, and monitoring have been attracting attention. The mainstream of conventional human recognition technology is the method using a camera or laser range finder. However, it is difficult to recognize with a camera due to fluctuations in lighting 1), and it is often affected by the recognition environment such as misrecognition 2) with a person's leg and a chair's leg with a laser range finder. Therefore, we propose a human recognition method using a thermal camera that can visualize human heat. This study aims to realize human-following autonomous movement based on human recognition. In addition, the distance from the robot to the person is measured with a stereo thermal camera that uses two thermal cameras. A coaxial two-wheeled robot that is compact and capable of super-credit turning is used as a mobile robot. Finally, we conduct an autonomous movement experiment of a coaxial mobile robot based on human recognition by combining these. We performed human-following experiments on a coaxial two-wheeled robot based on human recognition using a stereo thermal camera and confirmed that it moves appropriately to the location where the recognized person is in multiple use cases (scenarios). However, the accuracy of distance measurement by stereo vision is inferior to that of laser measurement. It is necessary to improve it in the case of movement that requires more accuracy.

References

A. Khan, B. Rinner, and A. Cavallaro, “Cooperative robots to observe moving targets: Review,” IEEE Trans. Cybern., vol. 48, no. 1, pp. 187–198, 2018, doi: 10.1109/TCYB.2016.2628161.

K. P. Sinaga and M. S. Yang, “Unsupervised K-means clustering algorithm,” IEEE Access, vol. 8, pp. 80716–80727, 2020, doi: 10.1109/ACCESS.2020.2988796.

A. J. Abougarair and M. M. Edardar, “Adaptive Neural Networks Based Robust Output Feedback Control for Nonlinear System,” Wseas Trans. Comput. Res., vol. 9, no. 1, pp. 125–136, 2021, doi: 10.37394/232018.2021.9.15.

A. Ouda and A. Mohamed, “Autonomous Fuzzy Heading Control for a Multi-Wheeled Combat Vehicle,” Int. J. Robot. Control Syst., vol. 1, no. 1, pp. 90–101, 2021, doi: 10.31763/ijrcs.v1i1.286.

M. Tahmasebi, M. Gohari, and A. Emami, “An Autonomous Pesticide Sprayer Robot with a Color-based Vision System,” Int. J. Robot. Control Syst., vol. 2, no. 1, pp. 115–123, 2022, doi: 10.31763/ijrcs.v2i1.480.

J. Smids, S. Nyholm, and H. Berkers, “Robots in the Workplace: a Threat to—or Opportunity for—Meaningful Work?,” Philos. Technol., vol. 33, no. 3, pp. 503–522, 2020, doi: 10.1007/s13347-019-00377-4.

R. Tomari, Y. Kobayashi, and Y. Kuno, “Analysis of socially acceptable smart wheelchair navigation based on head cue information,” Procedia Comput. Sci., vol. 42, pp. 198–205, 2014, doi: 10.1016/j.procs.2014.11.052.

G. D’Andréa, L. Bordenave, F. Nguyen, Y. Tao, V. Paleri, S. Temam, A. Moya-Plana, and P. Gorphe, “A prospective longitudinal study of quality of life in robotic-assisted salvage surgery for oropharyngeal cancer,” Eur. J. Surg. Oncol., vol. 48, no. 6, pp. 1243-1250, 2022, doi: 10.1016/j.ejso.2022.01.017.

C. M. Song, H. S. Bang, H. G. Kim, H. J. Park, and K. Tae, “Health-related quality of life after transoral robotic thyroidectomy in papillary thyroid carcinoma,” Surg. (United States), vol. 170, no. 1, pp. 99–105, 2021, doi: 10.1016/j.surg.2021.02.042.

R. R. Galin and R. V Meshcheryakov, “Human-robot interaction efficiency and human-robot collaboration,” in Robotics: Industry 4.0 Issues & New Intelligent Control Paradigms, pp. 55–63, 2020.

A. Dzedzickis, J. Subačiūtė-Žemaitienė, E. Šutinys, U. Samukaitė-Bubnienė, and V. Bučinskas, “Advanced Applications of Industrial Robotics: New Trends and Possibilities,” Applied Sciences, vol. 12, no. 1. 2022, doi: 10.3390/app12010135.

J. Zhang, R. Liu, K. Yin, Z. Wang, M. Gui, and S. Y. Chen, “Intelligent Collaborative Localization among Air-Ground Robots for Industrial Environment Perception,” IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9673-9681, 2018, doi: 10.1109/TIE.2018.2880727.

W. Sheng, A. Thobbi, and Y. Gu, “An Integrated Framework for Human-Robot Collaborative Manipulation,” IEEE Trans. Cybern., vol. 45, no. 10, pp. 2030–2041, 2015, doi: 10.1109/TCYB.2014.2363664.

H. Xing, L. Shi, K. Tang, S. Guo, X. Hou, Y. Liu, H. Liu, and Y. Hu, “Robust RGB-D camera and IMU fusion-based cooperative and relative close-range localization for multiple turtle-inspired amphibious spherical robots,” J. Bionic Eng., vol. 16, no. 3, pp. 442–454, 2019.

C. F. Juang, M. G. Lai, and W. T. Zeng, “Evolutionary Fuzzy Control and Navigation for Two Wheeled Robots Cooperatively Carrying an Object in Unknown Environments,” IEEE Trans. Cybern., vol. 45, no. 9, pp. 1731–1743, 2015, doi: 10.1109/TCYB.2014.2359966.

J. W. Wang, Y. Guo, M. Fahad, and B. Bingham, “Dynamic Plume Tracking by Cooperative Robots,” IEEE/ASME Trans. Mechatronics, vol. 24, no. 2, pp. 609–620, 2019, doi: 10.1109/TMECH.2019.2892292.

S.-J. Chung and J.-J. E. Slotine, “Cooperative Robot Control and Concurrent Synchronization of Lagrangian Systems,” IEEE Trans. Robot., vol. 25, no. 3, pp. 686–700, 2009, doi: 10.1109/tro.2008.2014125.

I. Hassani, I. Ergui, and C. Rekik, “Turning Point and Free Segments Strategies for Navigation of Wheeled Mobile Robot,” Int. J. Robot. Control Syst., vol. 2, no. 1, pp. 172–186, 2022, doi: 10.31763/ijrcs.v2i1.586.

Y. Zou, C. Wen, M. Shan, and M. Guan, “An adaptive control strategy for indoor leader-following of wheeled mobile robot,” J. Franklin Inst., vol. 357, no. 4, pp. 2131–2148, 2020, doi: 10.1016/j.jfranklin.2019.11.054.

D. Gu and K. S. Chen, “Design and performance evaluation of wiimote-based two-dimensional indoor localization systems for indoor mobile robot control,” Meas. J. Int. Meas. Confed., vol. 66, pp. 95–108, 2015, doi: 10.1016/j.measurement.2015.01.009.

Y. Nakamori, Y. Hiroi, and A. Ito, “Multiple player detection and tracking method using a laser range finder for a robot that plays with human,” ROBOMECH J., vol. 5, no. 1, p. 25, 2018, doi: 10.1186/s40648-018-0122-x.

P. T. Nguyen, S.-W. Yan, J.-F. Liao, and C.-H. Kuo, “Autonomous Mobile Robot Navigation in Sparse LiDAR Feature Environments,” Applied Sciences, vol. 11, no. 13. 2021, doi: 10.3390/app11135963.

H. Liu, J. Luo, P. Wu, S. Xie, and H. Li, “People detection and tracking using RGB-D cameras for mobile robots,” Int. J. Adv. Robot. Syst., vol. 13, no. 5, 2016, doi: 10.1177/1729881416657746.

G. Du, S. Long, F. Li, and X. Huang, “Active Collision Avoidance for Human-Robot Interaction With UKF, Expert System, and Artificial Potential Field Method,” Frontiers in Robotics and AI, vol. 5. 2018.

B. K. Patle, G. Babu L, A. Pandey, D. R. K. Parhi, and A. Jagadeesh, “A review: On path planning strategies for navigation of mobile robot,” Def. Technol., vol. 15, no. 4, pp. 582–606, 2019, doi: 10.1016/j.dt.2019.04.011.

U. Côtéallard et al., “A Convolutional Neural Network for robotic arm guidance using sEMG based frequency-features,” in IEEE International Conference on Intelligent Robots and Systems. 2016, pp. 2464–2470, doi: 10.1109/IROS.2016.7759384.

W. Guan, S. Chen, S. Wen, Z. Tan, H. Song, and W. Hou, “High-Accuracy Robot Indoor Localization Scheme Based on Robot Operating System Using Visible Light Positioning,” IEEE Photonics J., vol. 12, no. 2, 2020, doi: 10.1109/JPHOT.2020.2981485.

T. Kanda, D. F. Glas, M. Shiomi, and N. Hagita, “Abstracting peoples trajectories for social robots to proactively approach customers,” IEEE Trans. Robot., vol. 25, no. 6, pp. 1382–1396, 2009, doi: 10.1109/TRO.2009.2032969.

S. Saunderson and G. Nejat, “Investigating Strategies for Robot Persuasion in Social Human-Robot Interaction,” IEEE Trans. Cybern., vol. 52, no. 1, pp. 641–653, 2022, doi: 10.1109/TCYB.2020.2987463.

G. Song, K. Yin, Y. Zhou, and X. Cheng, “A surveillance robot with hopping capabilities for home security,” IEEE Trans. Consum. Electron., vol. 55, no. 4, pp. 2034–2039, 2009, doi: 10.1109/TCE.2009.5373766.

J. N. K. Liu, M. Wang, and B. Feng, “iBotGuard: An internet-based intelligent robot security system using invariant face recognition against intruder,” IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 35, no. 1, pp. 97–105, 2005, doi: 10.1109/TSMCC.2004.840051.

M. Javaid, A. Haleem, A. Vaish, R. Vaishya, and K. P. Iyengar, “Robotics Applications in COVID-19: A Review,” J. Ind. Integr. Manag., vol. 5, no. 4, pp. 441–451, 2020, doi: 10.1142/S2424862220300033.

S. Wang, K. Wang, R. Tang, J. Qiao, H. Liu, and Z. G. Hou, “Design of a Low-Cost Miniature Robot to Assist the COVID-19 Nasopharyngeal Swab Sampling,” IEEE Trans. Med. Robot. Bionics, vol. 3, no. 1, pp. 289–293, 2021, doi: 10.1109/TMRB.2020.3036461.

S. H. Alsamhi and B. Lee, “Blockchain-Empowered Multi-Robot Collaboration to Fight COVID-19 and Future Pandemics,” IEEE Access, vol. 9, pp. 44173–44197, 2021, doi: 10.1109/ACCESS.2020.3032450.

S. K. von Bueren, A. Burkart, A. Hueni, U. Rascher, M. P. Tuohy, and I. J. Yule, “Deploying four optical UAV-based sensors over grassland: challenges and limitations,” Biogeosciences, vol. 12, no. 1, pp. 163–175, 2015, doi: 10.5194/bg-12-163-2015.

M. O. A. Aqel, M. H. Marhaban, M. I. Saripan, and N. B. Ismail, “Review of visual odometry: types, approaches, challenges, and applications,” SpringerPlus, vol. 5, no. 1, p. 1897, 2016, doi: 10.1186/s40064-016-3573-7.

A. Leigh, J. Pineau, N. Olmedo, and H. Zhang, “Person tracking and following with 2D laser scanners,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), 2015, pp. 726–733, doi: 10.1109/ICRA.2015.7139259.

E. Dorronzoro Zubiete et al., “Evaluation of a Home Biomonitoring Autonomous Mobile Robot.,” Comput. Intell. Neurosci., vol. 2016, p. 9845816, 2016, doi: 10.1155/2016/9845816.

Y. Hosoda, K. Yamamoto, R. Ichinose, S. Egawa, J. Tamamoto, and T. Tsubouchi, “Collision Avoidance Control of Human-Symbiotic Robot,” Trans. JAPAN Soc. Mech. Eng. Ser. C, vol. 77, no. 775, pp. 1051–1061, 2011, doi: 10.1299/kikaic.77.1051.

R. Ali, P. Yunfeng, A. Ali, H. Ali, N. Akhter, J. Ahmed, and A. Jalil, “Passive Autofocusing System for a Thermal Camera,” IEEE Access, vol. 8, no. 1, pp. 130014–130022, 2020, doi: 10.1109/ACCESS.2020.3006356.

T. E. Salem, D. Ibitayo, and B. R. Geil, “Validation of infrared camera thermal measurements on high-voltage power electronic components,” IEEE Trans. Instrum. Meas., vol. 56, no. 5, pp. 1973–1978, 2007, doi: 10.1109/TIM.2007.903590.

E. Benli, Y. Motai, and J. Rogers, “Human Behavior-Based Target Tracking with an Omni-Directional Thermal Camera,” IEEE Trans. Cogn. Dev. Syst., vol. 11, no. 1, pp. 36–50, 2019, doi: 10.1109/TCDS.2017.2726356.

A. K. Kashyap, D. R. Parhi, and A. Pandey, “Analysis of Hybrid Technique for Motion Planning of Humanoid NAO,” Int. J. Robot. Control Syst., vol. 1, no. 1, pp. 75–83, 2021, doi: 10.31763/ijrcs.v1i1.285.

J. Wang, Q. Xue, L. Li, B. Liu, L. Huang, and Y. Chen, “Dynamic analysis of simple pendulum model under variable damping,” Alexandria Eng. J., vol. 61, no. 12, pp. 10563–10575, 2022, doi: 10.1016/j.aej.2022.03.064.

M. Sasaki, E. Kunii, T. Uda, K. Matsushita, J. K. Muguro, bin M. S. A. Suhaimi, and W. Njeri, “Construction of an Environmental Map including Road Surface Classification Based on a Coaxial Two-Wheeled Robot,” J. Sustain. Res. Eng., vol. 5, no. 3, pp. 159 – 169, 2020.

M. Khaled, A. Mohammed, M. S. Ibraheem, and R. Ali, “Balancing a Two Wheeled Robot,” 2009.

J. Chen and X. Bai, “Thermal face segmentation based on circular shortest path,” Infrared Phys. Technol., vol. 97, pp. 391–400, 2019, doi: 10.1016/j.infrared.2019.01.021.

Z. H. Xie, L. J. Liu, X. Y. Wang, and C. Yang, “An iterative method with enhanced Laplacian-scaled thresholding for noise-robust compressive sensing magnetic resonance image reconstruction,” IEEE Access, vol. 8, pp. 177021–177040, 2020, doi: 10.1109/ACCESS.2020.3027313.

E. H. Houssein, M. M. Emam, and A. A. Ali, “An efficient multilevel thresholding segmentation method for thermography breast cancer imaging based on improved chimp optimization algorithm,” Expert Syst. Appl., vol. 185, p. 115651, 2021, doi: 10.1016/j.eswa.2021.115651.

H. El Khoukhi, Y. Filali, A. Yahyaouy, M. A. Sabri, and A. Aarab, “A hardware implementation of OTSU thresholding method for skin cancer image segmentation,” 2019 Int. Conf. Wirel. Technol. Embed. Intell. Syst. WITS 2019, pp. 1–5, 2019, doi: 10.1109/WITS.2019.8723815.

S. Umirzakova and T. K. Whangbo, “Detailed feature extraction network-based fine-grained face segmentation,” Knowledge-Based Syst., p. 109036, 2022, doi: 10.1016/j.knosys.2022.109036.

M. S. Yang and K. P. Sinaga, “A feature-reduction multi-view k-means clustering algorithm,” IEEE Access, vol. 7, pp. 114472–114486, 2019, doi: 10.1109/ACCESS.2019.2934179.

K. P. Sinaga, I. Hussain, and M. S. Yang, “Entropy K-Means Clustering with Feature Reduction under Unknown Number of Clusters,” IEEE Access, vol. 9, pp. 67736–67751, 2021, doi: 10.1109/ACCESS.2021.3077622.

A. Elngar, A. El, R. Naeem, A. Essa, and Z. Shaaban, “The Viola-Jones Face Detection Algorithm Analysis: A Survey,” Journal of Cybersecurity and Information Management (JCIM), vol. 6, pp. 85–95, Apr. 2021, doi: 10.5281/zenodo.4898039.

T. Paul, U. A. Shammi, M. U. Ahmed, R. Rahman, S. Kobashi, and M. A. R. Ahad, “A Study on Face Detection Using Viola-Jones Algorithm in Various Backgrounds, Angles and Distances,” Int. J. Biomed. Soft Comput. Hum. Sci. Off. J. Biomed. Fuzzy Syst. Assoc., vol. 23, no. 1, pp. 27–36, 2018, doi: 10.24466/ijbschs.23.1_27.

Downloads

Published

2022-05-01

How to Cite

[1]
M. I. Rusydi, “Autonomous Movement Control of Coaxial Mobile Robot based on Aspect Ratio of Human Face for Public Relation Activity Using Stereo Thermal Camera”, J Robot Control (JRC), vol. 3, no. 3, pp. 361–373, May 2022.

Issue

Vol. 3 No. 3 (2022): May

Section

Articles

License

Authors who publish with this journal agree to the following terms: 

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).

 

This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:

  1. Share – copy and redistribute the material in any medium or format.
  2. Adapt – remix, transform, and build upon the material for any purpose, even comercially.

The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:

  1. Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  2. ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
  3. No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

 

• Creative Commons Attribution-ShareAlike (CC BY-SA)

Creative Commons License
JRC is licensed under an International License