A Secured, Multilevel Face Recognition based on Head Pose Estimation, MTCNN and FaceNet

Thai-Viet Dang, Hoai-Linh Tran

Abstract


Artificial Intelligence and IoT have always attracted a lot of attention from scholars and researchers because of their high applicability, which make them a typical technology of the Fourth Industrial Revolution. The hallmark of AI is its self-learning ability, which enables computers to predict and analyze complex data such as bio data (fingerprints, irises, and faces), voice recognition, text processing. Among those application, the face recognition is under intense research due to the demand in users’ identification. This paper proposes a new, secured, two-step solution for an identification system that uses MTCNN and FaceNet networks enhanced with head pose estimation of the users. The model's accuracy ranges from 92% to 95%, which make it competitive with recent research to demonstrate the system's usability.

Keywords


Face Recognition, Head Pose Estimation, Computer Vision, MTCNN.

Full Text:

PDF

References


R. S. Peres, X. Jia, J. Lee, K. Sun, A. W. Colombo, and J. Barata, “Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook,” IEEE Access, vol. 8, pp. 220121-220139, 2020.

J. Huang, J. Chai, and S. Cho, “Deep learning in finance and banking: A literature review and classification,” Frontiers of Business Research in China, vol. 14, pp. 1-24, 2020.

R. Gautam, A. Gedam, and A. A. Mahawadiwar, “Review on Development of Industrial Robotic Arm,” Int. Res. J. Eng. Technol., vol. 4, no. 3, 2017.

T.V. Dang and N.T. Bui, “Multi-Scale Fully Convolutional Network-Based Semantic Segmentation for Mobile Robot Navigation,” Electronics, vol. 12, no. 3, p. 533, 2023.

A. Gee and R. Cipolla, “Determining the gaze of faces in images,” Image and Vision Computing, vol. 12, no. 10, pp. 639-647, 1994.

N. L. P. Kurniawati, M. W. A. Kesiman, and I. M. G. Sunarya, “Recognition of Balinese Traditional Ornament Carving Images with Convolutional Neural Network and Discrete Wavelet Transform,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 8, no. 4, pp. 670-678, 2022.

T. V. Dang and N. T. Bui, “Obstacle Avoidance Strategy for Mobile Robot based on Monocular Camera,” Electronics, vol. 12, no. 8, p. 1932, 2023.

T. V. Dang, “Smart Attendance System based on Improved Facial Recognition,” Journal of Robotics and Control, vol. 4, no. 1, pp. 46-53, 2023.

T. V. Dang, “Smart home Management System with Face Recognition based on ArcFace model in Deep Convolutional Neural Network,” Journal of Robotics and Control, vol. 3, no. 6, pp. 754-761, 2022.

F. Masalha and N. Hirzallah, “A students attendance system using QR code,” Int. J. Adv. Comput. Sci. Appl., vol. 5, pp. 75-79, 2014.

O. Arulogun, A. Olatunbosun, O. Fakolujo, and O. Olaniyi, “RFID based students attendance management system,” Int. J. Sci. & Eng. Res., vol. 4, pp. 1-9, 2013.

F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015, doi: 10.1109/CVPR.2015.7298682.

A. Wirdiani, P. Hridayami, and A. Widiari, “Face Identification Based on K-Nearest Neighbor,” Scientist J. Inform., vol. 6, no. 2, pp. 150- 159, 2019.

A. A. Süzen, B. Duman, and B. Şen, “Benchmark Analysis of Jetson TX2, Jetson Nano and Raspberry PI using Deep-CNN,” 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1-5, 2020, doi: 10.1109/HORA49412.2020.9152915.

S. Kurzadkar, “Hotel management system using Python Tkinler GUI,” Int. J. Comput. Sci. Mobile Comput., vol. 11, no. 1, pp. 204-208, 2022.

Y. Nan, J. Ju, Q. Hua, H. Zhang, and B. Wang, “A-MobileNet: An approach of facial expression recognition,” Alexandria Eng. J., vol. 61, no. 6, pp. 4435-4444, 2021.

J. Yu and W. Zhang, “Face Mask Wearing Detection Algorithm Based on Improved YOLO-v4,” Sensor, vol. 21, no. 9, p. 3263, 2021.

A. Bochkovskiy, C. Y. Wang, and H. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” arXiv preprint arXiv:2004.10934, 2020.

J. Jeong, H. Park, and N. Kwak, “Enhancement of SSD by concatenating feature maps for object detection,” arXiv preprint arXiv:1705.09587, 2017.

C. Angelo, Q. Gao, X. Zhang, and W. Pang, “Fast and Accurate Hand Visual Detection by Using a Spatial-Channel Attention SSD for Hand-Based Space Robot Teleoperation,” International Journal of Aerospace Engineering, vol. 2022, 2022.

B. Huo, C. Li,J. Zhang, Y. Xue, and Z. Lin, “SAFF-SSD: Self-Attention Combined Feature Fusion-Based SSD for Small Object Detection in Remote Sensing,” Remote Sensing, vol. 15, no. 12, p. 3027, 2023.

H. Zhao et al., “Local Binary Pattern-Based Adaptive Differential Evolution for Multimodal Optimization Problems,” in IEEE Transactions on Cybernetics, vol. 50, no. 7, pp. 3343-3357, July 2020, doi: 10.1109/TCYB.2019.2927780.

S. Biswas, S. Kundu, and S. Das, “Inducing Niching Behavior in Differential Evolution Through Local Information Sharing,” in IEEE Transactions on Evolutionary Computation, vol. 19, no. 2, pp. 246-263, April 2015, doi: 10.1109/TEVC.2014.2313659.

F. Caraffini, A. V. Kononova, and D. Corne, “Infeasibility and structural bias in differential evolution,” Infomation Science, vol. 496, pp. 161–179, 2019.

J. Wang, C. Zheng, X. Yang, and L. Yang, “EnhanceFace: Adaptive Weighted SoftMax Loss for Deep Face Recognition,” in IEEE Signal Processing Letters, vol. 29, pp. 65-69, 2022, doi: 10.1109/LSP.2021.3125267.

W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, “Sphereface: Deep hypersphere embedding for face recognition,” Proc. IEEE Conf. Comput. Vision Pattern Recognit., pp. 212-220, 2017.

X. Li, D. Chang, T. Tian, and J. Cao, “Large-Margin Regularized Softmax Cross-Entropy Loss,” IEEE Access, vol. 7, pp. 19572-19578, 2019.

I. Goodfellow, Y. Bengio, and A. Courville, “Softmax Units for Multinoulli Output Distributions,” Deep Learning, pp. 180–184, 2016.

W. Liu, Y. Wen, Z. Yu, and M. Yang, “Large-Margin Softmax Loss for Convolutional Neural Networks,” Proc. 33rd Inter. Conf. Mach. Learn., vol. 48, 2016.

G.Bolin and P. Lacra, “On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning,” arXiv preprint arXiv:1704.00805, 2017.

P. Reverdy and N. E. Leonard, “Parameter Estimation in Softmax Decision-Making Models With Linear Objective Functions,” in IEEE Transactions on Automation Science and Engineering, vol. 13, no. 1, pp. 54-67, Jan. 2016, doi: 10.1109/TASE.2015.2499244.

C. Wu and Y. Zhang, “MTCNN and FACENET based access control system for face detection and recognition,” Automatic Control and Computer Sciences, vol. 55, pp. 102-112, 2021.

M. A. Falalu, U. Ibrahim, I. Amina, S. B. Abdulkadir, M. A. Baballe, and A. Ya'u, “A Smart Attendance System Based on Face Recognition: Challenges and Effects,” Global Journal of Research in Engineering & Computer Sciences, vol. 3, no. 2, pp. 25-30, 2023.

T. V. Dang, “Smart home Management System with Face Recognition Based on ArcFace Model in Deep Convolutional Neural Network,” Journal of Robotics and Control (JRC), vol. 3, no. 6, pp. 754-761, 2022.

A. Gee and R. Cipolla, “Determining the gaze of faces in images,” Image and Vision Computing, vol. 12, no. 10, pp. 639-647, 1994.

H. Yuan, M. Li, J. Hou, and J. Xiao, “Single image-based head pose estimation with spherical parametrization and 3D morphing,” Pattern Recognition, vol. 103, pp. 107316, 2020.

N. Aunsri and S. Rattarom, “Novel eye-based features for head pose-free gaze estimation with web camera: New model and low-cost device,” Ain Shams Engineering Journal, vol. 13, no. 5, pp. 101731, 2022.

B. -C. Chen, C. -S. Chen, and W. H. Hsu, “Face Recognition and Retrieval Using Cross-Age Reference Coding With Cross-Age Celebrity Dataset,” in IEEE Transactions on Multimedia, vol. 17, no. 6, pp. 804-815, June 2015, doi: 10.1109/TMM.2015.2420374.

B. Zhang and Y. Bao, “Age Estimation of Faces in Videos Using Head Pose Estimation and Convolutional Neural Networks,” Sensors, vol. 22, p. 4171, 2022.

R. Rothe; R. Timofte, and L. V. Gool, “Deep expectation of real and apparent age from a single image without facial landmarks,” Int. J. Comput. Vis., vol. 126, pp. 1–14, 2016.

X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark localization in the wild,” 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879-2886, 2012, doi: 10.1109/CVPR.2012.6248014.

H. Zhou, F. Jiang, L. Xiong, and H. Lu, “An Intuitive and unconstrained 2D cube representation for simultaneous head detection and pose estimation,” arXiv preprint arXiv:2212.03623, 2022.

N. Ruiz, E. Chong, and J. M. Rehg, “Fine-Grained Head Pose Estimation Without Keypoints,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2155-215509, 2018, doi: 10.1109/CVPRW.2018.00281.

E. Murphy-Chutorian and M. M. Trivedi, “Head Pose Estimation in Computer Vision: A Survey,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 607-626, April 2009, doi: 10.1109/TPAMI.2008.106.

T. Martyniuk, O. Kupyn, Y. Kurlyak, I. Krashenyi, J. Matas, and V. Sharmanska, “DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Image,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20910-20920, 2022, doi: 10.1109/CVPR52688.2022.02027.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 1, pp. 886-893, 2005.

H. S. Dadi and P. G. Mohan, “Performance Evaluation of Eigen faces and Fisher faces with different preprocessed Data sets,” International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), vol. 4, no. 5, pp. 2110-2116, 2015.

R. Fischer, M. Hödlmoser, and M. Gelautz, “Evaluation of Camera Pose Estimation Using Human Head Pose Estimation,” SN Computer Science, vol. 4, no. 301, 2023.

F. Camposeco, A. Cohen, M. Pollefeys, and T. Sattler, “Hybrid Camera Pose Estimation,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 136-144, 2018, doi: 10.1109/CVPR.2018.00022.

Y. Xu, Y. -J. Li, X. Weng, and K. Kitani, “Wide-Baseline Multi-Camera Calibration using Person Re-Identification,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13129-13138, 2021, doi: 10.1109/CVPR46437.2021.01293.

K. Khan, R. U. Khan, R. U., Leonardi, R., Migliorati, P., & Benini, S. (2021). Head pose estimation: A survey of the last ten years. Signal Processing: Image Communication, vol. 99, p. 116479, 2021.

K. Khan, R. U. Khan, R. Leonardi, P. Migliorati, and S. Benini, “Head pose estimation: A survey of the last ten years,” Signal Processing: Image Communication, vol. 99, p. 116479, 2021.

T. Liu et al., “GMDL: Toward precise head pose estimation via Gaussian mixed distribution learning for students’ attention understanding,” Infrared Physics & Technology, vol. 122, pp. 104099, 2022.

D. Anitta and A. A. Fathima, “Human head pose estimation based on HF method,” Microprocessors and Microsystems, vol. 82, p. 103802, 2021.

B. G. D. A. Madhusanka, S. Ramadass, P. Rajagopal, and H. M. K. K. M. B. Herath, “Attention-Aware Recognition of Activities of Daily Living Based on Eye Gaze Tracking,” Internet of Things for Human-Centered Design, vol. 1011, pp. 155–179, 2022.

C. Bouras and E. Michos, “An online real-time face recognition system for police purposes,” 2022 International Conference on Information Networking (ICOIN), pp. 62-67, 2022, doi: 10.1109/ICOIN53446.2022.9687212.

S. Santhosh and S. V. Rajashekararadhya, “A Design of Face Recognition Model with Spatial Feature Extraction using Optimized Support Vector Machine,” 2023 2nd International Conference for Innovation in Technology (INOCON), pp. 1-8, 2023, doi: 10.1109/INOCON57975.2023.10101149.

H. Agobah, O. Bamisile, D. Cai, D. K. Bensah Kulevome, B. Darkwa Nimo, and Q. Huang, “Deep Facial Expression Recognition Using Transfer Learning and Fine-Tuning Techniques,” 2022 IEEE 6th Conference on Energy Internet and Energy System Integration (EI2), pp. 1856-1860, 2022, doi: 10.1109/EI256261.2022.10116540.

H. S. Dadi and G. K. M. Pillutla, “Improved Face Recognition Rate Using HOG Features and SVM Classifier,” IOSR Journal of Electronics and Communication Engineering (IOSR-JECE), vol. 11, no. 4, pp. 33-44, 2016.

D. Huang, C. Shan, M. Ardabilian, Y. Wang, and L. Chen, “Local Binary Patterns and Its Application to Facial Image Analysis: A Survey,” in IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 41, no. 6, pp. 765-781, Nov. 2011, doi: 10.1109/TSMCC.2011.2118750.




DOI: https://doi.org/10.18196/jrc.v4i4.18780

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Thai-Viet Dang, Linh H. Tran

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik