Human Activity Recognition Using Accelerometer & Gyroscope Smartphone Sensor by Extract Statistical Features

Muthana Hmod Abdullah, M. A. Ahmed

Abstract


Understanding behavioral patterns and forecasting the bodily motions of persons heavily relies on detecting human activities. This has profound ramifications in several domains, including healthcare, sports, and security. This study sought to identify and classify 18 human actions recorded by 90 people using smartphone sensors using the KU-HAR dataset. The primary aim of this study is to examine statistical features such as (mean, mod, entropy, max, median …etc.) derived from time-domain sensory data collected using accelerometers and gyroscopes. Activity detection utilizes many machine learning methods such as Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Logistic Regression (LG), Naïve Bayes (NB), and AdaBoost. The RF model achieves the highest overall accuracy of 99%. While the DT model gets 95%, SVM receives 93%, and the KNN gets 82%. At the same time, the other model didn’t get good results. The research is evaluated using accuracy, recall, precision, and f1-scor. The research contribution is to extract the statistical feature from the raw file of the sensor to create a new dataset. This research recommends employing statistical features in time series. Future research is recommended to solve misclassification in certain activities, which could be achieved using feature selection to reduce the number of features.

Keywords


Human Activity; Classification Algorithms; Wearable Sensor; Tsfresh; Statistical Feature.

Full Text:

PDF

References


L. Schrader et al., "Advanced sensing and human activity recognition in early intervention and rehabilitation of elderly people," Journal of Population Ageing, vol. 13, pp. 139-165, 2020.

A. Lentzas and D. Vrakas, "Non-intrusive human activity recognition and abnormal behavior detection on elderly people: A review," Artificial Intelligence Review, vol. 53, no. 3, pp. 1975-2021, 2020.

A. Subasi, K. Khateeb, T. Brahimi, and A. Sarirete, "Human activity recognition using machine learning methods in a smart healthcare environment," in Innovation in health informatics, pp. 123-144, 2020.

H. Sun and Y. Chen, "Real-time elderly monitoring for senior safety by lightweight human action recognition," in 2022 IEEE 16th International Symposium on Medical Information and Communication Technology (ISMICT), pp. 1-6, 2022.

P. P. Ariza-Colpas et al., "Human activity recognition data analysis: History, evolutions, and new trends," Sensors, vol. 22, no. 9, p. 3401, 2022.

D. Mukherjee, R. Mondal, P. K. Singh, R. Sarkar, and D. Bhattacharjee, "EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications," Multimedia Tools and Applications, vol. 79, pp. 31663-31690, 2020.

S. Iloga, A. Bordat, J. Le Kernec, and O. Romain, "Human activity recognition based on acceleration data from smartphones using HMMs," IEEE Access, vol. 9, pp. 139336-139351, 2021.

S. E. Ali, A. N. Khan, S. Zia, and M. Mukhtar, "Human activity recognition system using smart phone based accelerometer and machine learning," in 2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), pp. 69-74, 2020.

R. Ahmed Bhuiyan, N. Ahmed, M. Amiruzzaman, and M. R. Islam, "A robust feature extraction model for human activity characterization using 3-axis accelerometer and gyroscope data," Sensors, vol. 20, no. 23, p. 6990, 2020.

P. Yang, C. Yang, V. Lanfranchi, and F. Ciravegna, "Activity graph based convolutional neural network for human activity recognition using acceleration and gyroscope data," IEEE Transactions on Industrial Informatics, vol. 18, no. 10, pp. 6619-6630, 2022.

A. E. Minarno, W. A. Kusuma, H. Wibowo, D. R. Akbi, and N. Jawas, "Single triaxial accelerometer-gyroscope classification for human activity recognition," in 2020 8th international conference on information and communication technology (ICoICT), pp. 1-5, 2020.

W. Taylor, S. A. Shah, K. Dashtipour, A. Zahid, Q. H. Abbasi, and M. A. Imran, "An intelligent non-invasive real-time human activity recognition system for next-generation healthcare," Sensors, vol. 20, no. 9, p. 2653, 2020.

J. Liu, G. Teng, and F. Hong, "Human activity sensing with wireless signals: A survey," Sensors, vol. 20, no. 4, p. 1210, 2020.

H. Shahverdi, M. Nabati, P. Fard Moshiri, R. Asvadi, and S. A. Ghorashi, "Enhancing CSI-based human activity recognition by edge detection techniques," Information, vol. 14, no. 7, p. 404, 2023.

G. Bhola and D. K. Vishwakarma, "A review of vision-based indoor HAR: state-of-the-art, challenges, and future prospects," Multimedia Tools and Applications, vol. 83, no. 1, pp. 1965-2005, 2024.

E. Ramanujam, T. Perumal, and S. Padmavathi, "Human activity recognition with smartphone and wearable sensors using deep learning techniques: A review," IEEE Sensors Journal, vol. 21, no. 12, pp. 13029-13040, 2021.

W. Kong, L. He, and H. Wang, "Exploratory data analysis of human activity recognition based on smart phone," IEEE Access, vol. 9, pp. 73355-73364, 2021.

A. Gupta, K. Gupta, K. Gupta, and K. Gupta, "A survey on human activity recognition and classification," in 2020 international conference on communication and signal processing (ICCSP), pp. 0915-0919, 2020.

R. K. Athota and D. Sumathi, "Human activity recognition based on hybrid learning algorithm for wearable sensor data," Measurement: Sensors, vol. 24, p. 100512, 2022.

F. Demrozi, G. Pravadelli, A. Bihorac, and P. Rashidi, "Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey," IEEE access, vol. 8, pp. 210816-210836, 2020.

M. S. H. Bhuiyan, N. S. Patwary, P. K. Saha, and M. T. Hossain, "Sensor-based human activity recognition: A comparative study of machine learning techniques," in 2020 2nd International Conference on Advanced Information and Communication Technology (ICAICT), pp. 286-290, 2020.

D. Thakur, S. Biswas, E. S. Ho, and S. Chattopadhyay, "Convae-lstm: Convolutional autoencoder long short-term memory network for smartphone-based human activity recognition," IEEE Access, vol. 10, pp. 4137-4156, 2022.

B. Fu, N. Damer, F. Kirchbuchner, and A. Kuijper, "Sensing technology for human activity recognition: A comprehensive survey," Ieee Access, vol. 8, pp. 83791-83820, 2020.

D. R. Beddiar, B. Nini, M. Sabokrou, and A. Hadid, "Vision-based human activity recognition: a survey," Multimedia Tools and Applications, vol. 79, no. 41, pp. 30509-30555, 2020.

D. Bouchabou, S. M. Nguyen, C. Lohr, B. LeDuc, and I. Kanellos, "A survey of human activity recognition in smart homes based on IoT sensors algorithms: Taxonomies, challenges, and opportunities with deep learning," Sensors, vol. 21, no. 18, p. 6037, 2021.

J. Zhang, Y. Liu, and H. Yuan, "Attention-Based Residual BiLSTM Networks for Human Activity Recognition," IEEE Access, vol. 11, pp. 94173-94187, 2023.

S. Raziani and M. Azimbagirad, "Deep CNN hyperparameter optimization algorithms for sensor-based human activity recognition," Neuroscience Informatics, vol. 2, no. 3, p. 100078, 2022.

S. M. Bokhari, S. Sohaib, A. R. Khan, and M. Shafi, "DGRU based human activity recognition using channel state information," Measurement, vol. 167, p. 108245, 2021.

Z. Sun, Q. Ke, H. Rahmani, M. Bennamoun, G. Wang, and J. Liu, "Human action recognition from various data modalities: A review," IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 3, pp. 3200-3225, 2022.

N. Jaouedi, N. Boujnah, and M. S. Bouhlel, "A new hybrid deep learning model for human action recognition," Journal of King Saud University-Computer and Information Sciences, vol. 32, no. 4, pp. 447-453, 2020.

S. Qiu et al., "Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges," Information Fusion, vol. 80, pp. 241-265, 2022.

Y. Kong and Y. Fu, "Human action recognition and prediction: A survey," International Journal of Computer Vision, vol. 130, no. 5, pp. 1366-1401, 2022.

A. Manaf and S. Singh, "Computer vision-based survey on human activity recognition system, challenges and applications," in 2021 3rd International Conference on Signal Processing and Communication (ICPSC), pp. 110-114, 2021.

H. Liu, Y. Hartmann, and T. Schultz, "Motion Units: Generalized sequence modeling of human activities for sensor-based activity recognition," in 2021 29th European signal processing conference (EUSIPCO), pp. 1506-1510, 2021.

Z. Hussain, Q. Z. Sheng, and W. E. Zhang, "A review and categorization of techniques on device-free human activity recognition," Journal of Network and Computer Applications, vol. 167, p. 102738, 2020.

S. Wan, L. Qi, X. Xu, C. Tong, and Z. Gu, "Deep learning models for real-time human activity recognition with smartphones," Mobile Networks and Applications, vol. 25, no. 2, pp. 743-755, 2020.

A. Ray, M. H. Kolekar, R. Balasubramanian, and A. Hafiane, "Transfer learning enhanced vision-based human activity recognition: a decade-long analysis," International Journal of Information Management Data Insights, vol. 3, no. 1, p. 100142, 2023.

L. M. Dang, K. Min, H. Wang, M. J. Piran, C. H. Lee, and H. Moon, "Sensor-based and vision-based human activity recognition: A comprehensive survey," Pattern Recognition, vol. 108, p. 107561, 2020.

R. G. Ramos, J. D. Domingo, E. Zalama, and J. Gómez-García-Bermejo, "Daily human activity recognition using non-intrusive sensors," Sensors, vol. 21, no. 16, p. 5270, 2021.

M. Ehatisham-Ul-Haq, M. A. Azam, Y. Amin, and U. Naeem, "C2FHAR: Coarse-to-fine human activity recognition with behavioral context modeling using smart inertial sensors," IEEE Access, vol. 8, pp. 7731-7747, 2020.

J. Wang, T. Zhu, J. Gan, L. L. Chen, H. Ning, and Y. Wan, "Sensor data augmentation by resampling in contrastive learning for human activity recognition," IEEE Sensors Journal, vol. 22, no. 23, pp. 22994-23008, 2022.

K. Xia, J. Huang, and H. Wang, "LSTM-CNN architecture for human activity recognition," IEEE Access, vol. 8, pp. 56855-56866, 2020.

I. Jegham, A. B. Khalifa, I. Alouani, and M. A. Mahjoub, "Vision-based human action recognition: An overview and real world challenges," Forensic Science International: Digital Investigation, vol. 32, p. 200901, 2020.

A. Shenoy and N. Thillaiarasu, "A survey on different computer vision based human activity recognition for surveillance applications," in 2022 6th International Conference on Computing Methodologies and Communication (ICCMC), pp. 1372-1376, 2022.

J. Arunnehru et al., "Machine vision-based human action recognition using spatio-temporal motion features (STMF) with difference intensity distance group pattern (DIDGP)," Electronics, vol. 11, no. 15, p. 2363, 2022.

Z. Yang, M. Qu, Y. Pan, and R. Huan, "Comparing Cross-Subject Performance on Human Activities Recognition Using Learning Models," IEEE Access, vol. 10, pp. 95179-95196, 2022.

S. Bian, M. Liu, B. Zhou, and P. Lukowicz, "The state-of-the-art sensing techniques in human activity recognition: A survey," Sensors, vol. 22, no. 12, p. 4596, 2022.

G. Diraco, G. Rescio, P. Siciliano, and A. Leone, "Review on human action recognition in smart living: Sensing technology, multimodality, real-time processing, interoperability, and resource-constrained processing," Sensors, vol. 23, no. 11, p. 5281, 2023.

M. Webber and R. F. Rojas, "Human activity recognition with accelerometer and gyroscope: A data fusion approach," IEEE Sensors Journal, vol. 21, no. 15, pp. 16979-16989, 2021.

Y. Feng, D. Duives, W. Daamen, and S. Hoogendoorn, "Data collection methods for studying pedestrian behaviour: A systematic review," Building and Environment, vol. 187, p. 107329, 2021.

D. Vitazkova et al., "Advances in Respiratory Monitoring: A Comprehensive Review of Wearable and Remote Technologies," Biosensors, vol. 14, no. 2, p. 90, 2024.

A. Yazici et al., "A smart e-health framework for monitoring the health of the elderly and disabled," Internet of Things, vol. 24, p. 100971, 2023.

A. Hoelzemann, J. L. Romero, M. Bock, K. V. Laerhoven, and Q. Lv, "Hang-time HAR: A benchmark dataset for basketball activity recognition using wrist-worn inertial sensors," Sensors, vol. 23, no. 13, p. 5879, 2023.

R. Mondal, D. Mukherjee, P. K. Singh, V. Bhateja, and R. Sarkar, "A new framework for smartphone sensor-based human activity recognition using graph neural network," IEEE Sensors Journal, vol. 21, no. 10, pp. 11461-11468, 2020.

N. Sikder and A.-A. Nahid, "KU-HAR: An open dataset for heterogeneous human activity recognition," Pattern Recognition Letters, vol. 146, pp. 46-54, 2021.

S. Caro-Alvaro, E. Garcia-Lopez, A. Brun-Guajardo, A. Garcia-Cabot, and A. Mavri, "Gesture-Based Interactions: Integrating Accelerometer and Gyroscope Sensors in the Use of Mobile Apps," Sensors, vol. 24, no. 3, p. 1004, 2024.

T. Hasegawa, "Smartphone sensor-based human activity recognition robust to different sampling rates," IEEE Sensors Journal, vol. 21, no. 5, pp. 6930-6941, 2020.

R. T. Al_Hassani and D. C. Atilla, "Human activity detection using smart wearable sensing devices with feed forward neural networks and PSO," Applied Sciences, vol. 13, no. 6, p. 3716, 2023.

S. K. Yadav, K. Tiwari, H. M. Pandey, and S. A. Akbar, "A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directions," Knowledge-Based Systems, vol. 223, p. 106970, 2021.

G. Saleem, U. I. Bajwa, and R. H. Raza, "Toward human activity recognition: a survey," Neural Computing and Applications, vol. 35, no. 5, pp. 4145-4182, 2023.

K. Muralidharan, A. Ramesh, G. Rithvik, S. Prem, A. Reghunaath, and M. Gopinath, "1D Convolution approach to human activity recognition using sensor data and comparison with machine learning algorithms," International Journal of Cognitive Computing in Engineering, vol. 2, pp. 130-143, 2021.

Z. Chen, C. Jiang, S. Xiang, J. Ding, M. Wu, and X. Li, "Smartphone sensor-based human activity recognition using feature fusion and maximum full a posteriori," IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 7, pp. 3992-4001, 2019.

V. Ghate, "Hybrid deep learning approaches for smartphone sensor-based human activity recognition," Multimedia Tools and Applications, vol. 80, no. 28, pp. 35585-35604, 2021.

R. Mutegeki and D. S. Han, "A CNN-LSTM approach to human activity recognition," in 2020 international conference on artificial intelligence in information and communication (ICAIIC), pp. 362-366, 2020.

J. A. Gamble and J. Huang, "Convolutional neural network for human activity recognition and identification," in 2020 IEEE International Systems Conference (SysCon), pp. 1-7, 2020.

M. Christ, N. Braun, J. Neuffer, and A. W. Kempa-Liehr, "Time series feature extraction on basis of scalable hypothesis tests (tsfresh–a python package)," Neurocomputing, vol. 307, pp. 72-77, 2018.

F. Thabtah, S. Hammoud, F. Kamalov, and A. Gonsalves, "Data imbalance in classification: Experimental evaluation," Information Sciences, vol. 513, pp. 429-441, 2020.

E. Rendon, R. Alejo, C. Castorena, F. J. Isidro-Ortega, and E. E. Granda-Gutierrez, "Data sampling methods to deal with the big data multi-class imbalance problem," Applied Sciences, vol. 10, no. 4, p. 1276, 2020.

M. Heydarian, T. E. Doyle, and R. Samavi, "MLCM: Multi-label confusion matrix," IEEE Access, vol. 10, pp. 19083-19095, 2022.

A. E. Maxwell, T. A. Warner, and L. A. Guillén, "Accuracy assessment in convolutional neural network-based deep learning remote sensing studies—Part 1: Literature review," Remote Sensing, vol. 13, no. 13, p. 2450, 2021.

S. Szabó, I. J. Holb, V. É. Abriha-Molnár, G. Szatmári, S. K. Singh, and D. Abriha, "Classification assessment tool: a program to measure the uncertainty of classification models in terms of class-level metrics," Applied Soft Computing, vol. 155, p. 111468, 2024.

X. Wang et al., "Evaluating the effectiveness of machine learning and deep learning models combined time-series satellite data for multiple crop types classification over a large-scale region," Remote Sensing, vol. 14, no. 10, p. 2341, 2022.

D. Chicco and G. Jurman, "The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation," BMC genomics, vol. 21, pp. 1-13, 2020.

M. M. Islam, M. R. Haque, H. Iqbal, M. M. Hasan, M. Hasan, and M. N. Kabir, "Breast cancer prediction: a comparative study using machine learning techniques," SN Computer Science, vol. 1, pp. 1-14, 2020.

M. Rottmann, K. Maag, R. Chan, F. Hüger, P. Schlicht, and H. Gottschalk, "Detection of false positive and false negative samples in semantic segmentation," in 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1351-1356, 2020.

J. Mijalkovic and A. Spognardi, "Reducing the false negative rate in deep learning based network intrusion detection systems," Algorithms, vol. 15, no. 8, p. 258, 2022.

A. M. Carrington et al., "Deep ROC analysis and AUC as balanced average accuracy, for improved classifier selection, audit and explanation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 329-341, 2022.

A. Anwyl-Irvine, E. S. Dalmaijer, N. Hodges, and J. K. Evershed, "Realistic precision and accuracy of online experiment platforms, web browsers, and devices," Behavior research methods, vol. 53, no. 4, pp. 1407-1425, 2021.

D. Chicco, N. Tötsch, and G. Jurman, "The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation," BioData mining, vol. 14, pp. 1-22, 2021.

M. A. R. Refat, M. Al Amin, C. Kaushal, M. N. Yeasmin, and M. K. Islam, "A comparative analysis of early stage diabetes prediction using machine learning and deep learning approach," in 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC), pp. 654-659, 2021.

Google. "Google Colab", 2024, https://research.google.com/colaboratory/faq.html#whats-colaboratory (accessed 2024).




DOI: https://doi.org/10.18196/jrc.v5i5.22443

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Muthana Hmod Abdullah, M. A. Ahmed

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik