AI-Driven Classification of Children’s Drawings for Pediatric Psychological Evaluation: An Ensemble Deep Learning Approach
DOI:
https://doi.org/10.18196/jrc.v6i1.23302Keywords:
Pediatric Psychological Assessment, Ensemble Deep Learning, Children’s Drawings Analysis, Mental Health Detection, Art Therapy in PsychologyAbstract
In the wake of contemporary challenges such as the COVID-19 pandemic, understanding children’s mental health through non-verbal forms like drawing has become paramount. This study enhances pediatric psychological assessments by employing an ensemble of deep learning models to interpret children’s drawings, aiming for early detection of psychological states. Traditional drawing analysis methods are often subjective, variable and time consuming. To ddress these limitations, we developed an ensemble model that combines the strengths of VGG16, VGG19, and MobileNet architectures using a hard voting mechanism. This approach reduces bias and enhances reliability by integrating the unique capabilities of each model. Our methodology involved rigorous data collection through a custom Android application, followed by exploratory data analysis, data preprocessing, and comprehensive model valuation. The ensemble model was trained and validated on the diverse Kids’ Hand Movement Dataset (KHMD), demonstrating superior accuracy and robustness in classifying drawings that indicate various psychological conditions. It significantly outperformed individual models, achieving a 99% accuracy rate. These findings underscore the potential of advanced machine learning techniques in providing more accurate and bias-free insights into children’s psychological health, suggesting that ensemble learning can greatly improve the precision of pediatric psychological evaluations. Future work will explore expanding the dataset and employing more sophisticated ensemble methods to further enhance diagnostic accuracy.
References
E. A. Rider, E. Ansari, P. H. Varrin, and J. Sparrow, “Mental health and wellbeing of children and adolescents during the covid-19 pandemic,” bmj, vol. 374, 2021.
C. S. M. Ng and S. S. L. Ng, “Impact of the COVID-19 pandemic on children’s mental health: A systematic review,” Front. Psychiatry, vol. 13, p. 975936, Oct. 2022.
I. Braito, T. Rudd, D. Buyuktaskin, M. Ahmed, C. Glancy, and A. Mulligan, “Review: systematic review of effectiveness of art psychotherapy in children with mental health disorders,” Ir. J. Med. Sci., vol. 191, no. 3, pp. 1369–1383, Jun. 2022.
Z. Amod, R. Gericke, and K. Bain, “Projective assessment using the draw-a-person test and kinetic family drawing in south africa,” Psychological, p. 375, 2013.
W. A. Bainbridge, “A tutorial on capturing mental representations through drawing and crowd-sourced scoring,” Behav. Res. Methods, vol. 54, no. 2, pp. 663–675, Apr. 2022.
O. Altun and O. Nooruldeen, “Sketrack: Stroke-based recognition of online hand-drawn sketches of arrow-connected diagrams and digital logic circuit diagrams,” Scientific Programming, vol. 2019, no. 1, p. 6501264, 2019.
R. R. Rachala and M. R. Panicker, “Hand-drawn electrical circuit recognition using object detection and node recognition,” SN Computer Science, vol. 3, no. 3, p. 244, 2022.
K. Wrobel, R. Doroz, P. Porwik, T. Orczyk, A. B. Cavalcante, and M. Grajzer, “Features of hand-drawn spirals for recognition of parkinson’s disease,” in Asian Conference on Intelligent Information and Database Systems, pp. 458–469, 2022.
S. Roy, A. Bhattacharya, N. Sarkar, S. Malakar, and R. Sarkar, “Offline hand-drawn circuit component recognition using texture and shapebased features,” Multimedia Tools and Applications, vol. 79, pp. 353–373, 2020.
M. Gupta, P. Mehndiratta, and A. Bhardwaj, “Object recognition in hand drawn images using machine ensembling techniques and smote sampling,” in Information, Communication and Computing Technology: 4th International Conference, ICICCT 2019, pp. 228–239, 2019.
M. Gupta and P. Mehndiratta, “Analysis and recognition of handdrawn images with effective data handling,” in Big Data Analytics: 7th International Conference, BDA 2019, pp. 389–407, 2019.
W. Adorno, A. Yi, M. Durieux, and D. Brown, “Hand-drawn symbol recognition of surgical flowsheet graphs with deep image segmentation,” in 2020 IEEE 20th international conference on bioinformatics and bioengineering (BIBE), pp. 295–302, 2020.
J. Adhikari, M. Aththanayake, C. Kularathna, A. Wijayasiri, and A. Munasinghe, “Deep learning based hand-drawn molecular structure recognition and 3d visualisation using augmented reality,” in 2022 22nd International Conference on Advances in ICT for Emerging Regions (ICTer), pp. 31–38, 2022.
M. S. Thangakrishnan and K. Ramar, “Retracted article: Automated hand-drawn sketches retrieval and recognition using regularized particle swarm optimization based deep convolutional neural network,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 6, pp. 6407–6419, 2021.
S. Ali, N. Aslam, D. Kim, A. Abbas, S. Tufail, and B. Azhar, “Context awareness based sketch-deepnet architecture for hand-drawn sketches classification and recognition in aiot,” PeerJ Computer Science, vol. 9, p. e1186, 2023.
Z. Li, J. Yang, Y. Wang, M. Cai, X. Liu, and K. Lu, “Early diagnosis of parkinson’s disease using continuous convolution network: Handwriting recognition based on off-line hand drawing without template,” Journal of biomedical informatics, vol. 130, p. 104085, 2022.
X. Hou, X. Rong, and X. Yu, “Light-srnet: a lightweight dual-attention feature fusion network for hand-drawn sketch recognition,” Journal of Electronic Imaging, vol. 32, no. 1, pp. 013 005–013 005, 2023.
A. Keerthi Priya, N. Gaganashree, K. Hemalatha, J. S. Chembeti, and T. Kavitha, “Ai-based online hand drawn engineering symbol classification and recognition,” in Innovations in Electronics and Communication Engineering: Proceedings of the 9th ICIECE 2021, pp. 195–204, 2022.
J. Singh, K. Upreti, A. K. Gupta, N. Dave, A. Surana, and D. Mishra, “Deep learning approach for hand drawn emoji identification,” in 2022 IEEE International Conference on Current Development in Engineering and Technology (CCET), pp. 1–6, 2022.
S. Dey, A. Dutta, J. Llado´s, A. Forne´s, and U. Pal, “Shallow neural network model for hand-drawn symbol recognition in multi-writer scenario,” in 2017 14th IAPR international conference on document analysis and recognition (ICDAR), vol. 2, pp. 31–32, 2017.
H. Cecotti, C. Boumedine, and M. Callaghan, “Hand-drawn symbol recognition in immersive virtual reality using deep extreme learning machines,” in Recent Trends in Image Processing and Pattern Recognition: First International Conference, RTIP2R 2016, pp. 80–92, 2017.
S. Hayat, K. She, Y. Yu, and M. Mateen, “Deep cnn-based features for hand-drawn sketch recognition via transfer learning approach,” Editorial Preface From the Desk of Managing Editor, vol. 10, no. 9, 2019.
L. Akter et al., “Early identification of parkinson’s disease from hand-drawn images using histogram of oriented gradients and machine learning techniques,” in 2020 Emerging Technology in Computing, Communication and Electronics (ETCCE), pp. 1–6, 2020.
K. Simonyan, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
A. G. Howard et al., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
X. Dong, Z. Yu, W. Cao, Y. Shi, and Q. Ma, “A survey on ensemble learning,” Frontiers of Computer Science, vol. 14, pp. 241–258, 2020.
D. Pysal, S. J. Abdulkadir, S. R. Mohd Shukri, and H. Alhussian, “Classification of children’s drawing strategies on touch-screen of seriation objects using a novel deep learning hybrid model,” Alexandria Engineering Journal, vol. 60, no. 1, pp. 115–129, Feb. 2021.
M. F. Ahmadsaraei, A. Bastanfard, and A. Amini, “Child psychological drawing pattern detection on OBGET dataset, a case study on accuracy based on MYOLO v5 and MResNet 50,” Multimed. Tools Appl., vol. 83, no. 13, pp. 283–313, Apr. 2024.
F. Shi, W. Sun, H. Duan, X. Liu, M. Hu, W. Wang, and G. Zhai, “Drawing reveals hallmarks of children with autism,” Displays, vol. 67, p. 102000, Apr. 2021.
I. Kamran, S. Naz, I. Razzak, and M. Imran, “Handwriting dynamics assessment using deep neural network for early identification of Parkinson’s disease,” Future Gener. Comput. Syst., vol. 117, pp. 234–244, Apr. 2021.
J. Zhang, Y. Lee, T.-M. Chung, and H. Park, “Development of a Handwriting Drawings Assessment System for Early Parkinson’s Disease Identification with Deep Learning Methods,” in Future Data and Security Engineering. Big Data, Security and Privacy, pp. 484–499, 2023.
K. Nadeem, M. Ahmad, and M. Asif Habib, "Emotional States Detection Model from Handwriting by using Machine Learning," 2022 International Conference on Frontiers of Information Technology (FIT), pp. 284-289, 2022, doi: 10.1109/FIT57066.2022.00059.
A. A. Elngar, N. Jain, D. Sharma, H. Negi, A. Trehan, and A. Srivastava, “A Deep Learning Based Analysis of the Big Five Personality Traits from Handwriting Samples Using Image Processing,” Journal of Information Technology Management, vol. 12, pp. 3–35, Dec. 2020.
S. Ghosh, P. Shivakumara, P. Roy, U. Pal, and T. Lu, “Graphology based handwritten character analysis for human behaviour identification,” CAAI Trans. Intell. Technol., vol. 5, no. 1, pp. 55–65, Mar. 2020.
T. Mekhaznia, C. Djeddi, and S. Sarkar, “Personality Traits Identification Through Handwriting Analysis,” in Pattern Recognition and Artificial Intelligence, pp. 155–169, 2021.
A. Saraswal and U. R. Saxena, "Personality Trait Prediction Using Handwriting Recognition with KNN," 2022 International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), pp. 551-555, 2022, doi: 10.1109/CISES54857.2022.9844344.
Y. Hamdi et al., “Deep learned BLSTM for online handwriting modeling simulating the Beta-Elliptic approach,” Engineering Science and Technology, an International Journal, vol. 35, p. 101215, 2022.
Y. Hamdi, H. Boubaker, B. Rabhi, W. Ouarda, and A. Alimi, “Hybrid architecture based on rnn-svm for multilingual handwriting recognition using beta-elliptic and cnn models,” Authorea Preprints, 2023.
M. C. Data, M. Komorowski, D. C. Marshall, J. D. Salciccioli, and Y. Crutain, “Exploratory data analysis,” Secondary analysis of electronic health records, pp. 185–203, 2016.
C. Li, “Preprocessing methods and pipelines of data mining: An overview,” arXiv preprint arXiv:1906.08510, 2019.
V. C¸ etin and O. Yildiz, “A comprehensive review on data preprocessing techniques in data analysis,” Pamukkale U¨ niversitesi Mu¨hendislik Bilimleri Dergisi, vol. 28, no. 2, pp. 299–312, 2022.
A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative adversarial networks: An overview,” IEEE signal processing magazine, vol. 35, no. 1, pp. 53–65, 2018.
L. Girin, S. Leglaive, X. Bie, J. Diard, T. Hueber, and X. AlamedaPineda, “Dynamical variational autoencoders: A comprehensive review,” arXiv preprint arXiv:2008.12595, 2020.
K. Maharana, S. Mondal, and B. Nemade, “A review: Data preprocessing and data augmentation techniques,” Global Transitions Proceedings, vol. 3, no. 1, pp. 91–99, 2022.
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: analysis, applications, and prospects,” IEEE transactions on neural networks and learning systems, vol. 33, no. 12, pp. 6999–7019, 2021.
G. M. Devi and V. Neelambary, “Computer-aided diagnosis of white blood cell leukemia using vgg16 convolution neural network,” in 2022 4th International Conference on Inventive Research in Computing Applications (ICIRCA), pp. 1064–1068, 2022.
R. Kaur, R. Kumar, and M. Gupta, “Review on transfer learning for convolutional neural network,” in 2021 3rd International Conference on Advances in Computing, Communication Control and Networking (ICAC3N), pp. 922–926, 2021.
B. S. Kolla, B. R. Reddy, S. V. Sahithi, and L. P. Madala, “Comparative analysis of vgg19, resnet50, and googlenet inception models for bci,” Researchsquare, 2023.
Z.-P. Jiang, Y.-Y. Liu, Z.-E. Shao, and K.-W. Huang, “An improved vgg16 model for pneumonia image classification,” Applied Sciences, vol. 11, no. 23, p. 11185, 2021.
H. Qassim, D. Feinzimer, and A. Verma, “Residual squeeze vgg16,” arXiv preprint arXiv:1705.03004, 2017.
S. Mascarenhas and M. Agarwal, “A comparison between vgg16, vgg19 and resnet50 architecture frameworks for image classification,” in 2021 International conference on disruptive technologies for multidisciplinary research and applications (CENTCON), vol. 1, pp. 96–99, 2021.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520, 2018.
K. Dong, C. Zhou, Y. Ruan, and Y. Li, “Mobilenetv2 model for image classification,” in 2020 2nd International Conference on Information Technology and Computer Application (ITCA), pp. 476– 480, 2020.
Q. Xiang, X. Wang, R. Li, G. Zhang, J. Lai, and Q. Hu, “Fruit image classification based on mobilenetv2 with transfer learning technique,” in Proceedings of the 3rd international conference on computer science and application engineering, pp. 1–7, 2019.
B. J. Erickson and F. Kitamura, “Magician’s corner: 9. Performance metrics for machine learning models,” Radiology: Artificial Intelligence, vol. 3, no. 3, p. e200126, 2021.
G. Naidu, T. Zuva, and E. M. Sibanda, “A review of evaluation metrics in machine learning algorithms,” in Computer Science On-line Conference, pp. 15–25, 2023.
J. Davis and M. Goadrich, “The relationship between precision-recall and roc curves,” in Proceedings of the 23rd international conference on Machine learning, pp. 233–240, 2006.
T. Kynka¨a¨nniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila, “Improved precision and recall metric for assessing generative models,” Advances in neural information processing systems, vol. 32, 2019.
H. R. Sofaer, J. A. Hoeting, and C. S. Jarnevich, “The area under the precision-recall curve as a performance metric for rare binary events,” Methods in Ecology and Evolution, vol. 10, no. 4, pp. 565–577, 2019.
P. Mu¨ller, M. Brummel, and A. Braun, “Spatial recall index for machine learning algorithms,” in London Imaging Meeting, vol. 2, pp. 58–62, 2021.
R. Poojary and A. Pai, “Comparative study of model optimization techniques in fine-tuned cnn models,” in 2019 International Conference on Electrical and Computing Technologies and Applications (ICECTA), pp. 1–4, 2019.
E. M. Dogo, O. Afolabi, N. Nwulu, B. Twala, and C. Aigbavboa, “A comparative analysis of gradient descent-based optimization algorithms on convolutional neural networks,” in 2018 international conference on computational techniques, electronics and mechanical systems (CTEMS), pp. 92–99, 2018.
D. O. Melinte and L. Vladareanu, “Facial expressions recognition for human–robot interaction using deep convolutional neural networks with rectified adam optimizer,” Sensors, vol. 20, no. 8, p. 2393, 2020.
K. K. Kumar et al., “An efficient image classification of malaria parasite using convolutional neural network and adam optimizer,” Turkish Journal of Computer and Mathematics Education (TURCOMAT), vol. 12, no. 2, pp. 3376–3384, 2021.
Q. Wang, Y. Ma, K. Zhao, and Y. Tian, “A comprehensive survey of loss functions in machine learning,” Annals of Data Science, pp. 1–26, 2020.
F. Nie, Z. Hu, and X. Li, “An investigation for loss functions widely used in machine learning,” Communications in Information and Systems, vol. 18, no. 1, pp. 37–52, 2018.
A. Jung. Machine learning: the basics. Springer Nature, 2022.
Y. Zhang, J. Wen, G. Yang, Z. He, and J. Wang, “Path loss prediction based on machine learning: Principle, method, and data expansion,” Applied Sciences, vol. 9, no. 9, p. 1908, 2019.
J. T. Townsend, “Theoretical analysis of an alphabetic confusion matrix,” Perception & Psychophysics, vol. 9, pp. 40–50, 1971.
O. Caelen, “A bayesian interpretation of the confusion matrix,” Annals of Mathematics and Artificial Intelligence, vol. 81, no. 3, pp. 429–450, 2017.
N. D. Marom, L. Rokach, and A. Shmilovici, “Using the confusion matrix for improving ensemble classifiers,” in 2010 IEEE 26-th Convention of Electrical and Electronics Engineers in Israel, pp. 555–559, 2010.
S. Visa, B. Ramsay, A. L. Ralescu, and E. Van Der Knaap, “Confusion matrix-based feature selection.” Maics, vol. 710, no. 1, pp. 120–127, 2011.
B. P. Salmon, W. Kleynhans, C. P. Schwegmann, and J. C. Olivier, “Proper comparison among methods using a confusion matrix,” in 2015 IEEE International geoscience and remote sensing symposium (IGARSS), pp. 3057–3060, 2015.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Ali Ibrahim Khlaif, Mohamed Saber Naceur, Monji Kherallahr
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
