Improving CBIR Techniques with Deep Learning Approach: An Ensemble Method Using NASNetMobile, DenseNet121, and VGG12

Shereen Saleem Sadiq

Abstract


In the evolving field of Content-Based Image Retrieval (CBIR), we introduce a novel approach that integrates deep learning models—NASNetMobile, DenseNet121, and VGG16—with ensemble methods to enhance retrieval accuracy and relevance. This study uniquely combines NASNetMobile's adaptability, DenseNet121's feature extraction, and VGG16's robustness through hard and soft voting techniques, aiming to effectively bridge the semantic gap in CBIR systems. Our comparative analysis against existing CBIR algorithms using diverse online datasets demonstrates superior performance, with our approach achieving up to 98% in accuracy, precision, recall, and F1-score, thereby redefining performance benchmarks. This advancement proves particularly impactful in medical imaging and surveillance, where precise image retrieval is crucial. Our research contributes to CBIR by (1) demonstrating the integrated deep learning ensemble's ability to narrow the semantic gap and (2) providing a comparative performance analysis, underscoring our method's improvement over current technologies. The combination of these models marks a significant step forward in CBIR, offering a more accurate and efficient solution for image retrieval challenges.

Keywords


Content-Based Image Retrieval (CBIR); Deep Learning; Ensemble Learning; NASNetMobile; DenseNet121; VGG16; Image Retrieval Accuracy.

Full Text:

PDF

References


D. Srivastava, S. S. Singh, B. Rajitha, M. Verma, M. Kaur, and H.-N. Lee, “Content-Based Image Retrieval: A Survey on Local and Global Features Selection, Extraction, Representation, and Evaluation Parameters,” IEEE Access, vol. 11, pp. 95410–95431, 2023, doi: 10.1109/ACCESS.2023.3308911.

S. Iqbal, A. N. Qureshi, M. Alhussein, I. A. Choudhry, K. Aurangzeb, and T. M. Khan, "Fusion of Textural and Visual Information for Medical Image Modality Retrieval Using Deep Learning-Based Feature Engineering," in IEEE Access, vol. 11, pp. 93238-93253, 2023, doi: 10.1109/ACCESS.2023.3310245.

D. C. Lepcha, B. Goyal, A. Dogra, and V. Goyal, “Image super-resolution: A comprehensive review, recent trends, challenges and applications,” Information Fusion, vol. 91, pp. 230–260, 2023, doi: 10.1016/j.inffus.2022.10.007.

K. Zhou, W. Wang, L. Huang, and B. Liu, “Comparative study on the time series forecasting of web traffic based on statistical model and Generative Adversarial model,” Knowledge-Based Systems, vol. 213, p. 106467, 2021, doi: 10.1016/j.knosys.2020.106467.

L. Alzubaidi et al., “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions,” J. Big Data, vol. 8, no. 1, pp. 53–74, 2021, doi: 10.1186/s40537-021-00444-8.

M. Elahi, S. O. Afolaranmi, J. L. Martinez Lastra, and J. A. Perez Garcia, “A comprehensive literature review of the applications of AI techniques through the lifecycle of industrial equipment,” Discov. Artif. Intell., vol. 3, no. 1, pp. 43–78, 2023, doi: 10.1007/s44163-023-00089-x.

J. Ma, X. Jiang, A. Fan, J. Jiang, and J. Yan, “Image Matching from Handcrafted to Deep Features: A Survey,” Int. J. Comput. Vision, vol. 129, no. 1, pp. 23–79, 2021, doi: 10.1007/s11263-020-01359-2.

T. D. Akinosho et al., “Deep learning in the construction industry: A review of present status and future innovations,” Journal of Building Engineering, vol. 32, p. 101827, 2020, doi: 10.1016/j.jobe.2020.101827.

S. A. Singh, A. S. Kumar, and K. A. Desai, “Comparative assessment of common pre-trained CNNs for vision-based surface defect detection of machined components,” Expert Syst. Appl., vol. 218, p. 119623, 2023, doi: 10.1016/j.eswa.2023.119623.

A. Naeem, T. Anees, K. T. Ahmed, R. A. Naqvi, S. Ahmad, and T. Whangbo, “Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval,” Complex Intell. Syst., vol. 9, no. 2, pp. 1729–1751, 2023, doi: 10.1007/s40747-022-00866-8.

K. T. Ahmed, S. Jaffar, M. G. Hussain, S. Fareed, A. Mehmood, and G. S. Choi, “Maximum Response Deep Learning Using Markov, Retinal & Primitive Patch Binding With GoogLeNet & VGG-19 for Large Image Retrieval,” IEEE Access, vol. 9, pp. 41934–41957, 2021, doi: 10.1109/ACCESS.2021.3063545.

L. R. Nair, K. Subramaniam, G. K. D. PrasannaVenkatesan, P. S. Baskar, and T. Jayasankar, “RETRACTED ARTICLE: Essentiality for bridging the gap between low and semantic level features in image retrieval systems: an overview,” J. Ambient Intell. Hum. Comput., vol. 12, no. 6, pp. 5917–5929, 2021, doi: 10.1007/s12652-020-02139-z.

A. Naeem, T. Anees, K. T. Ahmed, R. A. Naqvi, S. Ahmad, and T. Whangbo, “Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval,” Complex Intell. Syst., vol. 9, no. 2, pp. 1729–1751, 2023, doi: 10.1007/s40747-022-00866-8.

X. Wu, D. Sahoo, and S. C. H. Hoi, “Recent advances in deep learning for object detection,” Neurocomputing, vol. 396, pp. 39–64, 2020, doi: 10.1016/j.neucom.2020.01.085.

M. Malik, M. K. Malik, K. Mehmood, and I. Makhdoom, “Automatic speech recognition: a survey,” Multimed. Tools Appl., vol. 80, no. 6, pp. 9411–9457, 2021, doi: 10.1007/s11042-020-10073-7.

C. Zheng et al., “Deep Learning-based Human Pose Estimation: A Survey,” ACM Comput. Surv., vol. 56, no. 1, pp. 1–37, 2023, doi: 10.1145/3603618.

K. R. Chowdhary, “Natural Language Processing. Fundamentals of Artificial Intelligence,” Fundamentals of artificial intelligence, pp. 603-649, 2020, doi: 10.1007/978-81-322-3972-7_19.

S. Gkelios, A. Sophokleous, S. Plakias, Y. Boutalis, and S. A. Chatzichristofis, “Deep convolutional features for image retrieval,” Expert Syst. Appl., vol. 177, p. 114940, 2021, doi: 10.1016/j.eswa.2021.114940.

Y. Zhao et al., “Classification of Zambian grasslands using random forest feature importance selection during the optimal phenological period,” Ecol. Indic., vol. 135, p. 108529, 2022, doi: 10.1016/j.ecolind.2021.108529.

J. Estévez et al., “Gaussian processes retrieval of crop traits in Google Earth Engine based on Sentinel-2 top-of-atmosphere data,” Remote sensing of environment, vol. 273, p. 112958, 2022.

M. K. Alsmadi, “Content-Based Image Retrieval Using Color, Shape and Texture Descriptors and Features,” Arab. J. Sci. Eng., vol. 45, no. 4, pp. 3317–3330, 2020, doi: 10.1007/s13369-020-04384-y.

M. Garg and G. Dhiman, “A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants,” Neural Comput. &. Applic., vol. 33, no. 4, pp. 1311–1328, 2021, doi: 10.1007/s00521-020-05017-z.

H. Lu, M. Zhang, X. Xu, Y. Li, and H. T. Shen, “Deep Fuzzy Hashing Network for Efficient Image Retrieval,” IEEE Trans. Fuzzy Syst., vol. 29, no. 1, pp. 166–176, 2020, doi: 10.1109/TFUZZ.2020.2984991.

H. Wang, Z. Li, Y. Li, B. B. Gupta, and C. Choi, “Visual saliency guided complex image retrieval,” Pattern Recognit. Lett., vol. 130, pp. 64–72, 2020, doi: 10.1016/j.patrec.2018.08.010.

X. Zhang, L. Wang, and Y. Su, “Visual place recognition: A survey from deep learning perspective,” Pattern Recognit., 113, 107760, 2021, doi: 10.1016/j.patcog.2020.107760.

A. Naeem, T. Anees, K. T. Ahmed, R. A. Naqvi, S. Ahmad, and T. Whangbo, “Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval,” Complex Intell. Syst., vol. 9, no. 2, pp. 1729–1751, 2023, doi: 10.1007/s40747-022-00866-8.

G.-H. Liu and J.-Y. Yang, “Exploiting deep textures for image retrieval,” Int. J. Mach. Learn. Cybern., vol. 14, no. 2, pp. 483–494, 2023, doi: 10.1007/s13042-022-01645-0.

S. R. Waheed, M. S. M. Rahim, N. M. Suaib, and A. A. Salim, “CNN deep learning-based image to vector depiction,” Multimed. Tools Appl., vol. 82, no. 13, pp. 20283–20302, 2023, doi: 10.1007/s11042-023-14434-w.

X. Wang et al., “RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval,” Med. Image Anal., vol. 83, p. 102645, 2023, doi: 10.1016/j.media.2022.102645.

D. Y. Y. Tang et al., “Application of regression and artificial neural network analysis of Red-Green-Blue image components in prediction of chlorophyll content in microalgae. Bioresour. Technol., 370, 128503, 2023, doi: 10.1016/j.biortech.2022.128503.

G. Dhiman, “Multi-modal active learning with deep reinforcement learning for target feature extraction in multi-media image processing applications,” Multimed. Tools Appl., vol. 82, no. 4, pp. 5343–5367, 2023, doi: 10.1007/s11042-022-12178-7.

S. Y. Alaba and J. E. Ball, “Deep Learning-Based Image 3-D Object Detection for Autonomous Driving: Review,” IEEE Sens. J., vol. 23, no. 4, pp. 3378–3394, 2023, doi: 10.1109/JSEN.2023.3235830.

A. Sharma and S. Kumar, “Machine learning and ontology-based novel semantic document indexing for information retrieval,” Comput. Ind. Eng., vol. 176, p. 108940, 2023, doi: 10.1016/j.cie.2022.108940.

S. Sakshi and V. Kukreja, “Image Segmentation Techniques: Statistical, Comprehensive, Semi-Automated Analysis and an Application Perspective Analysis of Mathematical Expressions,” Arch. Comput. Methods Eng., vol. 30, no. 1, pp. 457–495, 2023, doi: 10.1007/s11831-022-09805-9.

S. Bickel, B. Schleich, and S. Wartzack, “A Novel Shape Retrieval Method for 3D Mechanical Components Based on Object Projection, Pre-Trained Deep Learning Models and Autoencoder,” Comput.-Aided Des., vol. 154, p. 103417, 2023, doi: 10.1016/j.cad.2022.103417.

L. Putzu, L. Piras, and G. Giacinto, “Convolutional neural networks for relevance feedback in content based image retrieval,” Multimed. Tools Appl., vol. 79, no. 37, pp. 26995–27021, 2020, doi: 10.1007/s11042-020-09292-9.

V. Kumar, V. Tripathi, and B. Pant, "Content based Fine-Grained Image Retrieval using Convolutional Neural Network," 2020 7th International Conference on Signal Processing and Integrated Networks (SPIN), pp. 1120-1125, 2020, doi: 10.1109/SPIN48934.2020.9071334.

D. Walkoli, V. Sali, S. Patil, R. Sonawane, and B. Mahalakshmi, "Content-Based Image Retrieval using SIFT and CNN," 2021 Asian Conference on Innovation in Technology (ASIANCON), pp. 1-5, 2021, doi: 10.1109/ASIANCON51346.2021.9544699.

R. Kapoor, D. Sharma, and T. Gulati, “State of the art content based image retrieval techniques using deep learning: a survey,” Multimed. Tools Appl., vol. 80, no. 19, pp. 29561–29583, 2021, doi: 10.1007/s11042-021-11045-1.

K. Zhang, “Content-based image retrieval with a Convolutional Siamese Neural Network: Distinguishing lung cancer and tuberculosis in CT images,” Comput. Biol. Med., vol. 140, p. 105096, 2022, doi: 10.1016/j.compbiomed.2021.105096.

U. A. Khan, A. Javed, R. Ashraf, “An effective hybrid framework for content based image retrieval (CBIR),” Multimed. Tools Appl., vol. 80, no. 17, pp. 26911–26937, 2021, doi: 10.1007/s11042-021-10530-x.

M. S. Ghaleb, H. M. Ebied, H. A. Shedeed, and M. F. Tolba, "Content-based Image Retrieval based on Convolutional Neural Networks," 2021 Tenth International Conference on Intelligent Computing and Information Systems (ICICIS), pp. 149-153, 2021, doi: 10.1109/ICICIS52592.2021.9694146.

S. Agrawal, A. Chowdhary, S. Agarwala, V. Mayya, and S. S. Kamath, “Content-based medical image retrieval system for lung diseases using deep CNNs,” Int. j. inf. tecnol., vol. 14, no. 7, pp. 3619–3627, 2022, doi: 10.1007/s41870-022-01007-7.

R. Bibi, Z. Mehmood, R. M. Yousaf, T. Saba, M. Sardaraz, and A. Rehman, “Query-by-visual-search: multimodal framework for content-based image retrieval,” J. Ambient Intell. Hum. Comput., vol. 11, no. 11, pp. 5629–5648, 2020, doi: 10.1007/s12652-020-01923-1.

P. Desai, J. Pujari, C. Sujatha, A. Kamble, and A. Kambli, “Hybrid Approach for Content-Based Image Retrieval using VGG16 Layered Architecture and SVM: An Application of Deep Learning,” SN Comput. Sci., vol. 2, no. 3, pp. 170–179, 2021, doi: 10.1007/s42979-021-00529-4.

F. Ahmad and T. Ahmad, “Content Based Image Retrieval System Based on Deep Convolution Neural Network Model by Integrating Three-Fold Geometric Augmentation,” Opt. Mem. Neural Networks, vol. 30, no. 3, pp. 236–249, 2021, doi: 10.3103/S1060992X21030061.

M. S. Ghaleb, H. M. Ebied, H. A. Shedeed, and M. F. Tolba, “Content-Based Image Retrieval Using Fused Convolutional Neural Networks,” Proceedings of the 8th International Conference on Advanced Intelligent Systems and Informatics, pp. 260-270, 2022, doi: 10.1007/978-3-031-20601-6_24.

V. T. H. Tuyet, N. T. Binh, N. K. Quoc, and A. Khare, “Content Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning,” Mobile Netw. Appl., vol. 26, no. 3, pp. 1300–1310, 2021, doi: 10.1007/s11036-021-01762-0.

X. Zhang, C. Bai, and K. Kpalma, “OMCBIR: Offline mobile content-based image retrieval with lightweight CNN optimization,” Displays, vol. 76, p. 102355, 2023, doi: 10.1016/j.displa.2022.102355.

P. Desai and J. Pujari, “Artificial Intelligence Framework for Content-Based Image Retrieval: Performance Analysis,” Congress on Intelligent Systems, vol. 2, pp. 535-547, 2022, doi: 10.1007/978-981-16-9113-3_39.

V. S. Mahalle, N. M. Kandoi, and S. B. Patil, “A powerful method for interactive content-based image retrieval by variable compressed convolutional info neural networks,” Vis. Comput., pp. 1–27, 2023, doi: 10.1007/s00371-023-03104-5.

G. V. R. M. Kumar and D. Madhavi, “Stacked Siamese Neural Network (SSiNN) on Neural Codes for Content-Based Image Retrieval,” IEEE Access, vol. 11, pp. 77452–77463, 2023, doi: 10.1109/ACCESS.2023.3298216.

D. Pathak and U. S. N. Raju, “Shuffled-Xception-DarkNet-53: A content-based image retrieval model based on deep learning algorithm,” Comput. Electr. Eng., vol. 107, p. 108647, 2023, doi: 10.1016/j.compeleceng.2023.108647.

D. K. Sudhish, L. R. Nair, and S. Shailesh, “Content-based image retrieval for medical diagnosis using fuzzy clustering and deep learning,” Biomed. Signal Process. Control, vol. 88, p. 105620, 2024, doi: 10.1016/j.bspc.2023.105620.

R. Shetty, V. S. Bhat, and J. Pujari, “Content-based medical image retrieval using deep learning-based features and hybrid meta-heuristic optimization,” Biomed. Signal Process. Control, vol. 92, p. 106069, 2024, doi: 10.1016/j.bspc.2024.106069.

S. Öztürk, E. Çelik, and T. Çukur, “Content-based medical image retrieval with opponent class adaptive margin loss,” Inform. Sci., vol. 637, p. 118938, 2023, doi: 10.1016/j.ins.2023.118938.

Z. Hu and A. G. Bors, “Co-attention enabled content-based image retrieval,” Neural Networks, vol. 164, pp. 245–263, 2023, doi: 10.1016/j.neunet.2023.04.009.

M. K. Kelishadrokhi, M. Ghattaei, and S. Fekri-Ershad, “Innovative local texture descriptor in joint of human-based color features for content-based image retrieval,” SIViP., vol. 17, no. 8, pp. 4009–4017, 2023, doi: 10.1007/s11760-023-02631-x.

K. Kakizaki, K. Fukuchi, and J. Sakuma, “Certified defense for content based image retrieval,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4561-4570, 2023.

N. Arora, A. Kakde, and S. C. Sharma, “An optimal approach for content-based image retrieval using deep learning on COVID-19 and pneumonia X-ray Images,” Int. J. Syst. Assur. Eng. Manag., vol. 14, no. 1, pp. 246–255, 2023, doi: 10.1007/s13198-022-01846-4.

Y. Shen, J. Qin, J. Chen, M. Yu, L. Liu, F. Zhu, F. Shen, and L. Shao, “Auto-encoding twin-bottleneck hashing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2818– 2827, 2020.

J. I. Forcen, M. Pagola, E. Barrenechea, and H. Bustince, “Co-occurrence of deep convolutional features for image search,” Image Vision Comput., vol. 97, p. 103909, 2020, doi: 10.1016/j.imavis.2020.103909.

R. Wang, R. Wang, S. Qiao, S. Shan, and X. Chen, “Deep positionaware hashing for semantic continuous image retrieval,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 2493–2502, 2020.

H. A. Al-Jubouri and S. M. Mahmmod, “A comparative analysis of automatic deep neural networks for image retrieval,” TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 19, no. 3, pp. 858–871, 2021, doi: 10.12928/telkomnika.v19i3.18157.

S. Camalan et al., “OtoMatch: Content-based eardrum image retrieval using deep learning,” PLoS One, vol. 15, no. 5, p. e0232776, 2020, doi: 10.1371/journal.pone.0232776.

R. Chen, L. Pan, Y. Zhou, and Q. Lei, “Image Retrieval Based on Deep Feature Extraction and Reduction with Improved CNN and PCA,” Journal of Information Hiding and Privacy Protection, vol. 2, no. 2, pp. 67–76, 2020, doi: 10.32604/jihpp.2020.010472.




DOI: https://doi.org/10.18196/jrc.v5i3.21805

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Shereen Saleem Sadiq

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik