Integrating Multi-Sensors and AI to Develop Improved Surveillance Systems
DOI:
https://doi.org/10.18196/jrc.v6i2.25596Keywords:
Artificial Intelligence, Internet of Things, Surveillance Systems, Sensors, Sensor FusionAbstract
This paper explores advancements in surveillance systems, focusing on the integration of multisensory and AI technologies in urban and environmental monitoring. It highlights the fusion of data sources such as video feeds, LiDAR, and wireless networks for enhanced real-time surveillance in complex environments. Artificial intelligence (AI) plays a critical role in anomaly detection, object identification, and behavior analysis, improving response times in high-traffic and security-sensitive areas. However, these technologies raise privacy concerns, emphasizing the need for responsible data management and ethical frameworks. Also, there is probability of false positives which can lead to unnecessary action disturbing the normal mode of life. These technologies involve high financial requirements hence must be used judiciously. In current study human surveillance is carried out in indoor environments by two AI algorithms: YOLOV5 and R-CNN. The results of these algorithms can be fused with LiDAR data for better decision making. R-CNN produced better results than YOLOV5 but the fusion with sensor data led to accurate detection of humans in indoor environments. R-CNN showcased better results than YOLOV5. The future of surveillance should focus on balancing safety and personal rights while adapting policies to ensure privacy and accountability in an increasingly tech-driven world.
References
P. A. Jones et al., “Unifying public health surveillance: a scoping review of global use of the Youth Risk Behavior Survey,” Journal of Adolescent Health, vol. 75, no. 3, pp. 383-391, 2024.
S. Guo, X. Yu, Y. Sha, Y. Ju, M. Zhu, and J. Wang, “Online camera autocalibration appliable to road surveillance,” Machine Vision and Applications, vol. 35, no. 4, p. 91, 2024.
D. Montero, N. Aranjuelo, P. Leskovsky, E. Loyo, M. Nieto, and N. Aginako, “Multicamera BEV video-surveillance system for efficient monitoring of social distancing,” Multimedia Tools and Applications, vol. 82, no. 22, pp. 34995-35019, 2023.
K. Shim, K. Ko, J. Hwang, H. Jang, and C. Kim, “Fast online multitarget multicamera tracking for vehicles,” Applied Intelligence, vol. 53, no. 23, pp. 28994-29004, 2023.
B. B. Bhagat, R. R. Sharma, and D. Tilante, “Moving camera-based automated system for drone identification using focus measures,” Signal, Image and Video Processing, vol. 17, no. 6, pp. 2763-2770, 2023.
J. Alfred Daniel, C. Chandru Vignesh, B. A. Muthu, R. Senthil Kumar, C. B. Sivaparthipan, and C. E. M. Marin, “Fully convolutional neural networks for LIDAR–camera fusion for pedestrian detection in autonomous vehicle,” Multimedia Tools and Applications, vol. 82, no. 16, pp. 25107-25130, 2023.
A. C. Tran, L. Nguyen-Van, and N. Duong-Trung, “Toward real-time secured IP camera via state machine architecture,” International Journal of Information Technology, vol. 16, no. 1, pp. 329-336, 2024.
N. Perwaiz, M. M. Fraz, and M. Shahzad, “Smart surveillance with simultaneous person detection and reidentification,” Multimedia tools and applications, vol. 83, no. 5, pp. 15461-15482, 2024.
Y. Arslan and H. Canbolat, “Sound based alarming based video surveillance system design,” Multimedia Tools and Applications, vol. 81, no. 6, pp. 7969-7991, 2022.
S. Dodd, E. Antrobus, and M. Sydes, “Body-worn camera activation in prisons: Understanding correctional officers’ decision-making and use of discretion,” Security Journal, vol. 37, no. 2, pp. 354-374, 2024.
S. S. Brar and N. Jindal, “Camera Based Wearable Devices: A Strategic Survey from 2010 to 2021,” Wireless Personal Communications, vol. 133, no. 1, pp. 667-681, 2023.
S. Bereg et al., “Optimal placement of base stations in border surveillance using limited capacity drones,” Theoretical Computer Science, vol. 928, pp. 183-196, 2022.
R. Yang, K. Uchida, Y. Miyamoto, H. Arakawa, R. Hagita, and T. Aikawa, “Development of a ship-based camera monitoring system for floating marine debris,” Marine Pollution Bulletin, vol. 206, p. 116722, 2024.
J. Konrad, M. Cokbas, P. Ishwar, T. D. Little, and M. Gevelber, “High-accuracy people counting in large spaces using overhead fisheye cameras,” Energy and Buildings, vol. 307, p. 113936, 2024.
K. Aminiyeganeh, R. W. Coutinho, and A. Boukerche, “IoT video analytics for surveillance-based systems in smart cities,” Computer Communications, 2024.
Wang. X.; Wang. M.; Liu. X.; Mao. Y.; Chen. Y.; Dai. S. Surveillance-image-based outdoor air quality monitoring. Environmental Science and Ecotechnology, 2024, 18, pp.100319.
A. P. Garcia, C. K. Umezu, E. C. M. Polania, A. F. Dias Neto, R. Rossetto, and D. Albiero, “Sensor-based technologies in sugarcane agriculture,” Sugar Tech, vol. 24, no. 3, pp. 679-698, 2022.
S. B. Sukhavasi, S. B. Sukhavasi, K. Elleithy, S. Abuzneid, and A. Elleithy, “CMOS image sensors in surveillance system applications,” Sensors, vol. 21, no. 2, p. 488, 2021.
P. Singh et al., “Magnetic Induction Technology‐Based Wireless Sensor Network for Underground Infrastructure, Monitoring Soil Conditions, and Environmental Observation Applications: Challenges and Future Aspects,” Journal of Sensors, vol. 2022, no. 1, p. 9332917, 2022.
J. Baek et al., "Intelligent Multi-Sensor Detection System for Monitoring Indoor Building Fires," in IEEE Sensors Journal, vol. 21, no. 24, pp. 27982-27992, 2021.
M. Verma, R. S. Kaler, and M. Singh, “Sensitivity enhancement of Passive Infrared (PIR) sensor for motion detection,” Optik, vol. 244, p. 167503, 2021.
A. Vakil, J. Liu, P. Zulch, E. Blasch, R. Ewing, and J. Li, "A Survey of Multimodal Sensor Fusion for Passive RF and EO Information Integration," in IEEE Aerospace and Electronic Systems Magazine, vol. 36, no. 7, pp. 44-61, 2021.
S. Asadzadeh, W. J. de Oliveira, and C. R. de Souza Filho, “UAV-based remote sensing for the petroleum industry and environmental monitoring: State-of-the-art and perspectives,” Journal of Petroleum Science and Engineering, vol. 208, p. 109633, 2022.
G. Korotcenkov, N. P. Simonenko, E. P. Simonenko, V. V. Sysoev, and V. Brinzari, “Based Humidity Sensors as Promising Flexible Devices, State of the Art, Part 2: Humidity-Sensor Performances,” Nanomaterials, vol. 13, no. 8, p. 1381, 2023.
K. K. Gola and B. Gupta, “Underwater acoustic sensor networks: An energy efficient and void avoidance routing based on grey wolf optimization algorithm,” Arabian Journal for Science and Engineering, vol. 46, no. 4, pp. 3939-3954, 2021.
L. Nithy, B. David, K. Sudha, and C. Sunil, “Automatic Battery Replacement of Home Surveillance Robot using WSN,” Int. J. Aquat. Sci, vol. 12, pp. 671-676, 2021.
M. Hasanujjaman, M. Z. Chowdhury, and Y. M. Jang, “Sensor fusion in autonomous vehicle with traffic surveillance camera system: detection, localization, and AI networking,” Sensors, vol. 23, no. 6, p. 3335, 2023.
G. Soldi et al., "Space-Based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques," in IEEE Aerospace and Electronic Systems Magazine, vol. 36, no. 9, pp. 30-42, 2021.
J. Dudczyk, R. Czyba, and K. Skrzypczyk, “Multi-sensory data fusion in terms of UAV detection in 3D space,” Sensors, vol. 22, no. 12, p. 4323, 2022.
A. Abbasi, S. Queirós, N. M. da Costa, J. C. Fonseca, and J. Borges, “Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case,” Sensors, vol. 23, no. 8, p. 3993, 2023.
A. Munir, E. Blasch, J. Kwon, J. Kong, and A. Aved, "Artificial Intelligence and Data Fusion at the Edge," in IEEE Aerospace and Electronic Systems Magazine, vol. 36, no. 7, pp. 62-78, 2021.
M. V. Kulikova and G. Y. Kulikov, “MATLAB-based general approach for square-root extended-unscented and fifth-degree cubature Kalman filtering methods,” European Journal of Control, vol. 59, pp. 1-12, 2021.
M. Wu, Y. He, S. Luo, and W. Liu, “Particle filter-based real-time phase line bias estimation for GNSS-based attitude determination with common-clock receivers,” Advances in Space Research, vol. 71, no. 3, pp. 1682-1699, 2023.
A. Cartone and P. Postiglione, “Principal component analysis for geographical data: the role of spatial effects in the definition of composite indicators,” Spatial Economic Analysis, vol. 16, no. 2, pp. 126-147, 2021.
A. Mihalik et al., “Canonical correlation analysis and partial least squares for identifying brain–behavior associations: A tutorial and a comparative study,” Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, vol. 7, no. 11, pp. 1055-1067, 2022.
P. Liu, J. Li, L. Wang, and G. He, "Remote Sensing Data Fusion With Generative Adversarial Networks: State-of-the-art methods and future research directions," in IEEE Geoscience and Remote Sensing Magazine, vol. 10, no. 2, pp. 295-328, 2022.
X. Duan, Q. Fan, W. Bi, and A. Zhang, “Belief Exponential Divergence for DS Evidence Theory and its Application in Multi-Source Information Fusion,” Journal of Systems Engineering and Electronics, vol. 35, no. 6, pp. 1454-1468, 2024.
S. Mandal, A. Yadav, F. A. Panme, K. M. Devi, and S. K. SM, “Adaptation of smart applications in agriculture to enhance production,” Smart agricultural technology, p. 100431, 2024.
A. Srivastava, T. Badal, P. Saxena, A. Vidyarthi, and R. Singh, “UAV surveillance for violence detection and individual identification,” Automated Software Engineering, vol. 29, no. 1, p. 28, 2022.
Y. Himeur, K. Ghanem, A. Alsalemi, F. Bensaali, and A. Amira, “Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives,” Applied Energy, vol. 287, p. 116601, 2021.
M. Hosseinzadeh et al., “Data cleansing mechanisms and approaches for big data analytics: a systematic study,” Journal of Ambient Intelligence and Humanized Computing, pp. 1-13, 2023.
B. Bala and S. Behal, "A Brief Survey of Data Preprocessing in Machine Learning and Deep Learning Techniques," 2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), pp. 1755-1762, 2024.
M. Samani, A. Khademzadeh, and K. Badie, “A Novel Algorithm for Redundant Data Filtering in WSN and RFID Integrated Networks,” Nexo Revista Científica, vol. 35, no. 4, pp. 1078-1090, 2022.
D. Ruano-Ordás, “Machine Learning-Based Feature Extraction and Selection,” Applied Sciences, vol. 14, no. 15, p. 6567, 2024.
S. Abbasian Dehkordi, K. Farajzadeh, J. Rezazadeh, R. Farahbakhsh, K. Sandrasegaran, and M. Abbasian Dehkordi, “A survey on data aggregation techniques in IoT sensor networks,” Wireless Networks, vol. 26, no. 2, pp. 1243-1263, 2020.
C. H. Chen. Computational Intelligence And Image Processing In Medical Applications (Vol. 8). World Scientific, 2022.
M. Aubey, K. M. Britt, and K. Gold, “Policing, Power, and Protests: Landscapes of Surveillance in Private and Public Spaces in Lower Manhattan,” Historical Archaeology, pp. 1-18, 2024.
A. Zhaxalikov, A. Mombekov, and Z. Sotsial, “Surveillance Camera Using Wi-Fi Connection,” Procedia Computer Science, vol. 231, pp. 721-726, 2024.
F. A. S. Oliveira, R. J. S. Castro, A. Silva-Júnior, R. E. E. Pinheiro, R. A. P. Sfaciotte, and D. G. G. Schwarz, “Cross-border surveillance of sheep and goat rabies in Brazil,” Comparative Immunology, Microbiology and Infectious Diseases, p. 102233, 2024.
A. Soularidis, K. I. Kotis, and G. A. Vouros, “Real-Time Semantic Data Integration and Reasoning in Life-and Time-Critical Decision Support Systems,” Electronics, vol. 13, no. 3, p. 526, 2024.
N. Wang, N. Mutzner, and K. Blanchet, “Societal acceptance of urban drones: A scoping literature review,” Technology in Society, p. 102377, 2023.
S. P. Chu, F. S. Pezzella, and J. D. Evans, “Surveillance load: A burden of search borne by Black and Brown bodies,” Critical Criminology, vol. 31, no. 2, pp. 451-466, 2023.
M. N. Uddin and H. Nyeem, “Engineering a multisensor surveillance system with secure alerting for next-generation threat detection and response,” The results in Engineering, vol. 22, p. 101984, 2024.
G. P. Lakshmi, P. N. Asha, G. Sandhya, S. V. Sharma, S. Shilpashree, and S. G. Subramanya, “An intelligent IOT sensor coupled precision irrigation model for agriculture,” Measurement: Sensors, vol. 25, p. 100608, 2023.
F. Ahmed, T. A. M. R. Shahriar, R. Paul, and A. Ahammad, "Design and Development of a Smart Surveillance System for Security of an Institution," 2021 International Conference on Electronics, Communications and Information Technology (ICECIT), pp. 1-4, 2021.
H. Miawarni, T. A. Sardjono, E. Setijadi, Wijayanti, and M. H. Purnomo, "Fall Detection System for Elderly (FDS-E) using Low-cost Camera Based on LSTM and OpenPose," 2023 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1-6, 2023.
C. Liu, “More-than-human practices of making and unmaking the smart home: A socioemotional investigation,” Emotion, Space and Society, vol. 49, p. 100982, 2023.
N. Yu, “Fault location measurement of sensor nodes based on fuzzy control algorithm,” Soft Computing, vol. 27, no. 2, pp. 1189-1199, 2023.
A. L. Imoize, O. A. Ajibola, T. R. Oyedare, J. O. Ogbebor, and S. O. Ajose, “Development of an energy-efficient wireless sensor network model for perimeter surveillance,” International Journal of Electrical Engineering and Applied Sciences (IJEEAS), vol. 4, no. 1, 2021.
P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, “A Review of Yolo algorithm developments,” Procedia computer science, vol. 199, pp. 1066-1073, 2022.
T. Diwan, G. Anirudh, and J. V. Tembhurne, “Object detection using YOLO: Challenges, architectural successors, datasets and applications,” multimedia Tools and Applications, vol. 82, no. 6, pp. 9243-9275, 2023.
M. L. Ali and Z. Zhang, “The YOLO framework: A comprehensive review of evolution, applications, and benchmarks in object detection,” Computers, vol. 13, no. 12, p. 336, 2024.
Z. Hao et al., “Automated tree-crown and height detection in a young forest plantation using mask region-based convolutional neural network (Mask R-CNN),” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 178, pp. 112-123, 2021.
A. Z. Syaharuddin, Z. Zainuddin, and Andani, "Multi-Pole Road Sign Detection Based on Faster Region-based Convolutional Neural Network (Faster R-CNN)," 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), 2021, pp. 1-5, 2021.
C. Yan, H. Zhang, X. Li, and D. Yuan, “R-SSD: Refined single shot multibox detector for pedestrian detection,” Applied Intelligence, vol. 52, no. 9, pp. 10430-10447, 2022.
S. A. Magalhães et al., “Evaluating the single-shot multibox detector and YOLO deep learning models for the detection of tomatoes in a greenhouse,” Sensors, vol. 21, no. 10, p. 3569, 2021.
R. D. Altarez, “Faster R–CNN, RetinaNet and Single Shot Detector in different ResNet backbones for marine vessel detection using cross polarization C-band SAR imagery,” Remote Sensing Applications: Society and Environment, vol. 36, p. 101297, 2024.
S. Liu, T. Cai, X. Tang, Y. Zhang, and C. Wang, “Visual recognition of traffic signs in natural scenes based on improved RetinaNet,” Entropy, vol. 24, no. 1, p. 112, 2022
S. Chen et al., “Info-FPN: An Informative Feature Pyramid Network for object detection in remote sensing images,” Expert Systems with Applications, vol. 214, p. 119132, 2023.
L. Zhu, F. Lee, J. Cai, H. Yu, and Q. Chen, “An improved feature pyramid network for object detection,” Neurocomputing, vol. 483, pp. 127-139, 2022.
G. Zamanakos, L. Tsochatzidis, A. Amanatiadis, and I. Pratikakis, “A comprehensive survey of LIDAR-based 3D object detection methods with deep learning for autonomous driving,” Computers & Graphics, vol. 99, pp. 153-181, 2021.
W. Zhangyu, Y. Guizhen, W. Xinkai, L. Haoran, and L. Da, "A Camera and LiDAR Data Fusion Method for Railway Object Detection," in IEEE Sensors Journal, vol. 21, no. 12, pp. 13442-13454, 2021.
H. Zhu et al., "VPFNet: Improving 3D Object Detection With Virtual Point Based LiDAR and Stereo Data Fusion," in IEEE Transactions on Multimedia, vol. 25, pp. 5291-5304, 2023.
Y. C. Fan, C. M. Yelamandala, T. W. Chen, and C. J. Huang, “Real‐Time Object Detection for LiDAR Based on LS‐R‐YOLOv4 Neural Network,” Journal of Sensors, vol. 2021, no. 1, p. 5576262, 2021.
C. Wisultschew, G. Mujica, J. M. Lanza-Gutierrez, and J. Portilla, "3D-LIDAR Based Object Detection and Tracking on the Edge of IoT for Railway Level Crossing," in IEEE Access, vol. 9, pp. 35718-35729, 2021.
J. Fang et al., "LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection," 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 14822-14829, 2024.
D. Jiang, G. Li, C. Tan, L. Huang, Y. Sun, and J. Kong, “Semantic segmentation for multiscale target based on object recognition using the improved Faster-RCNN model,” Future Generation Computer Systems, vol. 123, pp. 94-104, 2021.
R. F. Mansour, J. Escorcia-Gutierrez, M. Gamarra, J. A. Villanueva, and N. Leal, “Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model,” Image and Vision Computing, vol. 112, p. 104229, 2021.
Y. Zhu et al., “Faster-RCNN based intelligent detection and localization of dental caries,” Displays, vol. 74, p. 102201, 2022.
S. Singh, U. Ahuja, M. Kumar, K. Kumar, and M. Sachdeva, “Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment,” Multimedia Tools and Applications, vol. 80, pp. 19753-19768, 2021.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Preeti Mohanty, Manu S R, Shreyas M, Vishnumahanthi Uttam, Bhagya R Navada, Sravani V, Santhosh K V
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
