Online Digital Image Stabilization for an Unmanned Aerial Vehicle (UAV)
DOI:
https://doi.org/10.18196/jrc.2484Keywords:
K-means, Kalman filter, image stabilization, optical flow, UAVAbstract
The Unmanned Aerial Vehicle (UAV) video system uses a portable camera mounted on the robot to monitor scene activities. In general, UAVs have very little stabilization equipment, so getting good and stable images of UAVs in real-time is still a challenge. This paper presents a novel framework for digital image stabilization for online applications using a UAV. This idea aims to solve the problem of unwanted vibration and motion when recording video using a UAV. The proposed method is based on dense optical flow to select features representing the displacement of two consecutive frames. K-means clustering is used to find the cluster of the motion vector field that has the largest members. The centroid of the largest cluster was chosen to estimate the rigid transform motion that handles rotation and translation. Then, the trajectory is compensated using the Kalman filter. The experimental results show that the proposed method is suitable for online video stabilization and achieves an average computation time performance of 47.5 frames per second (fps).References
Z. Zheng, Y. Ma, H. Zheng, Y. Gu, and M. Lin, “Industrial part localization and grasping using a robotic arm guided by 2D monocular vision,” Ind. Rob., vol. 45, no. 6, pp. 794–804.
B. N. Vahrenkamp and T. Asfour, “Simultaneous grasp and motion planning: Humanoid robot ARMAR-III,” IEEE Robotics and Automation Magazine, vol. 19, no. 2, pp. 43–57, 2012.
W. Rahmaniar and A. Wicaksono, “Design and Implementation of a Mobile Robot for Carbon Monoxide Monitoring,” J. Robot. Control, vol. 2, no. 1, 2021.
M. Dunbabin and L. Marques, “Robots for environmental monitoring: Significant advancements and applications,” IEEE Robot. Autom. Mag., vol. 19, no. 1, pp. 24–39, 2012.
W. Rahmaniar, W. Wang, and H. Chen, “Real-time detection and recognition of multiple moving objects for aerial surveillance,” Electronics, vol. 8, no. 12, pp. 1373-1390, 2019.
W. C. Hu, C. H. Chen, T. Y. Chen, D. Y. Huang, and Z. C. Wu, “Moving object detection and tracking from video captured by moving camera,” J. Vis. Commun. Image Represent., vol. 30, pp. 164–180, 2015.
Y. Lu, Z. Xue, G.-S. Xia, and L. Zhang, “A survey on vision-based UAV navigation,” Geo-spatial Inf. Sci., vol. 21, no. 1, pp. 21–32, 2018.
S. Minaeian, J. Liu, and Y.-J. Son, “Vision-based target detection and localization via a team of cooperative UAV and UGVs,” IEEE Trans. Syst. Man, Cybern. Syst., vol. 46, no. 7, pp. 1005–1016, Jul. 2016.
D. H. Yeom, “Optical image stabilizer for digital photographing apparatus,” IEEE Trans. Consum. Electron., vol. 55, no. 3, pp. 1028–1031, 2009.
P. Pournazari, R. Nagamune, and M. Chiao, “A concept of a magnetically-actuated optical image stabilizer for mobile applications,” IEEE Trans. Consum. Electron., vol. 60, no. 1, pp. 10–17, 2014.
M. Hashimoto, T. Kuno, and H. Sugiura, “A new image-stabilizing method by transferring electric charges,” in Proc. of Digest of Technical Papers - IEEE International Conference on Consumer Electronics, 2007, pp. 1-2.
C. Song, H. Zhao, W. Jing, and H. Zhu, “Robust video stabilization based on particle filtering with weighted feature points,” IEEE Trans. Consum. Electron., vol. 58, no. 2, pp. 570–577, 2012.
J. H. Woo, J. H. Yoon, Y. J. Hur, N. C. Park, Y. P. Park, and K. S. Park, “Optimal design of a ferromagnetic yoke for reducing crosstalk in optical image stabilization actuators,” IEEE Transactions on Magnetics, 2011, vol. 47, no. 10, pp. 4298–4301.
C. Kim, M. G. Song, Y. Kim, N. C. Park, K. S. Park, and Y. P. Park, “Design of a new triple electro-magnetic optical image stabilization actuator to compensate for hand trembling,” Microsyst. Technol., vol. 18, no. 9–10, pp. 1323–1334, 2012.
T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T. Imaide, “Electronic image stabilizer for video camera use,” IEEE Trans. Consum. Electron., vol. 36, no. 3, pp. 520–525, 1990.
Y. Zhang and M. Xie, “Robust digital image stabilization technique for car camera,” Inf. Technol. J., vol. 10, no. 2, pp. 335–347, 2011.
L. Kejriwal and I. Singh, “A hybrid filtering approach of digital video stabilization for UAV using kalman and low pass filter,” Procedia Comput. Sci., vol. 93, no. September, pp. 359–366, 2016.
J. Li, T. Xu, and K. Zhang, “Real-time feature-based video stabilization on FPGA,” IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 4, pp. 907–919, 2017.
B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. of International Joint Conference on Artificial Intelligence, 1981, pp. 674--679.
A. Lim, B. Ramesh, Y. Yang, C. Xiang, Z. Gao, and F. Lin, “Real-time optical flow-based video stabilization for unmanned aerial vehicles,” J. Real-Time Image Process., pp. 1–11, 2017.
C. Feichtenhofer and A. Pinz, “Good features to track,” in Proc. of IEEE International Conference on Computer Vision, 1994, pp. 246–253.
Y. G. Ryu and M. J. Chung, “Robust online digital image stabilization based on point-feature trajectory without accumulative global motion estimation,” IEEE Signal Process. Lett., vol. 19, no. 4, pp. 223–226, 2012.
C. Wang, J. Kim, K. Byun, J. Ni, and S. Ko, “Robust digital image stabilization using the Kalman filter,” IEEE Trans. Consum. Electron., vol. 55, no. 1, pp. 6–14, 2009.
X. Wang, X. He, Q. Teng, and M. Gao, “Digital image stabilization based on harmony filter,” J. Softw., vol. 9, no. 4, pp. 913–920, 2014.
W. G. Aguilar and C. Angulo, “Real-Time model-based video stabilization for microaerial vehicles,” Neural Process. Lett., vol. 43, no. 2, pp. 459–477, 2016.
T. Yabuki and Y. Yamaguchi, “Real-time video stabilization on an FPGA,” in Proc. of 2013 IEEE International Conference on Smart Structures and Systems, 2013, pp. 114–119.
S. Kumar, H. Azartash, M. Biswas, and T. Nguyen, “Real-time affine global motion estimation using phase correlation and its application for digital image stabilization.,” IEEE Trans. image Process., vol. 20, no. 12, pp. 3406–18, 2011.
G. Liu, T. Wang, L. Yu, Y. Li, and J. Gao, “The improved research on K-means clustering algorithm in initial values,” in Proc. of Int. Conf. Mechatron. Sci. Electr. Eng. Comput., 2013, pp. 2124–2127, 2013.
R. Tibshirani, G. Walther, and T. Hastie, “Estimating the number of clusters in a data set via the gap statistic,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 63, no. 2, pp. 411–423, 2001.
G. Farnebäck, “Two-frame motion estimation based on polynomial expansion,” in Proc. of Scandinavian Conference on Image Analysis, 2003, pp. 363–370.
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This journal is based on the work at https://journal.umy.ac.id/index.php/jrc under license from Creative Commons Attribution-ShareAlike 4.0 International License. You are free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even comercially.
The licensor cannot revoke these freedoms as long as you follow the license terms, which include the following:
- Attribution. You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- ShareAlike. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
- No additional restrictions. You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
• Creative Commons Attribution-ShareAlike (CC BY-SA)
JRC is licensed under an International License
