Multi Cost Function Fuzzy Stereo Matching Algorithm for Object Detection and Robot Motion Control
Abstract
Keywords
Full Text:
PDFReferences
Y. S. Heo, K. M. Lee, and S. U. Lee, “Robust Stereo Matching Using Adaptive Normalized Cross-Correlation,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 807-822, April 2011, doi: 10.1109/TPAMI.2010.136.
D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” International journal of computer vision, vol. 47, pp. 7-42, 2002.
R. Szeliski, Computer vision: algorithms and applications. Springer Nature, 2022.
A. Hosni, M. Bleyer, M. Gelautz, and C. Rhemann, “Local stereo matching using geodesic support weights,” 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2093-2096, 2009, doi: 10.1109/ICIP.2009.5414478.
K-. J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650-656, April 2006, doi: 10.1109/TPAMI.2006.70.
M. Poggi et al., “On the Confidence of Stereo Matching in a Deep-Learning Era: A Quantitative Evaluation,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5293-5313, 1 Sept. 2022, doi: 10.1109/TPAMI.2021.3069706.
Q. Yang, “A non-local cost aggregation method for stereo matching,” 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1402-1409, 2012, doi: 10.1109/CVPR.2012.6247827.
G. Egnal, “Mutual Information as a Stereo Correspondence Measure,” Univ. Pennsylvania, USA, Jan. 2000.
H. Hirschmuller and D. Scharstein, “Evaluation of Stereo Matching Costs on Images with Radiometric Differences,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1582-1599, Sept. 2009, doi: 10.1109/TPAMI.2008.221.
G. Yang, H. Zhao, J. Shi, Z. Deng, and J. Jia, “SegStereo: Exploiting Semantic Information for Disparity Estimation,” in Proc. European Conf. Comput. Vis. (ECCV), pp. 636-651, 2018.
X. Cheng, P. Wang, and R. Yang, “Learning Depth with Convolutional Spatial Propagation Network,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 10, pp. 2361-2379, 1 Oct. 2020, doi: 10.1109/TPAMI.2019.2947374.
X. Song, X. Zhao, L. Fang, and H. Hu, “EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detection,” Int. J. Comput. Vis., vol. 128, no. 4, pp. 910-930, Apr. 2020.
A. A. Shetty, R. Shetty, N. T. Hegde, A. C. Vaz, and C. R. Srinivasan, “A Study on the Effect of Radiometric Variations on A Fuzzy Stereo Matching Algorithm: A Statistical Analysis,” Engineered Science, vol. 16, pp. 269-280, Dec. 2021.
A. A. Shetty, V. I. George, C. G. Nayak, and R. Shetty, “Performance analysis of a fuzzy disparity selector for stereo matching of image segments under radiometric variations,” Turk. J. Elec. Eng. & Comp. Sci., vol. 28, no. 4, pp. 1965 – 1983, Mar. 2020.
S. Thrun, “Probabilistic robotics,” Communications of the ACM, vol. 45, no. 3, pp. 52-57, Mar. 2002.
J. Zbontar and Y. LeCun, “Stereo matching by training a convolutional neural network to compare image patches,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 2287-2318, Jan. 2016.
S. Lee, J. H. Lee, J. Lim, and I. H. Suh, “Robust stereo matching using adaptive random walk with restart algorithm,” Image and Vision Computing, vol. 37, pp. 1-11, May 2015.
X. Li, J. Liu, G. Chen, and H. Fu, “Efficient methods using slanted support windows for slanted surfaces,” IET Computer Vision, vol. 10, no. 5, pp. 384-391, Aug. 2016.
O. Zeglazi, M. Rziza, A. Amine, and C. Demonceaux, “Accurate dense stereo matching for road scenes,” in IEEE International Conference on Image Processing (ICIP), pp. 720-724, 2017.
J. Kostkova and R. Sara, “Stratified Dense Matching for Stereopsis in Complex Scenes,” in British Machine Vision Conference, pp. 1-10, 2003.
S. Hadfield, K. Lebeda, and R. Bowden, “Stereo reconstruction using top-down cues,” Computer Vision and Image Understanding, vol. 157, pp. 206-222, Apr. 2017.
J. Čech, J. Sanchez-Riera, and R. Horaud, “Scene flow estimation by growing correspondence seeds,” CVPR 2011, pp. 3129-3136, 2011, doi: 10.1109/CVPR.2011.5995442.
A. Murarka and N. Einecke, “A Meta-Technique for Increasing Density of Local Stereo Methods through Iterative Interpolation and Warping,” 2014 Canadian Conference on Computer and Robot Vision, pp. 254-261, 2014, doi: 10.1109/CRV.2014.59.
J. Cech and R. Sara, “Efficient Sampling of Disparity Space for Fast And Accurate Matching,” 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007, doi: 10.1109/CVPR.2007.383355.
K. Zhang, J. Li, Y. Li, W. Hu, L. Sun, and S. Yang, “Binary stereo matching,” Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 356-359, 2012.
D. Dekkiche, B. Vincke, and A. M´erigot, “Vehicles Detection in Stereo Vision Based on Disparity Map Segmentation and Objects Classification,” in International Symposium on Visual Computing, pp. 762-773, 2015.
P. Bergmiller, M. Botsch, J. Speth, and U. Hofmann, “Vehicle rear detection in images with Generalized Radial-Basis-Function classifiers,” 2008 IEEE Intelligent Vehicles Symposium, pp. 226-233, 2008, doi: 10.1109/IVS.2008.4621273.
D. Alonso, L. Salgado, and M. Nieto, “Robust Vehicle Detection Through Multidimensional Classification for on Board Video Based Systems,” 2007 IEEE International Conference on Image Processing, pp. 321-324, 2007, doi: 10.1109/ICIP.2007.4380019.
H. Ding, L. Tian, Y. Liu, M. li, and B. Guan, “Stereovision Based Generic Obstacle Detection and Motion Estimation Using V-stxiel Algorithm,” 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 903-908, 2018, doi: 10.1109/ITOEC.2018.8740535.
K. Sabe, M. Fukuchi, J. S. Gutmann, T. Ohashi, K. Kawamoto, and T. Yoshigahara, “Obstacle avoidance and path planning for humanoid robots using stereo vision,” in IEEE Int. Conf. Robotics and Automation, pp. 592-597, 2004.
B. Xi, R. Guo, F. Sun, and Y. Huang, “Simulation research for active simultaneous localization and mapping based on extended kalman filter,” in IEEE Int. Conf. Automation and Logistics, pp. 2443-2448, 2008.
G. P. Huang, A. I. Mourikis, and S. I. Roumeliotis, “Analysis and improvement of the consistency of extended Kalman filter based SLAM,” 2008 IEEE International Conference on Robotics and Automation, pp. 473-479, 2008, doi: 10.1109/ROBOT.2008.4543252.
K. Kohara, N. Suganuma, T. Negishi, and T. Nanri, “Obstacle detection based on occupancy grid maps using stereovision system,” Int. J. Intelligent Transportation Systems Research, vol. 8, no. 2, pp. 85-95, May 2010.
A. Garulli, A. Giannitrapani, A. Rossi, and A. Vicino, “Mobile robot SLAM for line-based environment representation,” in Proceedings of the 44th IEEE Conference on Decision and Control, pp. 2041-2046, 2005.
J. Jiao, R. Wang, W. Wang, S. Dong, Z. Wang, and W. Gao, “Local Stereo Matching with Improved Matching Cost and Disparity Refinement,” in IEEE MultiMedia, vol. 21, no. 4, pp. 16-27, Oct.-Dec. 2014, doi: 10.1109/MMUL.2014.51.
M. Tahmasebi, M. Gohari, and A. Emami, “An autonomous pesticide sprayer robot with a color-based vision system,” Int. J. Robotics and Control Sys., vol. 2, no. 1, pp. 115-123, 2022.
J. Laarni, H. Koskinen, and A. Väätänen, “Concept of operations as a boundary object for knowledge sharing in the design of robotic swarms,” Int. J. Robotics and Control Sys., vol. 2, no. 4, pp. 692-708, 2022.
M. G. Mohanan, and A. Salgaonkar, “Robotic Motion Planning in Dynamic Environments and its Applications,” Int. J. Robotics and Control Sys., vol. 2, no. 4, pp. 666-691, 2022.
I. Hassani, I. Maalej, and C. Rekik, “Robot path planning with avoiding obstacles using free segments and turning points algorithms,” Mathematical problem in Engineering, vol. 2018, no. 6, pp. 1-13, 2018.
S. Sahloul, D. Ben Halima and C. Rekik, “An hybridization of global-local methods for autonomous mobile robot navigation in partially-known environments,” J. Robotics and Control, vol.2, no. 4, pp. 351-355, 2021.
L. Cai, L. He, Y. Xu, Y. Zhao, and X. Yang, “Multi-object detection and tracking by stereo vision,” Patt. Recogn., vol. 43, no.12, pp. 4028-4041, Dec. 2010.
H. Liu, Y. Cai, S. Zhou, and J. Yang, “Stereo Matching with Multi‐scale Based on Anisotropic Match Cost,” Concurrency and Computation: Practice and Experience, vol. 32, no. 1, Aug. 2020.
M. V. Bobyr, N. A. Milostnaya, S. V. Gorbachev, S. Bhattacharyya, and J. Cao, “The Method of Depth Map Calculating Based on Soft Operators in Multi-Agent Robotic Stereo Vision Systems,” Research Transcripts in Computer, Electrical and Electronics Engineering, vol. 2, pp. 83-98, 2021.
T. Mahalingam and M. Subramoniam, “A robust single and multiple moving object detection, tracking and classification,” Applied Computing and Informatics, vol. 17, no. 1, pp. 2-18, Jan. 2021.
W. Sun, “Robot Obstacle Recognition and Target Tracking Based on Binocular Vision,” Advances in Multimedia, vol. 2022, pp. 1-12, Aug. 2022.
M. T. Shahria, M. S. H. Sunny, M. I. I. Zarif, J. Ghommam, S. I. Ahamed, and M. H. Rahman, “A Comprehensive Review of Vision-Based Robotic Applications: Current State, Components, Approaches, Barriers, and Potential Solutions,” Robotics, vol. 11, no.6, pp. 1-20, Dec. 2022.
Q. Zhu, Y. Han, P. Liu, Y. Xiao, P. Lu, and C. Cai, “Motion planning of autonomous mobile robot using recurrent fuzzy neural network trained by extended Kalman filter,” Computational intelligence and neuroscience, vol. 2019, pp. 1-17, Jan. 2019.
K. S. Sharma and P. V. Manivannan, “Stereo Image Partitioning Based Fuzzy Logic,” Int. J. Mechanical Engineering and Robotics Research, vol. 9, no. 8, Aug. 2020.
H. Shabanian and M. Balasubramanian, “A novel factor graph-based optimization technique for stereo correspondence estimation,” Scientific Reports, vol. 12, no. 1, p. 15613, Sept. 2022.
S. Hożyń and B. Żak, “Stereo Vision System for Vision-Based Control of Inspection-Class ROVs,” Remote Sensing, vol. 13, no. 24, pp. 1-25, Dec. 2021.
Y. Wang et al., “Vision based obstacle detection using rover stereo images,” in International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, pp. 1471-1477, 2019.
S. Alqahtani, S. Taylor, I. Riley, R. Gamble, and R. Mailler, “Predictive path planning algorithm using kalman filters and mtl robustness,” in 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics, pp. 1-7, 2018,
T. Yan, Y. Gan, Z. Xia, and Q. Zhao, “Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching,” in IEEE Transactions on Image Processing, vol. 28, no. 8, pp. 3885-3897, Aug. 2019, doi: 10.1109/TIP.2019.2903318.
F. Tombari, S. Mattoccia, L. Di Stefano, and E. Addimanda, “Near real-time stereo based on effective cost aggregation,” 2008 19th International Conference on Pattern Recognition, pp. 1-4, 2008, doi: 10.1109/ICPR.2008.4761024.
Y. Xu, Y. Zhao, and M. Ji, “Local stereo matching with adaptive shape support window-based cost aggregation,” Appl. Opt., vol. 53, no. 29, pp. 6885-6892, Oct. 2014.
J. Zhang et al., “RGB-D saliency detection via cascaded mutual information minimization,” In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4338-4347, 2021.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “SLIC Superpixels Compared to State-of-the-Art Superpixel Methods,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274-2282, Nov. 2012, doi: 10.1109/TPAMI.2012.120.
T. J. Ross, Fuzzy logic with engineering applications. John Wiley & Sons, 2009.
A. A. Shetty, V. I. George, C. G. Nayak, and R. Shetty, “Fuzzy logic-based disparity selection using multiple data costs for stereo correspondence,” Turk. J. Elec. Eng. & Comp. Sci., vol. 27, no. 1, pp. 377-391, Jan. 2019.
F. Mroz and T. P. Breckon, “An empirical comparison of real-time dense stereo approaches for use in the automotive environment,” J. Image Video Proc., vol. 2012, no. 13, pp. 1-19, Aug. 2012.
N. Dhanachandra, K. Manglem, and Y. J. Chanu, “Image segmentation using K-means clustering algorithm and subtractive clustering algorithm,” Procedia Comp. Sci., vol. 54, pp. 764-771, Jan. 2015.
Y. Marom and D. Feldman, “k-Means clustering of lines for big data,” Advances in Neural Information Processing Systems, vol. 32, 2019.
S. Zhang, H. Wang, W. Huang, and Z. You, “Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG,” Optik, vol. 157, pp. 866-872, Mar. 2018.
S. Duggal, S. Wang, W. C. Ma, R. Hu, and R. Urtasun, “Deeppruner: Learning efficient stereo matching via differentiable patchmatch,” In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4384-4393, 2019.
S. Ma, X. Bai, Y. Wang, and R. Fang, “Robust stereo visual-inertial odometry using nonlinear optimization,” Sensors, vol. 19, no. 17, p. 3747, 2019.
H. Luo, C. Pape, and E. Reithmeier, “Robust RGBD Visual Odometry Using Windowed Direct Bundle Adjustment and Slanted Support Plane,” in IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 350-357, Jan. 2022, doi: 10.1109/LRA.2021.3126347.
Z. Rao, M. He, Y. Dai, Z. Zhu, B. Li, and R. He, “MSDC-Net: Multi-Scale Dense and Contextual Networks for Stereo Matching,” 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 578-583, 2019, doi: 10.1109/APSIPAASC47483.2019.9023237.
Y. P. Chandra and T. Matuska, “Stratification analysis of domestic hot water storage tanks: A comprehensive review,” Energy and Buildings, vol. 187, pp. 110-131, 2019.
Z. Su, L. Xu, Z. Zheng, T. Yu, Y. Liu, and L. Fang, “Robustfusion: Human volumetric capture with data-driven visual cues using a rgbd camera,” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pp. 246-264, 2020.
R. Battrawy, R. Schuster, O. Wasenmüller, Q. Rao, and D. Stricker, “LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Images,” 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7762-7769, 2019, doi: 10.1109/IROS40897.2019.8967739.
S. Lee, Y. Kim, and E. Eisemann, “Iterative depth warping,” ACM Transactions on Graphics (TOG), vol. 37, no. 5, pp. 1-13, 2018.
Q. Wang, S. Shi, S. Zheng, K. Zhao, and X. Chu, “FADNet: A Fast and Accurate Network for Disparity Estimation,” 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 101-107, 2020, doi: 10.1109/ICRA40945.2020.9197031.
H. I. F. Ibrahim, H. Khaled, N. A. Seada, and H. M. Faheem, “Parallel Dense Binary Stereo Matching Using CUDA,” 2020 15th International Conference on Computer Engineering and Systems (ICCES), pp. 1-6, 2020, doi: 10.1109/ICCES51560.2020.9334591.
Yaman and S. Kalkan, “Performance evaluation of similarity measures for dense multimodal stereo vision,” J. Elect. Imag., vol. 25, no. 3, pp. 1-31, Jun. 2016.
M. Baydoun and M. A. Al-Alaoui, “Enhancing stereo matching with varying illumination through histogram information and normalized cross correlation,” in IEEE International Conference on Systems, Signals and Image Processing, pp. 5-9, 2013.
Y. C. Tsai, K. H. Chen, Y. Chen, and J. H. Cheng, “Accurate and fast obstacle detection method for automotive applications based on stereo vision,” in 2018 International Symposium on VLSI Design, Automation and Test, pp. 1-4, 2018.
S. D. Perkasa, P. Megantoro, and H. A. Winarno, “Implementation of a camera sensor pixy 2 CMUcam5 to a two wheeled robot to follow colored object,” Journal of Robotics and Control, vol. 2, no. 6, pp. 469-501, 2021.
R. Fan and M. Liu, “Road Damage Detection Based on Unsupervised Disparity Map Segmentation,” in IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 11, pp. 4906-4911, Nov. 2020, doi: 10.1109/TITS.2019.2947206.
H. Tao and X. Lu, “Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi‐feature fusion,” IET Intelligent Transport Systems, vol. 13, no. 2, pp. 252-259, 2019.
N. Y. Ershadi, J. M. Menéndez, and D. Jiménez, “Robust vehicle detection in different weather conditions: Using MIPM,” PloS one, vol. 13, no. 3, p. e0191355, 2018.
J. Lian et al., “Study on Obstacle Detection and Recognition Method Based on Stereo Vision and Convolutional Neural Network,” 2019 Chinese Control Conference (CCC), pp. 8766-8771, 2019, doi: 10.23919/ChiCC.2019.8866348.
O. Amimi, A. Mansouri, S. D. Bennani, and Y. Ruichek, “Stereo vision based advanced driver assistance system,” in 2017 International Conference on wireless Technologies, Embedded and Intelligent Systems, pp. 1-5, 2017.
T. O. Kvålseth, “On Normalized Mutual Information: Measure Derivations and Properties,” Entropy, vol. 19, no. 11, p. 631, 2017.
D. H. Lee, J. Y. Chang, and Y. S. Heo, “Stereo matching using cost volume fusion for high dynamic range scenes,” Electronics Letters, vol. 53, no. 23, pp. 1522-1524, 2017.
M. Lopez-Franco, E. N. Sanchez, A. Y. Alanis, and C. López-Franco, “Neural control for driving a mobile robot integrating stereo vision feedback,” Neural Processing Letters, vol. 43, no. 2, pp. 425-444, 2016.
L. Pérez, Í. Rodríguez, N. Rodríguez, R. Usamentiaga, and D. García, “Robot guidance using machine vision techniques in industrial environments: A comparative review,” Sensors, vol. 16, no. 3, pp. 335, 2016.
R. A. Hamzah, H. Ibrahim, and A. H. A. Hassan, “Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation,” Journal of Visual Communication and Image Representation, vol. 42, pp. 145-160, 2017.
L. Li, S. Zhang, X. Yu, and L. Zhang, “PMSC: Patch Match-based superpixel cut for accurate stereo matching,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 3, pp. 679-692, 2016.
M. Abdullah-Al-Noman, A. N. Eva, T. B. Yeahyea, and R. Khan, “Computer Vision-based Robotic Arm for Object Color, Shape, and Size Detection,” Journal of Robotics and Control, vol. 3, no. 2, pp. 180-186, 2022.
E. Arnold, O. Y. Al-Jarrah, M. Dianati, S. Fallah, D. Oxtoby, and A. Mouzakitis, “A survey on 3D object detection methods for autonomous driving applications,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 10, pp. 3782-3795, 2019.
G. Qi, H. Wang, M. Haner, C. Weng, S. Chen, and Z. Zhu, “Convolutional neural network-based detection and judgement of environmental obstacle in vehicle operation,” CAAI Transactions on Intelligence Technology, vol. 4, no. 2, pp. 80-91, 2019.
Q. Dong and J. Feng, “Outlier detection and disparity refinement in stereo matching,” Journal of Visual Communication and Image Representation, vol. 60, pp. 380-390, 2019.
N. Choi, J. Jang, and J. Paik, “Illuminant-invariant stereo matching using cost volume and confidence-based disparity refinement,” Journal of the Optical Society of America A, vol. 36, no. 10, pp. 1768-1776, 2019.
D. Falanga, K. Kleber, and D. Scaramuzza, “Dynamic obstacle avoidance for quadrotors with event cameras,” Science Robotics, vol. 5, no. 40, pp. 1-51, 2020.
V. Kramar, O. Kramar, and A. Kabanov, “Self-Collision Avoidance Control of Dual-Arm Multi-Link Robot Using Neural Network Approach,” Journal of Robotics and Control, vol. 3, no. 3, pp. 309-319, 2022.
DOI: https://doi.org/10.18196/jrc.v4i3.17041
Refbacks
- There are currently no refbacks.
Copyright (c) 2023 Akhil Appu Shetty, Navya Thirumaleshwar Hegde, Aldrin Claytus Vaz, C R Srinivasan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Journal of Robotics and Control (JRC)
P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com