Visual Servo Kinematic Control of Delta Robot using YOLOv5 Algorithm

Kawin Yamtuan, Trirat Radomngam, Pradya Prempraneerach

Abstract


Industrial delta robots with motion-and vision-base controllers, providing high-precision and fast/flexible motion, normally come with high cost and complex software. Low-cost delta robots with vision solution could be beneficial to vast industries to increase productivity and to reduce labor cost. A delta robot using a low-cost motion controller and an open-source vision system is developed to accomplish real-time visual servoing with high motion accuracy. In the low-cost motion controller, three parallel links’ upper-arm angles, computed from inverse kinematics for a given desired target position by a high-level computer, are used as reference position commands for three AC-motor drives. A low-level Arduino microcontroller is employed to convert these links’ angles to high-frequency pulses and on-off signals for synchronously controlling three motor angles and direction. Experimental results of a point-to-point motion tracking exhibit high-precision repeatability. Synchronous pulse generation from Arduino microcontroller and structural misalignments of parallel links are major challenges for achieving high motion accuracy.  For the vision-based system, the YOLOv5 algorithm is implemented along with a Python GUI Application. Then, the visual-servo performance is evaluated on localization accuracy and recognition rate of 3-color objects. However, a partial object occlusion can reduce the visual classification rate. A sorting task of 4-category medicine boxes demonstrate a high-speed pick-and-place operation using the low-cost visual-servo system of this delta robot. Therefore, integration of low-cost visual servoing with this delta robot can revolutionize various industries, like automobile, pharmaceutical, and food sectors, in separating, sorting and packing applications.

Keywords


Delta Robot; Inverse Kinematics; Pick-and-Place; Microcontroller; Visual Servoing; YOLOv5 Algorithm.

Full Text:

PDF

References


A. Filipescu, D. Ionescu, A. Filipescu, E. Mincă, and G. Simion, “Multifunctional Technology of Flexible Manufacturing on a Mechatronics Line with IRM and CAS, Ready for Industry 4.0,” Processes, vol. 9, no. 5, p. 864, 2021, doi: 10.3390/pr9050864.

A. S. Phung, J. Malzahn, F. Hoffmann, and T. Bertram, “Data based kinematic model of a multi-flexible-link robot arm for varying payloads,” 2011 IEEE International Conference on Robotics and Biomimetics, pp. 1255-1260, 2012, doi: 10.1109/ROBIO.2011.6181460.

M. Abdelaal, “A Study of Robot Control Programing for an Industrial Robotic Arm,” 2019 6th International Conference on Advanced Control Circuits and Systems (ACCS) & 2019 5th International Conference on New Paradigms in Electronics & information Technology (PEIT), pp. 23-28, 2019, doi: 10.1109/ACCS-PEIT48329.2019.9062878.

S. Ahangar, M. V. Mehrabani, A. P. Shorijeh, and M. T. Masouleh, “Design a 3-DOF Delta Parallel Robot by One Degree Redundancy along the Conveyor Axis, A Novel Automation Approach,” 2019 5th Conference on Knowledge-Based Engineering and Innovation, pp. 413-418, 2019, doi:10.1109/ KBEI.2019.8734975.

H. Rakhodaei, “Design and Analysi of 9 DOF Hybrid Parallel Robot,” Doctor of Philosophy Thesis, School of Mechanical Engineering, University of Birmingham, Birmingham, United Kingdom, 2013.

M. Michelin, P. -E. Hervė, O. Tempier, J. -B. Izard, and M. Gouttefarde, “Path Following Demonstration of a Hybrid Cable-Driven Parallel Robot,” CableCon 2021 – 5th International Conference on Cable-Driven Parallel Robots, pp. 323-335, 2021, doi: 10.1007/978-3-030-75789_26.

H. Park and M. Hwang, “Design of a Multi-bending Flexible Manipulator for Gastrointestinal Surgery,” 2022 22nd International Conference on Control, Automation and Systems (ICCAS), pp. 1733-1735, 2022, doi: 10.23919/ ICCAS55662.2022.10003802.

C. Choosui and P. Prempraneerach, “Prototype of Delta Robot for Industrial Application,” RSU National Research Conference 2016, pp. 237-246, 2016.

S. Patil, S. C. Alvares, P. Mannam, O. Kroemer, and F. Z. Temel, “DeltaZ: An Accessible Compliant Delta Robot Manipulator for Research and Education,” 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 13213-13219, 2022, doi: 10.1109/IROS47612.2022.9981257.

M. Ratiu, A. Rus, D. Melentie Anton, and R. Vladimir Ghincu, “Command and Control System for a Parallel Robot,” 2021 16th International Conference on Engineering of Modern Electric Systems (EMES), pp. 1-4, 2021, doi: 10.1109/ EMES52337.2021.9484151.

B. Mehrafrooz, M. Mohammadi, and M. T. Masouleh, “Kinematic Sensitivity Evaluation of Revolute and Prismatic 3-DOF Delta Robots,” 2017 5th RSI International Conference on Robotics and Mechatronics, pp. 225-231, 2017, doi: 10.1109/ICRoM.2017.8466159.

Susanto, M. Rohim, and R. Analia, “The Implementation of a Modest Kinematic Solving for Delta Robot,” 2019 2nd International Conference on Applied Engineering (ICAE), pp. 1-5, 2019, doi: 10.1109/ICAE47758.2019.9221720.

F. A. Azad, S. A. Rad, and M. Arashpour, “Back-stepping control of delta parallel robots with smart dynamic model selection for construction applications,” Automation in Construction, vol. 137, p. 104211, 2022, doi: 10.1016/j.autcon.2022.104211.

D. Malyshev, L. Rybak, G. Carbone, T. Semenenko, and A. Nozdracheva, “Optimal Design of a Parallel Manipulator for Aliquoting of Biomaterials Considering Workspace and Singularity Zones,” Appl. Sci., vol. 12, no. 4, p. 2070, 2022, doi: 10.3390/ app12042070.

Y. D. Patel and P. M. George, “Parallel Manipulators Applications – A Survey,” Modern Mechanical Engineering, vol. 2, no. 3, pp. 57-64, 2012, doi: 10.4236/mme.2012.23008.

P. Prempraneerach, “Workspace and Dynamic Trajectory Tracking of Delta Parallel Robot,” 2014 International Computer science and Engineering Conference, pp. 1-6, 2014, doi: 10.1109/ ICSEC.2014.6978242.

R. D. J. Portillo-Vélez, I. A. Burgos-Castro, J. A. Vásquez-Santacruz, and L. F. Marín-Urías, “Integrated conceptual mechatronic design of a delta robot,” Machines, vol. 10, no. 3, p. 186, 2022, doi: 10.3390/ machines10030186.

Z. lklima, M.I. Muthahhar, A. Khan, and A. Zody, “Self-learning of delta robot using inverse kinematics and artificial neural network,” Sinergi, vol. 25, no. 3, pp. 237-244, 2021, doi: 10.2244/sinergi.2021.3.001.

Y.-L. Kuo, “Mathematical modeling and analysis of the Delta robot with flexible links,” Computers and Mathematics with Applications, vol. 17, no. 10, pp. 1973-1989, 2016, doi: 10.1016/ j.camwa.2016.03.018.

M. Wu, J. Mei, Y. Zhao, and W. Niu, “Vibration reduction of delta robot based on trajectory planning,” Mechanism and Machine Theory, vol. 153, p. 104004, 2020, doi: 10.1016/ j.mechmachtheory.2020.104004.

J. Viola and L. Angel, “Delta Parallel Robotic Manipulator Tracking Control using Fractional Order Controllers,” IEEE Latin America Transactions, vol. 17, no. 3, pp. 393-400, 2019, doi: 10.1109/ TLA.2019.8863309.

P.-C. Pham and Y.-L. Kuo, “Robust Adaptive Finite-Time Synergetic Tracking Control of Delta Robot Based on Radial Basis Function Neural Networks,” Appl. Sci., vol. 12, no. 21, p. 10861, 2022, doi: 10.3390/app122110861.

C. E. Boudjedir, M. Bouri, and D. Boukhetala, “Model-Free Iterative Learning Control With Nonrepetitive Trajectories for Second-Order MIMO Nonlinear Systems- Application to a Delta Robot,” IEEE Transactions on Industrial Electronics, vol. 68, no. 8, pp. 7433-7443, 2021, doi: 10.1109/TIE.2020.3007091.

J. J. Fernandes, “Kinematic and dynamic analysis of 3PUU parallel manipulator for medical applications,” Procedia computer science, vol. 133, pp. 604-611, 2018, doi: 10.1016/ j.procs.2018.07.091.

M. Vulliez, S. Zeghloul, and O. Khatib, “Design strategy and issues of the Delthaptic, a new 6-DOF parallel haptic device,” Mechanism and Machine Theory, vol. 128, pp. 395-411, 2018, doi: 10.1016/ j.mechmachtheory.2018.06.015.

E. M. Young and K. J. Kuchenbecker, "Implementation of a 6-DOF Parallel Continuum Manipulator for Delivering Fingertip Tactile Cues," IEEE Transactions on Haptics, vol. 12, no. 3, pp. 295-306, 2019, doi: 10.1109/TOH.2019.2920928.

S. R. Williams, J. M. Suchoski, Z. Chua, and A. M. Okamura, "A 4-Degree-of-Freedom Parallel Origami Haptic Device for Normal, Shear, and Torsion Feedback," IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3310-3317, 2022, doi: 10.1109/ LRA.2022.3144798.

H. Dong, Y. Feng, C. Qiu, and I. -M. Chen, "Construction of Interaction Parallel Manipulator: Towards Rehabilitation Massage," IEEE/ASME Transactions on Mechatronics, vol. 28, no. 1, pp. 372-384, 2023, doi: 10.1109/TMECH.2022.3202694.

C. Li, X. Gu, X. Xiao, C. M. Lim, and H. Ren, "A Robotic System With Multichannel Flexible Parallel Manipulators for Single Port Access Surgery," IEEE Transactions on Industrial Informatics, vol. 15, no. 3, pp. 1678-1687, 2019, doi: 10.1109/ TII.2018.2856108.

M. E. Corominas, “Workspace and singularity determination of a 7-DoF wrist-partitioned serial manipulator,” Bachelor’s degree Thesis, Department of Mathematics and Industrial Technology Engineering, Georgia Institute of Technology, 2020.

H. Wang, Z. Zhou, X. Zhong, and Q. Chen, “Singular Configuration Analysis and Singularity Avoidance with Application in an Intelligent Robotic Manipulator,” Sensors, vol. 22, no. 3, p. 1239, 2022, doi: 10.3390/ s22031239.

M. Alexovič, P. L. Urban, H. Tabani, and J. Sabo, “Recent advances in robotic protein sample preparation for clinical analysis and other biomedical applications,” Clinica Chimica Acta, vol. 507, pp. 104-116, 2020.

Z. He, C. Wu, S. Zhang, and X. Zhao, "Moment-Based 2.5-D Visual Servoing for Textureless Planar Part Grasping," IEEE Transactions on Industrial Electronics, vol. 66, no. 10, pp. 7821-7830, Oct. 2019, doi: 10.1109/TIE.2018.2886783.

W. Zhang, J. Mei, and Y. Ding, “Design and Development of a High Speed Sorting System Based on Machine Vision Guiding,” Physics Procedia, vol. 25, pp. 1955-1965, 2012, doi: 10.1016/ j.phpro.2012.03.335.

H. Wu, “Research on Control System of Delta Robot Based on Visual Tracking Technology,” Journal of Physics: Conference Series, vol. 1883, p. 012110, 2021, doi: 10.1088/1742-6596/1883/1/012110.

Y. Zhang, Z. Wang, M. Li, and P. Gao, “RP-YOLOX-DL: A Deep Learning Hybrid Method for Parallel Robots Target Positioning,” Measurement Science and Technology, vol. 34, no. 10, 2023, doi: 10.1088/1361-6501/ace20a.

Z. Chen, S. Li, N. Zhang, Y. Hao, and X. Zhang, "Eye-to-Hand Robotic Visual Tracking Based on Template Matching on FPGAs," IEEE Access, vol. 7, pp. 88870-88880, 2019, doi: 10.1109/ ACCESS.2019.2926807.

M. Zhu, C. Huang, Z. Qiu, W. Zheng, and D. Gong, “Parallel Image-Based Visual Servoing/Force Control of a Collaborative Delta Robot,” Front. Neurorobot., vol. 16, pp. 1-14, 2022, doi: 10.3389/fn-bot.2022.922704.

M. Zhu, S. Briot, and A. Chriette, “Sensor-based design of a Delta parallel robot,” Mechatronics, vol. 87, p. 102893, 2022, doi: 10.1016/ j.mechatronics.2022.102893.

A. Sharida and I. Hashlamon, “Real Time Vision Based Controller for Delta Robots,” International Journal of Intelligent Systems Technologies and Applications, vol. 20, no. 4, pp. 271-295, 2021, doi: 10.1504/ijista.2021.121321.

S. Kansal and S. Mukherjee, “Vision-based kinematic analysis of the Delta robot for object catching,” Robotica, vol. 40, pp. 2010-2030, 2022, doi: 10.1017/S0263574721001491.

S. Geethapriya, N. Duraimurugan, and S. P. Chokkalingam, “Real-Time Object Detection with Yolo,” International Journal of Engineering and Advanced Technology (IJEAT), vol. 8, no. 3S, pp. 578-581, 2019.

Y. H. Lee and Y. Kim, “Comparison of CNN and YOLO for object detection,” Journal of the Semiconductor & Display Technology, vol. 19, no. 1, pp. 85-92, 2020.

P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, “A Review of Yolo Algorithm Developments,” Procedia Computer Science, vol. 199, pp. 1066-1073, 2022, doi: 10.1016/j.procs.2022.01.135.

T. Diwan, G. Anirudh, and J. V. Tembhurne, “Object detection using YOLO: challenges, architectural, successors, datasets and applications,” Multimed Tools and Appl., vol. 82, no. 6, pp. 9243-9275, 2023, doi: 10.1007/s11042-022-13644-y.

A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “YOLOv4: optimal speed and accuracy of object detection,” ArXiv, abs/2004.10934, 2020, doi: 10.48550/arXiv.2004.10934.

J. S. Murthy, G. M. Siddesh, W.-C. Lai, B. D. Parameshachari, S. N. Patil, and K. L. Hemalatha, “ObjectDetect: A Real-Time Object Detection Framework for Advanced Driver Assistant Systems Using YOLOv5,” Wireless Communications and Mobile Computing, vol. 2022, 2022, doi: 10.1155/2022/ 9444360.

D. Shao, Z. He, H. Fan, and K. Sun, “Detection of Cattle Key Parts Based on the Improved Yolov5 algorithm,” Agriculture, vol. 13, no. 6, p. 1110, 2023, doi: 10.3390/argiculture13061110.

W. Yang, J. Wu, J. Zhang, K. Gao, R. Du, Z. Wu, E. Firkat, and D. Li, “Deformable convolution and coordinate attention for fast cattle detection,” Computers and Electronics in Agriculture, vol. 211, p. 108006, 2023, doi: 10.1016/j.compag.2023.108006.

H. Zhong, T. Han, W. Xia, Y. Tian, and L. Wu, “Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms,” EURASIP J. Adv. Signal Process., vol. 2023, no.1, p. 55, 2023, doi: 10.1186/s13634-023-01019-w.

Q. Xu, Z. Zhu, H. Ge, Z. Zhang, and X. Zang, “Effective Face Detector Based on YOLOv5 and Superresolution Reconstruction,” Computational and Mathematical Methods in Medicine, vol. 2021, pp. 1-9, 2021, doi: 10.1155/2021/7748350.

M. A. Farooq, P. Corcoran, C. Rotariu, and W. Shariff, "Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)," IEEE Access, vol. 9, pp. 156465-156481, 2021, doi: 10.1109/ACCESS.2021.3129150.

A. Kosuge, S. Suehiro, M. Hamada, and T. Kuroda, "mmWave-YOLO: A mmWave Imaging Radar-Based Real-Time Multiclass Object Recognition System for ADAS Applications," IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-10, 2022, doi: 10.1109/TIM.2022.3176014.

I. Ahmed, S. Din, G. Jeon, F. Piccialli, and G. Fortino, "Towards Collaborative Robotics in Top View Surveillance: A Framework for Multiple Object Tracking by Detection Using Deep Learning," IEEE/CAA Journal of Automatica Sinica, vol. 8, no. 7, pp. 1253-1270, 2021, doi: 10.1109/JAS.2020.1003453.

Z. Wei, S. Dong, and X. Wang, “Petrochemical Equipment Detection by Improved Yolov5 with Multiscale Deep Feature Fusion and Attention Mechanism,” Journal of Electrical and Computer Engineering, vol. 2022, 2022, doi: 10.1155/2022/8612174.

Z. Wei, S. Dong, and X. Wang, “Petrochemical Equipment Tracking by Improved Yolov7 Network and Hybrid Matching in Moving Scenes,” Sensors, vol. 23, no. 9, p. 4546, 2023, doi: 10.3390/s23094546.

H. Sekkat, S. Tigani, R. Saadane, and A. Chehri, “Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Object Grasping,” Appl. Sci., vol. 11, no. 17, p. 7917, 2021, doi: 10.3390/app11177917.

X. Yue, H. Li, M. Shimizu, S. Kawamura, and L. Meng, “YOLO-GD: A Deep Learning-Based Object Detection Algorithm for Empty-Dish Recycling Robots,” Machines, vol. 10, no. 5, p. 294, 2022, doi: 10.3390/ machines10050294.

D. Kijdech and S. Vongbunyon, “Pick-and-place application using a dual arm collaborative robot and an RGB-D camera with YOLOv5,” IAES International Journal of Robotics and Automation (IJRA), vol. 12, no. 2, p. 197, 2023, doi: 10.11591/ijra.v12i2.

M. Gao, Q. Cai, B. Zheng, J. Shi, Z. Ni, J. Wang, and H. Lin, “A Hybrid YOLO v4 and Particle Filter Based Robotic Arm Grabbing System in Nonlinear and Non-Gaussian Environment,” Electronics, vol. 10, no. 10, p. 1140, 2021, doi: 10.3390/electronics10101140.

M. Mueller, B. Kuhlenkoetter, and H. Geiger, “Cycle Time Estimation for a Delta-Type Robot,” ISR/Robotik 2014; 41st International Symposium on Robotics, pp. 1-7, 2014.

G. -N. Zhu, Y. Zeng, Y. S. Teoh, E. Toh, C. Y. Wong, and I. -M. Chen, "A Bin-Picking Benchmark for Systematic Evaluation of Robotic-Assisted Food Handling for Line Production," IEEE/ASME Transactions on Mechatronics, vol. 28, no. 3, pp. 1778-1788, 2023, doi: 10.1109/TMECH.2022.3227038.

C. M. A. Vasques and F. A. V. Figueiredo, “The 3D-Printed Low-Cost Delta Robot Óscar: Technology Overview and Benchmarking,” Eng. Proc., vol. 11, no. 1, p. 43, 2021, doi: 10.3390/ASEC2021-11173.

L. H. Phuong, V. D. Cong, and H. T. Thanh, “Design a Low-cost Delta Robot Arm for Pick and Place Applications Based on Computer Vision,” FME Transactions, vol. 5. no. 1, pp. 99-108, 2023, doi: 10.5937/fme2301099P.

J. Daniel Martinez Reyes, G. G. Badillo, V. E. E. López, and G. G. Mora, “Objects color classification and repositioning operated by a delta robot,” 2016 XVIII Congreso Mexicano de Robotica, pp. 1-5, 2016, doi: 10.1109/COMROB.2016.7955157.

J. Lin, C.-H. Luo, and K.-H. Lin, “Design and Implementation of a New DELTA Parallel Robot in Robotics Competitions,” International Journal of Advanced Robotic Systems, vol. 12, no. 10, pp. 1-10, 2015, doi: 10.5772/61744.

K. Rahul, H. Raheman, and V. Paradkar, “Design of a 4 DOF parallel robot arm and the firmware implementation on embedded system to transplant pot seedlings,” Artificial Intelligence in Agriculture, vol. 4, pp. 172-183, 2020, doi: 10.1016/j.aiia.2020.09.003.

R. Hu, “Application of Microcontroller-Based Multipath Servos in Industrial Robot Control System,” Journal of Robotics, vol. 2023, 2023, doi: 10.1155/2023/7235120.

Yaskawa, “Sigma series SGM/SGMP/SGDA USER’S MANUAL”, 2023. [Online]. Available: https://www.yaskawa.com.

Copley Controls, April 2023, Accelnet ADP 180-30. [Online]. Available: https://copleycontrols.com/en/products/adp-180-30/

W. Ding, J. Gu, S. Tang, Z. Shang, E. A. Duodu, and C. Zheng, “Development of a calibrating algorithm for Delta Robot’s visual positioning based on artificial neural network,” Optik, vol. 127, no. 20, pp. 9095-9104, 2016, doi: 10.1016/ j.ijleo.2016.06.126.

R. Xu, H. Lin, K. Lu, L. Cao, and Y. Liu, “A Forest Fire Detection System Based on Ensemble Learning,” Forests, vol. 12, no. 2, p. 217, 2021, doi: 10.3390/f12020217.

I. Katsamenis, E. E. Karolou, A. Davradou, E. Protopapadakis, A. Doulamis, N. Doulamis, and D. Kalogeras, “TraCon: A novel dataset for real-time traffic cones detection using deep learning,” Novel & Intelligent Digital Systems: Proceedings of the 2nd International Conference (NiDS 2022), pp. 382-391, 2022, doi: 10.1007/978-3-031-17601-2_37.

K. -J. Kim, P. -K. Kim, Y. -S. Chung, and D. -H. Choi, "Multi-Scale Detector for Accurate Vehicle Detection in Traffic Surveillance Data," IEEE Access, vol. 7, pp. 78311-78319, 2019, doi: 10.1109/ ACCESS.2019.2922479.

S. -E. Ryu and K. -Y. Chung, “Detection Model of Occluded Object Based on YOLO Using Hard-Example Mining and Augmentation Policy Optimization,” Appl. Sci., vol. 11, no. 15, p. 7093, 2021, doi: 10.3390/app11157093.

K. He et al., "Visibility Maximization Controller for Robotic Manipulation," IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8479-8486, 2022, doi: 10.1109/ LRA.2022.3188430.

J. Millington, R. P. Monfared, and D. Vera, “Innovative mechanism to identify robot alignment in an automation system,” Robotics and Autonomous Systems, vol. 114, pp. 144-154, 2019, doi: 10.1016/j.robot.2019.01.015.

R. Zhao, L. Wu, and Y.-H. Chen, “Parallel Robot with Uncertainty: An Online Estimation Approach”, IEEE Access, vol. 8, pp. 97604-97617, 2020, doi: 10.1109/ACCESS.2020.2997093.

C. E. Boudjedir, D. Boukhetala, and M. Bouri, “Nonlinear PD plus sliding mode control with application to a parallel delta robot,” Journal of Electrical Engineering, vol. 69, no. 5, pp. 329-336, 2018, doi: 10.2478/jee-2018-0048.




DOI: https://doi.org/10.18196/jrc.v4i6.19102

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Kawin Yamtuan, Trirat Radomngam, Pradya Prempraneerach

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik