Electric Wheelchair Hybrid Operating System Coordinated with Working Range of a Robotic Arm

Laijun Yang, Nan Guo, Ryota Sakamoto, Norihiko Kato, Ken’ichi Yano

Abstract


Electric wheelchair-mounted robotic arms can help patients with disabilities to perform their activities in daily living (ADL). Joysticks or keypads are commonly used as the operating interface of Wheelchair-mounted robotic arms. Under different scenarios, some patients with upper limb disabilities such as finger contracture cannot operate such interfaces smoothly. Recently, manual interfaces for different symptoms to operate the wheelchair-mounted robotic arms are being developed. However, the stop the wheelchairs in an appropriate position for the robotic arm grasping task is still not easy. To reduce the individual’s burden in operating wheelchair in narrow spaces and to ensure that the chair always stops within the working range of a robotic arm, we propose here an operating system for an electric wheelchair that can automatically drive itself to within the working range of a robotic arm by capturing the position of an AR marker via a chair-mounted camera. Meanwhile, the system includes an error correction model to correct the wheelchair’s moving error. Finally, we demonstrate the effectiveness of the proposed system by running the wheelchair and simulating the robotic arm through several courses.

Keywords


Electric Wheelchair; Self-driving; Robotic Arm; GUI; Error Correction;

Full Text:

PDF

References


Cabinet Office, “Annual Report on Government Measures for Persons with Disabilities,” pp. 213, 2021, link: link: https://www8.cao.go.jp/shougai/whitepaper/index-w.htm .

Cabinet Office, “Annual Report on Government Measures for Persons with Disabilities,” pp. 1, 2011, link: link: https://www8.cao.go.jp/shougai/whitepaper/index-w.htm

Y. Suga, W. Wang, S. Sugano, “Development of Light-Weight Robot Arm Mounted On Wheel Chair with Storage Mechanism,” The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec), 1A1-D14, 2010, doi: 10.1299 /jsmermd. 2010._1A1-A17_1.

N. Yamanobe, Y. Wakita, “Daily Life Support for Persons with Upper-limb Disabilities by Robotic Arm,” The 24th Annual Conference of the Japanese Society for Artificial Intelligence, 1H2-NFC3b-10, 2010, doi: 10.7210/jrsj.29.255 .

V. Maheu, P. Archambault, J. Frappier, F. Routhier, “Evaluation of the JACO robotic arm: Clinico-economic study for powered wheelchair users with upper-extremity disabilities,” IEEE International Conference on Rehabilitation Robotics, pp. 1-5, 2011, doi: 10.1109/ICORR.2011.5975397.

J. Vogel, D. Leidner, A. Hagengruber, M. Panzirsch, B. Bauml, M. Denninger, U. Hillenbrand, L. Suchenwirth, P. Schmaus, M. Sewtz, A. Bauer, T. Hulin, M. Iskandar, G. Quere, A. Albu-Schaffer, A. Dietrich, “An Ecosystem for Heterogeneous Robotic Assistants in Caregiving: Core Functionalities and Use Cases,” IEEE Robotics & Automation Magazine, vol. 28, no. 3, pp. 12-28, 2021, doi: 10.1109/MRA.2020.3032142.

W. Yoon, Y. Wakita, N. Yamanobe, “Self-reliance Support by RAPUDA, a Robot Arm that Supports the Lives of People with Disabilities in the Upper Limbs,” Journal of the Society of Biomechanisms, vol. 37, no. 2, pp. 100-104, 2013, doi: 10.3951/sobim.37.100.

C. Chung, H. Ka, H. Wang, D. Ding, A. Kelleher, A. Cooper, “Performance Evaluation of a Mobile Touchscreen Interface for Assistive Robotic Manipulators: A Pilot Study,” Top Spinal Cord Inj Rehabil, vol. 23, no. 2, pp. 131–139, 2017, doi: 10.1310/sci2302-131.

K. Ichikawa, T. Oka, K. Matsusima, “The Evaluation of Manipulator Operation Interface Using 3D Mouse,” The 78th National Convention of IPSJ, 2Y-01, 2016.

K. Sudheer, T. V. Janardhana Rao, C. Sridevi, M. S. MadhanMohan, “Voice and Gesture Based Electric-Powered Wheelchair Using Arm,” International Journal of Research in Computer and Communication technology (IJRCCT), vol. 1, no. 6, pp. 278-283, 2012.

K. Yamasaki, M. Anraku, T. Shibata, M. Suwa, “Electric Wheelchair Control Using Head Velocity-Based Gesture Recognition Specializing in Turning Motion,” Proceeding of LEM21, pp. 646-650, 2021, doi: 10.1299 /jsmelem. 2021.10.125-118.

M. Mazo, F. J. Rodríguez, J. L. Lazaro, J. Urena, J. C. García, E. Santiso, P. A. Revenga, “Electronic control of a wheelchair guided by voice commands,” Control Engineer Practice, vol. 3, no. 5, pp. 665-674, 1995, doi: 10.1016/0967-0661(95)00042-S.

H. Jiang, T. Zhang, J. Wachs, Bradley Duerstock, “Enhanced Control of a Wheelchair-mounted Robotic Manipulator Using 3D Vision and Multimodal Interaction,” Computer Vision and Image Understanding, vol. 149, pp. 21-31, 2016, doi: 10.1016/j.cviu.2016.03.015.

D. Wang, H. Yu, “Development of the control system of a voice-operated wheelchair with multi-posture characteristics,” 2017 2nd Asia-Pacific Conference on Intelligent Robot Systems, pp. 151-155, 2017, doi: 10.1109/ACIRS.2017.7986083.

R. Alonso, E. Concas, “An Abstraction Layer Exploiting Voice Assistant Technologies for Effective Human—Robot Interaction,” Applied Sciences, vol. 11, no. 19, p. 9165, 2021, doi: 10.3390/app11199165.

K. Lee, W. Chang, S. Kim, C. Im, “Real-Time “Eye-Writing” Recongition Using Electrooculogram (EOG),” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 1, pp. 37-48, 2016, doi: 10.1109/TNSRE.2016.2542524.

C. Ishii, “Control of an Electric Wheelchair Based on EMG, EOG and EEG,” Journal of the Japan Society for Precision Engineering, vol. 83, pp. 1006–1009, 2017, doi: 10.2493/jjspe.83.1006.

J. Lopes, M. Simao, N. Mendes, M. Safeea, J. Afonso, P. Neto, “Hand/arm Gesture Segmentation by Motion Using IMU and EMG Sensing,” 27th International Conference on Flexible Automation and Intelligent Manufacturing, pp. 107-113, 2017, doi: 10.1016/j.promfg.2017.07.158.

C. Han, Y. Kim, D. Kim, S. Kim, Z. Nenadic, C. Im, “Electroencephalography-based endogenous brain-computer interface for online communication with a completely locked-in patient,” Journal of Neuro Engineering and Rehabilitation, vol. 16, no. 18, 2019, doi: 10.1186/s12984-019-0493-0.

W. Chen, S. Chen, Y. Liu,Y. Chen, C. Chen, An Electric Wheelchair Manipulating System Using SSVEP-Based BCI System,” Biosensors, vol. 12, no. 10, p. 772, 2022, doi: doi.org/10.3390/bios12100772.

M. Paing, A. Juhong, C. Pintavirooj, “Design and Development of an Assistive System Based on Eye Tracking,” Electronics, vol. 11, no. 4, p. 535, 2022, doi: 10.3390/electronics11040535.

N. Siribunyaphat, Y. Punsawad, “Steady-State Visual Evoked Potential-Based Brain–Computer Interface Using a Novel Visual Stimulus with Quick Response (QR) Code Pattern,” Sensors, vol. 22, no. 4, p. 1439, 2022, doi: 10.3390/s22041439.

L. Yang, T. Ichikawa,Y. Miyawaki, R. Sakamoto, N. Kato, K.Yano, S. Shimada, “Maloperation Prevention Filter for Touchscreen Used in Robot Arm,” Journal of Physics: Conference Series, vol. 1267, pp. 1–6, 2019, doi: 10.1088/1742-6596/1267/1/012075.

P. Abolghasemi, R. Rahmatizadeh, A. Behal, L. Boloni, “A Real-Time Technique for Positioning a Wheelchair-Mounted Robotic Arm for Household Manipulation Tasks,” The Workshop Program of the Association for the Advancement of Artificial Intelligence’s Thirtieth AAAI Conference on Artificial Intelligence, WS-16-01, pp. 2-7, 2016.

J. Wang, T. Wang, C. Yao, X. Li, C. Wu, “Active Tension Control for WT Wheelchair Robot by Using a Novel Control Law for Holonomic or Nonholonomic Systems,” Mathematical Problems in Engineerning, vol. 2014, 2014, doi: 10.1007/s11432-014-5142-4.

National Rehabilitation Center for Persons with Disabilities, “Electric Wheelchair Compatibility & Operation Practice Manual,” pp.8, 2006, link: https://iss.ndl.go.jp/books/R100000002-I000010852516-00.

A. Ma’arif, A. A. Nuryono, and Iswanto, "Vision-Based Line Following Robot in Webots," 2020 FORTEI-International Conference on Electrical Engineering (FORTEI-ICEE), 2020, pp. 24-28, doi: 10.1109/FORTEI-ICEE50915.2020.9249943.

A. Boubakri, S. Mettali Gamar, “A New Architecture of Autonomous Vehicles: Redundant Architecture to Improve Operational Safety,” International Journal of Robotics and Control Systems, vol. 1, no. 3, pp. 355-368, 2021, doi: 10.31763/ijrcs.v1i3.437.

A. N. Ouda, Amr Mohamed, “Autonomous Fuzzy Heading Controlfor a Multi-Wheeled Combat Vehicle,” International Journal of Robotics and Control Systems, vol. 1, no. 1, pp. 90-101, 2021, doi: 10.31763/ijrcs.v1i3.380ijrcs.

S. Bach, S. Yi, “An Efficient Approach for Line-Following Automated Guided Vehicles Based on Fuzzy Inference Mechanism,” Journal of Robotics and Control, vol. 3, no. 4, 2022, doi: 10.18196/jrc.v3i4.14787.

Y. Yasumuro, S. Kusakabe, H. Dan, “3D Barrier-Free Verification for Wheelchair Accsee,” Proceedings of the 13th International Conference on Construction Applications of Virtual Reality, pp. 564-571, 2013.

K. Tobita, Y. Shikanai, K. Mima, “Study on Automatic Operation of ManualWheelchair Prototype and Basic Experiments,” Journal of Robotics and Mechatronics, vol. 33, no. 1, pp. 69-77, 2021, doi: 10.20965/jrm.2021.p0069.

M. A. Shamseldin, E. Khaled, A. Youssef, D. Mohamed, S. Ahmed, “A New Design Identification and Control Based on GA Optimization for an Autonomous Wheelchair,” Robotics, vol. 11, no. 5, 2022, doi: 10.3390/robotics11050101.

B. Ngo and T. Nguyen, “A Semi-Automatic Wheelchair with Navigation Based on Virtual-Real 2D Grid Maps and EEG Signals,” Applied Science, vol. 12, no. 17, 2022, doi: 10.3390/app12178880.

S. Thrun, W. Burgard, and D. Fox, “Probabilistic Robotics,” The MIT Press, 2005.

Y. Mori, K. Nagao, “Automatic Generation of Multidestination Routesfor Autonomous Wheelchairs,” Journal of Robotics and Mechatronic, vol. 32, no. 6, pp. 1121-1136, 2020, doi: 10.20965/jrm.2020.p1121.

C. Gao, M. Sands, J. Spletzer, “Towards Autonomous Wheelchair Systems in Urban Environments,” Springer Tracts in Advanced Robotics, vol. 62, pp. 13–23, 2010, doi: 10.1007/978-3-642-13408-1_2.

H. Grewal, A. Matthews, R. Tea, K. George, LIDAR-based autonomous wheelchair, 2017 IEEE Sensors Applications Symposium, 2017, doi: 10.1109/SAS.2017.7894082.

M. Yokozuka, O. Matsumoto, “Accurate Localization for Making Maps to Mobile Robots Using Odometry and GPS Without Scan-Matching,” Journal of Robotics and Mechatronics, vol. 27, no. 4, pp. 410-418, 2015, doi: 10.20965/jrm.2015.p0410.

W. Rahiman, Z. Zainal, “An Overview of Development GPS Navigation for Autonomous car,” IEEE 8th Conference on Industrial Electronics and Applications, pp. 1112-1118, 2013, doi: 10.1109/ICIEA.2013.6566533.

N. Aktar, I. Jaharr, B. Lala, “Voice Recognition based intelligent Wheelchair and GPS Tracking System,” 2019 International Conference on Electrical, Computer and Communication Engineering, pp. 1-5, 2019, doi: 10.1109/ECACE.2019.8679163.

C. Qinghua, Z. Taihu., X. Yihong, S. Hongyu, “Design and Realization of Intelligent Wheelchair Based on STM32,” Journal of Physics: Conference Series, vol. 1678, p. 012114, 2020, doi: 10.1088/1742-6596/1678/1/012114.

N. Yokoya, “Mixed Reality: Merging Real and Virtual Worlds: I Introduction,” Systems, Control And Information, vol. 49, no. 12, pp. 489-494, 2005, doi: 10.11509/isciesci.49.12_489.

S. Hashizume, I. Suzuki, K. Takazawa, R. Sasaki, Y. Ochiai, “Telewheelchair: the Remote Controllable Electric Wheelchair System combined Human and Machine Intelligence,” Proceedings of the 9th Augmented Human International Conference, pp. 1-9, 2018, doi: 10.1145/3174910.3174914.

M. Zolotas, J. Elsdon, and Y. Demiris, “Head-mounted augmented reality for explainable robotic wheelchair assistance,” 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018, doi: 10.1109/IROS.2018.8594002.

K. Motooka, T. Okawara, K. Go, Y. Yamato, A. Miyata, “Study on Distance Reduction of a Barrier Simulator Using an HMD and an Electric Wheelchair,” The 24th Annual Conference of the Virtual Reality Society of Japan, 2019.

R. Quesada, Y. Demiris, “Augmented Reality Controlled Smart Wheelchair Using Dynamic Signifiers for Affordance Representation,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 4812-4818, 2019, doi: 10.1109/IROS40897.2019.8968290.

N. Matsunaga, Y. Takeuchi, H. Okajima, “A Construction Method of Obstacle Avoidance System for Welfare Vehicle by Spatial Mapping of Driving Environment using Mixed Reality,” Transactions of the JSME, vol. 87, no. 894, pp. 1-16, 2021, doi: 10.1299/transjsme.20-00033.

T. Mizunami, Y. Sakamura, A. Tomita, K. Inoue, I. Kitahara, E. Harada, “Reduction of Anxiety for an Autonomous Vehicle by Augmented Visual Information,” The 82nd Annual Convention of the Japanese Psychological Association, 3EV-056, 2018 doi: 10.4992/pacjpa.82.0_3EV-056 .

H. Otsuka, R. Yajima, K. Nagatati, D. Kubo, “Localization of a Small Multi−rotor UAV Using Dropping AR Markers,” The Proceedings of JSME annual Conference on Robotics and Mechatronics, 2A2-G02, 2015, doi: 10.1299/jsmermd.2015._2A2-G02_1.

F. Sato, T. Koshizen, T. Matsumoto, H. Kawase, S. Miyamoto, Y. Torimoto, “Self-driving system for electric wheelchair using smartphone to estimate travelable areas,” 2018 IEEE International Conference on Systems, Man, and Cybernetics, pp. 298–304, 2018, doi: 10.1109/SMC.2018.00061.

J. Yu,W. Jiang, Z. Luo, and L. Yang, “Application of a Vision-Based Single Target on Robot Positioning System,” Sensors, vol. 21, no. 5, p. 1829, 2021, doi: 10.3390/s21051829.

M. Teoh,K. Teo, H. Yoong, “Numerical Computation-Based Position Estimation for QR Code Object Marker: Mathematical Model and Simulation,” Computation, vol. 10, no. 9, p. 147, 2022, doi: 10.3390/computation10090147.

J. Kato, G. Deguchi, J. Inoue, M. Iwase, “Improvement of Performance of Navigation System for Supporting Independence Rehabilitation of Wheelchair – Bed Transfer,” Journal of Physics: Conf. Series, vol. 1487, p. 012041, 2020, doi: 10.1088/1742-6596/1487/1/012041.

P. Hebert, J. Ma, J.Borders, A.Aydemir, M. Bajracharya, N. Hudson, K. Shankar, S. Karumanchi, B. Douillard, J. Burdick, “Supervised Remote Robot with Guided Autonomy and Teleoperation (SURROGATE): A Framework for Whole-Body Manipulation,” IEEE International Conference on Robotics and Automation, pp. 5509-5516, 2015, doi; 10.1109/ICRA.2015.7139969.

L. Yang, T .Ichikawa, R. Sakamoto, T. Itami, N. Kato, K. Yano, S. Shimada, “Robot Arm Operating Interface Using a Touch Panel for Gripping an Object on the Floor by Designating the End Effector,” 2020 IEEE/SICE International Symposium on System Integration, pp. 401–404, 2020, doi: 10.1109/SII46433.2020.9026233 .

E. Tanaka, T. Ichikawa, R. Sakamoto, K. Yano, S. Shimada, “7-DOF Robot Arm "Udero" for Self-Reliance Support of Disabled Persons,” The 17th SICE System Integration Division Annual Conference, 2016.

Y. Miyawaki, L. Yang, R. Sakamoto, N. Kato, K. Yano, S. Shimada, “Interface for Operating Electric Wheelchair without Gazing Operation Screen,” JSME The Proceedings of Conference of Tokai Branch, vol. 68, 2019, doi: 10.1299/jsmetokai.2019.68.519.

Y. Miyawaki, L. Yang, R. Sakamoto, N. Kato, K. Yano, P. Minyong, “Electric Wheelchair Control System without Gazing the Operation Screen,” The 9th TSME International Conference on Mechanical Engineering, p. 2017-0105, 2018.

H. Kato, “ARToolKit: Library for Vision-based Augmented Reality,” IEICE Technical Report, vol. 101, no. 652, pp.79–86, 2002.

H. Tanaka, K. Ogata, Y. Matsumoto, “Improving the accuracy of visual markers by four dots and Image interpolation,” 2016 International Symposium on Robotics and Intelligent Sensors, pp. 178–183, 2016, doi: 10.1109/IRIS.2016.8066087.

F. Sato, “Improvement of Self Position Estimation of Electric Wheelchair Combining Multiple Positioning Methods,” Advanced Information Networking and Applications, pp 607–618, 2019, doi: 10.1007/978-3-030-15032-7_51.




DOI: https://doi.org/10.18196/jrc.v3i5.15944

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Laijun Yang, Nan Guo, Ryota Sakamoto, Norihiko Kato, Ken’ichi Yano

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik