Toward the flexible automation for robot learning from human demonstration using multimodal perception approach

Jing Hao Chen, Guan Yi Lu, Yi Hsing Chien, Hsin Han Chiang, Wei Yen Wang, Chen Chien Hsu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This study proposes a multi-modal perception approach to make a robotic arm perform flexible automation and further simplify the complicated coding process of controlling a robotic arm. The depth camera is utilized for detect face and hand gesture for recognizing operator identification and commands. In addition, the kinematics of the robotic arm associated with the position of manipulated objects can be derived based on the information through human demonstrations and detected objects. In the experiments, the proposed multi-modal perception system can firstly recognize the operator. Then, the operator can demonstrate a task to generate the learning data with the assistance of using gesture. Afterward, the robotic arm can perform the same task as human demonstration. During the process of imitating task, the robotic arm can also be guided by the gesture command of operator.

Original languageEnglish
Title of host publicationProceedings of 2019 International Conference on System Science and Engineering, ICSSE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages148-153
Number of pages6
ISBN (Electronic)9781728105253
DOIs
Publication statusPublished - 2019 Jul
Event2019 International Conference on System Science and Engineering, ICSSE 2019 - Dong Hoi City, Quang Binh Province, Viet Nam
Duration: 2019 Jul 202019 Jul 21

Publication series

NameProceedings of 2019 International Conference on System Science and Engineering, ICSSE 2019

Conference

Conference2019 International Conference on System Science and Engineering, ICSSE 2019
CountryViet Nam
CityDong Hoi City, Quang Binh Province
Period19/7/2019/7/21

Keywords

  • face recognition
  • gesture recognition
  • human demonstration
  • multi-modal perception
  • object recognition

ASJC Scopus subject areas

  • Energy Engineering and Power Technology
  • Safety, Risk, Reliability and Quality
  • Control and Optimization
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems and Management

Fingerprint Dive into the research topics of 'Toward the flexible automation for robot learning from human demonstration using multimodal perception approach'. Together they form a unique fingerprint.

  • Cite this

    Chen, J. H., Lu, G. Y., Chien, Y. H., Chiang, H. H., Wang, W. Y., & Hsu, C. C. (2019). Toward the flexible automation for robot learning from human demonstration using multimodal perception approach. In Proceedings of 2019 International Conference on System Science and Engineering, ICSSE 2019 (pp. 148-153). [8823444] (Proceedings of 2019 International Conference on System Science and Engineering, ICSSE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSSE.2019.8823444