Development of a Mimic Robot: Learning from Human Demonstration to Manipulate a Coffee Maker as an Example

Pin Jui Hwang, Chen Chien Hsu, Wei Yen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the trends of DIY movements and the maker economy, great needs for the applications of low volume automation (LVA) are predictable. To do so, a learning from demonstration (LfD) problem is addressed in this paper, where robots are taught through demonstrated actions to manipulate a coffee maker. The system is developed using YOLO deep learning architecture to recognize objects such as cups, coffee capsules. Employing a Kinect RGB-D camera, the robot is capable of obtaining the coordinates of the objects and the corresponding moving trajectories. Integrating the above two techniques, the robot is able to recognize the demonstrated actions to establish an action database comprising several sub-actions such as moving a cup, pouring coffee, trigger the coffee machine, etc. Finally, the manipulation of the robot is made by following the order of demonstrated actions. As a result, a vision-based LfD system is established, allowing the robot to learn from human demonstrations and act accordingly.

Original languageEnglish
Title of host publication2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages124-127
Number of pages4
ISBN (Electronic)9781728135700
DOIs
Publication statusPublished - 2019 Jun
Event23rd IEEE International Symposium on Consumer Technologies, ISCT 2019 - Ancona, Italy
Duration: 2019 Jun 192019 Jun 21

Publication series

Name2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019

Conference

Conference23rd IEEE International Symposium on Consumer Technologies, ISCT 2019
CountryItaly
CityAncona
Period19/6/1919/6/21

Fingerprint

Robot learning
coffee
Coffee
robots
learning
Demonstrations
Robots
pouring
economy
capsules
automation
manipulators
Automation
actuators
Cameras
cameras
Trajectories
trajectories
trends

Keywords

  • Deep Learning
  • Learning from Demonstration
  • Mimic Robot
  • Yolo

ASJC Scopus subject areas

  • Instrumentation
  • Biomedical Engineering
  • Electrical and Electronic Engineering
  • Media Technology

Cite this

Hwang, P. J., Hsu, C. C., & Wang, W. Y. (2019). Development of a Mimic Robot: Learning from Human Demonstration to Manipulate a Coffee Maker as an Example. In 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019 (pp. 124-127). [8901025] (2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCE.2019.8901025

Development of a Mimic Robot : Learning from Human Demonstration to Manipulate a Coffee Maker as an Example. / Hwang, Pin Jui; Hsu, Chen Chien; Wang, Wei Yen.

2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 124-127 8901025 (2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hwang, PJ, Hsu, CC & Wang, WY 2019, Development of a Mimic Robot: Learning from Human Demonstration to Manipulate a Coffee Maker as an Example. in 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019., 8901025, 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019, Institute of Electrical and Electronics Engineers Inc., pp. 124-127, 23rd IEEE International Symposium on Consumer Technologies, ISCT 2019, Ancona, Italy, 19/6/19. https://doi.org/10.1109/ISCE.2019.8901025
Hwang PJ, Hsu CC, Wang WY. Development of a Mimic Robot: Learning from Human Demonstration to Manipulate a Coffee Maker as an Example. In 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 124-127. 8901025. (2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019). https://doi.org/10.1109/ISCE.2019.8901025
Hwang, Pin Jui ; Hsu, Chen Chien ; Wang, Wei Yen. / Development of a Mimic Robot : Learning from Human Demonstration to Manipulate a Coffee Maker as an Example. 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 124-127 (2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019).
@inproceedings{922fe81e9cf440beb8a7146e40803f30,
title = "Development of a Mimic Robot: Learning from Human Demonstration to Manipulate a Coffee Maker as an Example",
abstract = "With the trends of DIY movements and the maker economy, great needs for the applications of low volume automation (LVA) are predictable. To do so, a learning from demonstration (LfD) problem is addressed in this paper, where robots are taught through demonstrated actions to manipulate a coffee maker. The system is developed using YOLO deep learning architecture to recognize objects such as cups, coffee capsules. Employing a Kinect RGB-D camera, the robot is capable of obtaining the coordinates of the objects and the corresponding moving trajectories. Integrating the above two techniques, the robot is able to recognize the demonstrated actions to establish an action database comprising several sub-actions such as moving a cup, pouring coffee, trigger the coffee machine, etc. Finally, the manipulation of the robot is made by following the order of demonstrated actions. As a result, a vision-based LfD system is established, allowing the robot to learn from human demonstrations and act accordingly.",
keywords = "Deep Learning, Learning from Demonstration, Mimic Robot, Yolo",
author = "Hwang, {Pin Jui} and Hsu, {Chen Chien} and Wang, {Wei Yen}",
year = "2019",
month = "6",
doi = "10.1109/ISCE.2019.8901025",
language = "English",
series = "2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "124--127",
booktitle = "2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019",

}

TY - GEN

T1 - Development of a Mimic Robot

T2 - Learning from Human Demonstration to Manipulate a Coffee Maker as an Example

AU - Hwang, Pin Jui

AU - Hsu, Chen Chien

AU - Wang, Wei Yen

PY - 2019/6

Y1 - 2019/6

N2 - With the trends of DIY movements and the maker economy, great needs for the applications of low volume automation (LVA) are predictable. To do so, a learning from demonstration (LfD) problem is addressed in this paper, where robots are taught through demonstrated actions to manipulate a coffee maker. The system is developed using YOLO deep learning architecture to recognize objects such as cups, coffee capsules. Employing a Kinect RGB-D camera, the robot is capable of obtaining the coordinates of the objects and the corresponding moving trajectories. Integrating the above two techniques, the robot is able to recognize the demonstrated actions to establish an action database comprising several sub-actions such as moving a cup, pouring coffee, trigger the coffee machine, etc. Finally, the manipulation of the robot is made by following the order of demonstrated actions. As a result, a vision-based LfD system is established, allowing the robot to learn from human demonstrations and act accordingly.

AB - With the trends of DIY movements and the maker economy, great needs for the applications of low volume automation (LVA) are predictable. To do so, a learning from demonstration (LfD) problem is addressed in this paper, where robots are taught through demonstrated actions to manipulate a coffee maker. The system is developed using YOLO deep learning architecture to recognize objects such as cups, coffee capsules. Employing a Kinect RGB-D camera, the robot is capable of obtaining the coordinates of the objects and the corresponding moving trajectories. Integrating the above two techniques, the robot is able to recognize the demonstrated actions to establish an action database comprising several sub-actions such as moving a cup, pouring coffee, trigger the coffee machine, etc. Finally, the manipulation of the robot is made by following the order of demonstrated actions. As a result, a vision-based LfD system is established, allowing the robot to learn from human demonstrations and act accordingly.

KW - Deep Learning

KW - Learning from Demonstration

KW - Mimic Robot

KW - Yolo

UR - http://www.scopus.com/inward/record.url?scp=85075638723&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075638723&partnerID=8YFLogxK

U2 - 10.1109/ISCE.2019.8901025

DO - 10.1109/ISCE.2019.8901025

M3 - Conference contribution

AN - SCOPUS:85075638723

T3 - 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019

SP - 124

EP - 127

BT - 2019 IEEE 23rd International Symposium on Consumer Technologies, ISCT 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -