Online Human Action Recognition Using Deep Learning for Indoor Smart Mobile Robots

Jih Tang Hsieh, Meng Lin Chiang, Chiung Yao Fang, Sei Wang Chen

研究成果: 書貢獻/報告類型會議論文篇章

摘要

This research proposes a vision-based online human action recognition system. This system uses deep learning methods to recognise human action under moving camera circumstances. The proposed system consists of five stages: human detection, human tracking, feature extraction, action classification and fusion. The system uses three kinds of input information: colour intensity, short-term dynamic information and skeletal joints. In the human detection stage, a two-dimensional (2D) pose estimator method is used to detect a human. In the human tracking stage, a deep SORT tracking method is used to track the human. In the feature extraction stage, three kinds of features, spatial, temporal and structural, are extracted to analyse human actions. In the action classification stage, three kinds of features of human actions are respectively classified by three kinds of long short-term memory (LSTM) classifiers. In the fusion stage, a fusion method is used to leverage the three output results from the LSTM classifiers. This study constructs a computer vision and image understanding (CVIU) Moving Camera Human Action dataset (CVIU dataset), containing 3, 646 human action sequences, including 11 types of single human actions and 5 types of interactive human actions. This dataset was used to train and evaluate the proposed system. Experimental results showed that the recognition rates of spatial features, temporal features and structural features were 96.64%, 81.87% and 68.10%, respectively. Finally, the fusion result of human action recognition for indoor smart mobile robots in this study was 96.84%.

原文英語
主出版物標題Proceedings - IEEE 2021 International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2021
編輯Parma Nand Astya, Manjeet Singh, Nihar Ranjan Roy, Gaurav Raj
發行者Institute of Electrical and Electronics Engineers Inc.
頁面425-433
頁數9
ISBN(電子)9781728185293
DOIs
出版狀態已發佈 - 2021 二月 19
事件2021 IEEE International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2021 - Greater Noida, 印度
持續時間: 2021 二月 192021 二月 20

出版系列

名字Proceedings - IEEE 2021 International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2021

會議

會議2021 IEEE International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2021
國家印度
城市Greater Noida
期間2021/02/192021/02/20

ASJC Scopus subject areas

  • Renewable Energy, Sustainability and the Environment
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

指紋 深入研究「Online Human Action Recognition Using Deep Learning for Indoor Smart Mobile Robots」主題。共同形成了獨特的指紋。

引用此