Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources

Chiung Yao Fang, An Chun Luo*, Yu Shan Deng, Chia Ju Lu, Sei Wang Chen


研究成果: 雜誌貢獻期刊論文同行評審

2 引文 斯高帕斯(Scopus)


Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.

頁(從 - 到)3759-3777
期刊Neural Computing and Applications
出版狀態已發佈 - 2019 8月 1

ASJC Scopus subject areas

  • 軟體
  • 人工智慧


深入研究「Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources」主題。共同形成了獨特的指紋。