Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources

Chiung Yao Fang, An Chun Luo*, Yu Shan Deng, Chia Ju Lu, Sei Wang Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.

Original languageEnglish
Pages (from-to)3759-3777
Number of pages19
JournalNeural Computing and Applications
Issue number8
Publication statusPublished - 2019 Aug 1


  • Automatic lecture-recording system
  • Shot selection
  • Virtual cameraman
  • Virtual director

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources'. Together they form a unique fingerprint.

Cite this