Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources

Chiung Yao Fang, An Chun Luo, Yu Shan Deng, Chia Ju Lu, Sei Wang Chen

Research output: Contribution to journalArticle

Abstract

Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.

LanguageEnglish
Pages1-19
Number of pages19
JournalNeural Computing and Applications
DOIs
Publication statusAccepted/In press - 2018 Jan 6

Fingerprint

Neural networks
Broadcasting
Learning systems
Cameras
Experiments

Keywords

  • Automatic lecture-recording system
  • Shot selection
  • Virtual cameraman
  • Virtual director

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources. / Fang, Chiung Yao; Luo, An Chun; Deng, Yu Shan; Lu, Chia Ju; Chen, Sei Wang.

In: Neural Computing and Applications, 06.01.2018, p. 1-19.

Research output: Contribution to journalArticle

@article{a9ae6a72e558419bbf07f5814bc95345,
title = "Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources",
abstract = "Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.",
keywords = "Automatic lecture-recording system, Shot selection, Virtual cameraman, Virtual director",
author = "Fang, {Chiung Yao} and Luo, {An Chun} and Deng, {Yu Shan} and Lu, {Chia Ju} and Chen, {Sei Wang}",
year = "2018",
month = "1",
day = "6",
doi = "10.1007/s00521-017-3328-6",
language = "English",
pages = "1--19",
journal = "Neural Computing and Applications",
issn = "0941-0643",
publisher = "Springer London",

}

TY - JOUR

T1 - Building a smart lecture-recording system using MK-CPN network for heterogeneous data sources

AU - Fang, Chiung Yao

AU - Luo, An Chun

AU - Deng, Yu Shan

AU - Lu, Chia Ju

AU - Chen, Sei Wang

PY - 2018/1/6

Y1 - 2018/1/6

N2 - Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.

AB - Nowadays, lecture-recording systems play a vital role in collecting spoken discourse for e-learning. However, in view of the growing development of e-learning, the lack of content is becoming a problem. This research presents a smart lecture-recording (SLR) system that can record orations at the same level of quality as a human team, but with a reduced degree of human involvement. The proposed SLR system is composed of two subsystems, referred to as virtual cameraman (VC), and virtual director (VD), respectively. All camera man components of VC subsystem are automatic and can take actions that include target and event detection, tracking, and view searching. The videos taken by these three components are forwarded to the VD subsystem, in which the representative shot is chosen for recording or direct broadcasting. We refer to this function of the VD subsystem as shot selection that is based on the content analysis. The capability of shot selection is pre-trained through a machine-learning process characterized by the counter-propagation neural (CPN) network. However, the CPN network yielded poor results when the input data were heterogeneous data. To increases the accuracy of shot selection, we applied multiple kernel learning (MKL) techniques into CPN network, called MK-CPN, to transform all the heterogeneous data from different content analysis methods into unified space. A series of experiments for real lecture has been conducted. The results showed that the proposed SLR system can provide oration records close to some extend to those taken by real human teams. We believe that the proposed system may not be limited to live speeches, if it can be configured with appropriate training materials.

KW - Automatic lecture-recording system

KW - Shot selection

KW - Virtual cameraman

KW - Virtual director

UR - http://www.scopus.com/inward/record.url?scp=85040086482&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85040086482&partnerID=8YFLogxK

U2 - 10.1007/s00521-017-3328-6

DO - 10.1007/s00521-017-3328-6

M3 - Article

SP - 1

EP - 19

JO - Neural Computing and Applications

T2 - Neural Computing and Applications

JF - Neural Computing and Applications

SN - 0941-0643

ER -