An infant emotion recognition system using visual and audio information

Chiung Yao Fang, Chung Wen Ma, Meng Lin Chiang, Sei Wang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

This study presents an infant emotion recognition system using visual and audio information for infants aged 1 to 7 months. The system is divided into two parts, image processing and speech processing. The image processing part detects the infant's face and extracts facial expressions features. In the face detection stage, the system selects the largest skin color region as the facial area, while in the facial expressions feature extraction stage, the system uses the local ternary pattern (LTP) technology to label facial contours and calculates their corresponding Zernike moments. In the speech processing part, the system uses common mel-frequency cepstral coefficients (MFCCs) and its delta cepstrum coefficients as vocalization features. Finally, the system uses support vector machines (SVMs) to classify the facial expressions features and vocalization features, respectively. By combining these types of classification results, the system reaches a decision about the infant's emotion. The average recognition rate of infant emotions is 85.3% in the experiments which, in our view, makes the proposed system robust and efficient.

Original languageEnglish
Title of host publication2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages284-291
Number of pages8
ISBN (Electronic)9781509067749
DOIs
Publication statusPublished - 2017 Jun 5
Event4th International Conference on Industrial Engineering and Applications, ICIEA 2017 - Nagoya, Japan
Duration: 2017 Apr 212017 Apr 23

Other

Other4th International Conference on Industrial Engineering and Applications, ICIEA 2017
CountryJapan
CityNagoya
Period17/4/2117/4/23

Fingerprint

Speech processing
Image processing
Face recognition
Support vector machines
Feature extraction
Labels
Skin
Color
Experiments
Emotion

Keywords

  • infant emotion recognition
  • infant monitoring system
  • local ternary pattern (LTP)
  • mel-frequency cepstral coefficients (MFCCs)
  • Zernike moments

ASJC Scopus subject areas

  • Strategy and Management
  • Industrial and Manufacturing Engineering

Cite this

Fang, C. Y., Ma, C. W., Chiang, M. L., & Chen, S. W. (2017). An infant emotion recognition system using visual and audio information. In 2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017 (pp. 284-291). [7939223] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEA.2017.7939223

An infant emotion recognition system using visual and audio information. / Fang, Chiung Yao; Ma, Chung Wen; Chiang, Meng Lin; Chen, Sei Wang.

2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 284-291 7939223.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fang, CY, Ma, CW, Chiang, ML & Chen, SW 2017, An infant emotion recognition system using visual and audio information. in 2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017., 7939223, Institute of Electrical and Electronics Engineers Inc., pp. 284-291, 4th International Conference on Industrial Engineering and Applications, ICIEA 2017, Nagoya, Japan, 17/4/21. https://doi.org/10.1109/IEA.2017.7939223
Fang CY, Ma CW, Chiang ML, Chen SW. An infant emotion recognition system using visual and audio information. In 2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 284-291. 7939223 https://doi.org/10.1109/IEA.2017.7939223
Fang, Chiung Yao ; Ma, Chung Wen ; Chiang, Meng Lin ; Chen, Sei Wang. / An infant emotion recognition system using visual and audio information. 2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 284-291
@inproceedings{0e407a174d2544f5b38146c48c6a78d7,
title = "An infant emotion recognition system using visual and audio information",
abstract = "This study presents an infant emotion recognition system using visual and audio information for infants aged 1 to 7 months. The system is divided into two parts, image processing and speech processing. The image processing part detects the infant's face and extracts facial expressions features. In the face detection stage, the system selects the largest skin color region as the facial area, while in the facial expressions feature extraction stage, the system uses the local ternary pattern (LTP) technology to label facial contours and calculates their corresponding Zernike moments. In the speech processing part, the system uses common mel-frequency cepstral coefficients (MFCCs) and its delta cepstrum coefficients as vocalization features. Finally, the system uses support vector machines (SVMs) to classify the facial expressions features and vocalization features, respectively. By combining these types of classification results, the system reaches a decision about the infant's emotion. The average recognition rate of infant emotions is 85.3{\%} in the experiments which, in our view, makes the proposed system robust and efficient.",
keywords = "infant emotion recognition, infant monitoring system, local ternary pattern (LTP), mel-frequency cepstral coefficients (MFCCs), Zernike moments",
author = "Fang, {Chiung Yao} and Ma, {Chung Wen} and Chiang, {Meng Lin} and Chen, {Sei Wang}",
year = "2017",
month = "6",
day = "5",
doi = "10.1109/IEA.2017.7939223",
language = "English",
pages = "284--291",
booktitle = "2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - An infant emotion recognition system using visual and audio information

AU - Fang, Chiung Yao

AU - Ma, Chung Wen

AU - Chiang, Meng Lin

AU - Chen, Sei Wang

PY - 2017/6/5

Y1 - 2017/6/5

N2 - This study presents an infant emotion recognition system using visual and audio information for infants aged 1 to 7 months. The system is divided into two parts, image processing and speech processing. The image processing part detects the infant's face and extracts facial expressions features. In the face detection stage, the system selects the largest skin color region as the facial area, while in the facial expressions feature extraction stage, the system uses the local ternary pattern (LTP) technology to label facial contours and calculates their corresponding Zernike moments. In the speech processing part, the system uses common mel-frequency cepstral coefficients (MFCCs) and its delta cepstrum coefficients as vocalization features. Finally, the system uses support vector machines (SVMs) to classify the facial expressions features and vocalization features, respectively. By combining these types of classification results, the system reaches a decision about the infant's emotion. The average recognition rate of infant emotions is 85.3% in the experiments which, in our view, makes the proposed system robust and efficient.

AB - This study presents an infant emotion recognition system using visual and audio information for infants aged 1 to 7 months. The system is divided into two parts, image processing and speech processing. The image processing part detects the infant's face and extracts facial expressions features. In the face detection stage, the system selects the largest skin color region as the facial area, while in the facial expressions feature extraction stage, the system uses the local ternary pattern (LTP) technology to label facial contours and calculates their corresponding Zernike moments. In the speech processing part, the system uses common mel-frequency cepstral coefficients (MFCCs) and its delta cepstrum coefficients as vocalization features. Finally, the system uses support vector machines (SVMs) to classify the facial expressions features and vocalization features, respectively. By combining these types of classification results, the system reaches a decision about the infant's emotion. The average recognition rate of infant emotions is 85.3% in the experiments which, in our view, makes the proposed system robust and efficient.

KW - infant emotion recognition

KW - infant monitoring system

KW - local ternary pattern (LTP)

KW - mel-frequency cepstral coefficients (MFCCs)

KW - Zernike moments

UR - http://www.scopus.com/inward/record.url?scp=85021439723&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021439723&partnerID=8YFLogxK

U2 - 10.1109/IEA.2017.7939223

DO - 10.1109/IEA.2017.7939223

M3 - Conference contribution

AN - SCOPUS:85021439723

SP - 284

EP - 291

BT - 2017 4th International Conference on Industrial Engineering and Applications, ICIEA 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -