Predicting behavioral competencies automatically from facial expressions in real-time video-recorded interviews

Yu Sheng Su, Hung Yue Suen*, Kuo En Hung*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Citations (Scopus)


This work aims to develop a real-time image and video processor enabled with an artificial intelligence (AI) agent that can predict a job candidate’s behavioral competencies according to his or her facial expressions. This is accomplished using a real-time video-recorded interview with a histogram of oriented gradients and support vector machine (HOG-SVM) plus convolutional neural network (CNN) recognition. Different from the classical view of recognizing emotional states, this prototype system was developed to automatically decode a job candidate’s behaviors by their microexpressions based on the behavioral ecology view of facial displays (BECV) in the context of employment interviews using a real-time video-recorded interview. An experiment was conducted at a Fortune 500 company, and the video records and competency scores were collected from the company’s employees and hiring managers. The results indicated that our proposed system can provide better predictive power than can human-structured interviews, personality inventories, occupation interest testing, and assessment centers. As such, our proposed approach can be utilized as an effective screening method using a personal-value-based competency model.

Original languageEnglish
Pages (from-to)1011-1021
Number of pages11
JournalJournal of Real-Time Image Processing
Issue number4
Publication statusPublished - 2021 Aug


  • Behavioral ecology view of facial displays (BECV)
  • Convolutional neural network (CNN)
  • Employment selection
  • Histogram of oriented gradients (HOG)
  • Real-time image and video processing
  • Support vector machine (SVM)

ASJC Scopus subject areas

  • Information Systems


Dive into the research topics of 'Predicting behavioral competencies automatically from facial expressions in real-time video-recorded interviews'. Together they form a unique fingerprint.

Cite this