A Social Condition-Enhanced Network for Recognizing Power Distance using Expressive Prosody and Intrinsic Brain Connectivity

Fu Sheng Tsai, Wei Wen Chang, Chi Chun Lee

Research output: Contribution to journalArticlepeer-review

Abstract

Culture is the social norm that often dictates a person's thoughts, decision-making, and social behaviors during interaction at an individual level. In this study, we present a computational framework that automatically assesses an individual culture attribute of power distance (PDI), i.e., the measure to describe one's acceptance of social status, power and authority in organizations through multimodal modeling of a participant's expressive prosodic structures and brain connectivity using a social condition-enhanced network. In specific, we propose a joint learning approach of center-loss embedding network architecture that learns to "centerize" the embedding space given a particular social interaction condition to enhance the PDI discriminability of the representation. Our proposed method achieves 88.5% and 73.1% in binary classification task of recognizing low versus high power distance on prosodic and fMRI modality separately. After performing multimodal fusion, it improves to 96.2% of 2-class recognition rate (7.7% relative improvement). Further analyses reveal that average and standard deviation of speech energy are significantly correlated with power distance index; the right middle cingulate cortex (MCC) of brain region achieves the best recognition accuracy demonstrating its role in processing a person's belief about power distance.

Original languageEnglish
JournalIEEE Transactions on Multimedia
DOIs
Publication statusAccepted/In press - 2021

Keywords

  • center-loss embedding
  • Cultural differences
  • culture dimensions
  • fMRI
  • Global communication
  • Loss measurement
  • Organizations
  • power distance index
  • prosody
  • Quality of experience
  • Reliability
  • Speech recognition

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A Social Condition-Enhanced Network for Recognizing Power Distance using Expressive Prosody and Intrinsic Brain Connectivity'. Together they form a unique fingerprint.

Cite this