Popularity prediction of social media based on multi-modal feature mining

Chih Chung Hsu, Jun Yi Lee, Li Wei Kang, Zhong Xuan Zhang, Chia Yen Lee, Shao Min Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Popularity prediction of social media becomes a more attractive issue in recent years. It consists of multi-type data sources such as image, meta-data, and text information. In order to effectively predict the popularity of a specified post in the social network, fusing multi-feature from heterogeneous data is required. In this paper, a popularity prediction framework for social media based on multi-modal feature mining is presented. First, we discover image semantic features by extracting their image descriptions generated by image captioning. Second, an effective text-based feature engineering is used to construct an effective word-to-vector model. The trained word-to-vector model is used to encode the text information and the semantic image features. Finally, an ensemble regression approach is proposed to aggregate these encoded features and learn the final regressor. Extensive experiments show that the proposed method significantly outperforms other state-of-the-art regression models. We also show that the multi-modal approach could effectively improve the performance in the social media prediction challenge.

Original languageEnglish
Title of host publicationMM 2019 - Proceedings of the 27th ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery, Inc
Pages2687-2691
Number of pages5
ISBN (Electronic)9781450368896
DOIs
Publication statusPublished - 2019 Oct 15
Event27th ACM International Conference on Multimedia, MM 2019 - Nice, France
Duration: 2019 Oct 212019 Oct 25

Publication series

NameMM 2019 - Proceedings of the 27th ACM International Conference on Multimedia

Conference

Conference27th ACM International Conference on Multimedia, MM 2019
Country/TerritoryFrance
CityNice
Period2019/10/212019/10/25

Keywords

  • CNN
  • Ensemble learning
  • Image captioning
  • Multimodal learning
  • Regression

ASJC Scopus subject areas

  • Computer Science(all)
  • Media Technology

Fingerprint

Dive into the research topics of 'Popularity prediction of social media based on multi-modal feature mining'. Together they form a unique fingerprint.

Cite this