Enhanced Language Modeling with Proximity and Sentence Relatedness Information for Extractive Broadcast News Summarization

Shih Hung Liu*, Kuan Yu Chen, Berlin Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)


The primary task of extractive summarization is to automatically select a set of representative sentences from a text or spoken document that can concisely express the most important theme of the original document. Recently, language modeling (LM) has been proven to be a promising modeling framework for performing this task in an unsupervised manner. However, there still remain three fundamental challenges facing the existing LM-based methods, which we set out to tackle in this article. The first one is how to construct a more accurate sentence model in this framework without resorting to external sources of information. The second is how to take into account sentence-level structural relationships, in addition to word-level information within a document, for important sentence selection. The last one is how to exploit the proximity cues inherent in sentences to obtain a more accurate estimation of respective sentence models. Specifically, for the first and second challenges, we explore a novel, principled approach that generates overlapped clusters to extract sentence relatedness information from the document to be summarized, which can be used not only to enhance the estimation of various sentence models but also to render sentence-level structural relationships within the document, leading to better summarization effectiveness. For the third challenge, we investigate several formulations of proximity cues for use in sentence modeling involved in the LM-based summarization framework, free of the strict bag-of-words assumption. Furthermore, we also present various ensemble methods that seamlessly integrate proximity and sentence relatedness information into sentence modeling. Extensive experiments conducted on a Mandarin broadcast news summarization task show that such integration of proximity and sentence relatedness information is indeed beneficial for speech summarization. Our proposed summarization methods can significantly boost the performance of an LM-based strong baseline (e.g., with a maximum ROUGE-2 improvement of 26.7% relative) and also outperform several state-of-the-art unsupervised methods compared in the article.

Original languageEnglish
Article number3377407
JournalACM Transactions on Asian and Low-Resource Language Information Processing
Issue number3
Publication statusPublished - 2020 Feb 7
Externally publishedYes


  • Extractive summarization
  • language modeling
  • overlapped clustering
  • proximity information

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'Enhanced Language Modeling with Proximity and Sentence Relatedness Information for Extractive Broadcast News Summarization'. Together they form a unique fingerprint.

Cite this