KNOT-MCTS: An Effective Approach to Addressing Hallucinations in Generative Language Modeling for Question Answering

Chung Wen Wu, Guan Tang Huang, Yue Yang He, Berlin Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Contemporary large language models (LLMs) have made significant advancements, capable of generating fluent conversations with humans and accomplishing various tasks such as programming and question answering (QA). Nevertheless, current LLMs are still faced with numerous challenges, including generating hallucinations, lacking the latest information, suffering from biases, and others. In this paper, we proposed a technique, Knowledge-based Navigation for Optimal Truthfulness Monte Carlo Tree Search (KNOT-MCTS), which can reduce hallucinations of LLMs by aligning semantics of responses with external knowledge during the generation process. This technique acts as a plug-and-play knowledge injection method, which does not require any training and can be applied to any (large) language model. First, we retrieve relevance knowledge snippets, incorporating them into the prompt section and subsequently fed into the decoding process. Then, during the decoding process, we utilize our semantic alignment heuristic function to guide the response generation process of LMs through the Monte Carlo Tree Search (MCTS) decoding process. In our experiments on the TruthfulQA dataset, KNOT-MCTS paired with various LMs consistently outperforms their respective baselines. Our results demonstrate that KNOT-MCTS can effectively inject knowledge into various LMs to reduce hallucinations of LMs.

Original languageEnglish
Title of host publicationROCLING 2023 - Proceedings of the 35th Conference on Computational Linguistics and Speech Processing
EditorsJheng-Long Wu, Ming-Hsiang Su, Hen-Hsen Huang, Yu Tsao, Hou-Chiang Tseng, Chia-Hui Chang, Lung-Hao Lee, Yuan-Fu Liao, Wei-Yun Ma
PublisherThe Association for Computational Linguistics and Chinese Language Processing (ACLCLP)
Pages215-221
Number of pages7
ISBN (Electronic)9789869576963
Publication statusPublished - 2023
Event35th Conference on Computational Linguistics and Speech Processing, ROCLING 2023 - Taipei City, Taiwan
Duration: 2023 Oct 202023 Oct 21

Publication series

NameROCLING 2023 - Proceedings of the 35th Conference on Computational Linguistics and Speech Processing

Conference

Conference35th Conference on Computational Linguistics and Speech Processing, ROCLING 2023
Country/TerritoryTaiwan
CityTaipei City
Period2023/10/202023/10/21

Keywords

  • Knowledge Injection
  • Knowledge Retrieval
  • Monte Carlo Tree Search
  • Semantic Alignment

ASJC Scopus subject areas

  • Language and Linguistics
  • Speech and Hearing

Fingerprint

Dive into the research topics of 'KNOT-MCTS: An Effective Approach to Addressing Hallucinations in Generative Language Modeling for Question Answering'. Together they form a unique fingerprint.

Cite this