Abstract
This paper reports an approach to automatic thesaurus construction for Chinese news articles. An effective Chinese word segmentation and keyword extraction algorithm is first presented. For each document, an average of 33% keywords unknown to a lexicon of 123,226 terms can be identified. The extraction error rate is 3.6%. Keywords extracted from each document are then further filtered for term association analysis by a modified Dice coefficient formula. Association weights larger than a threshold are then accumulated over all the documents to yield the final term pair similarities. Compared to previous studies, this method not only speeds up the thesaurus generation process drastically, but also achieves a similar percentage level of term relatedness.
| Original language | English |
|---|---|
| Pages (from-to) | 853-858 |
| Number of pages | 6 |
| Journal | Proceedings of the IEEE International Conference on Systems, Man and Cybernetics |
| Volume | 2 |
| Publication status | Published - 2001 Dec 1 |
| Event | 2001 IEEE International Conference on Systems, Man and Cybernetics - Tucson, AZ, United States Duration: 2001 Oct 7 → 2001 Oct 10 |
Keywords
- Chinese
- Co-occurrence analysis
- Co-occurrence thesaurus
- Unknown word identification
- Word segmentation
ASJC Scopus subject areas
- Control and Systems Engineering
- Hardware and Architecture