Searching Documents with Composite Chinese Word Segmentations

• Main Authors: Chun-Wei Lin, 林俊偉
• Other Authors: Ping-Yu Hsu
• Format: Others
• Language: zh-TW
• Published: 2010
• Online Access: http://ndltd.ncl.edu.tw/handle/26152106905264225206

碩士 === 國立中央大學 === 企業管理研究所 === 98 === In natural language, “word” is the most basic element. Owing to an article is constituted by lots of words; we must separate those words apart first then go to research. Because there is no word-spacing in Chinese article, like the one as the boundary between every word in English article; therefore, we need to divide Chinese words through Word Segmentation from the very first beginning then go to further analysis. In Chinese Word Segmentation Process, CKIP in Academia Sinica is the most prominent. However, due to the design principle of CKIP has been constrained by the words only in the data, as for the words not included (named Unidentified Word) are hardly to sort out correct word segmentation result. The purpose of this research is to solve the problem of Unidentified Word: to raise the accuracy of Word Segmentation System for further Keyword, which is really able to be represented the whole article, Search Processing. The foundation of this research is the result from Chinese Word Segmentation System, and I tried to combine the result to piece up the complete lexicon and original meaning, compensating for the deficiency of undiscovered when Unidentified Word being separated. Afterward I snatched further Keyword through the original word segmentation result and combined lexicon. The results showed that precision and recall of composite Chinese word segmentation are much better than CKIP and Google, it also verify the validity of the system.