Scroll to navigation

SLMSEG(1) User Contributed Perl Documentation SLMSEG(1)


slmseg - maximum matching segment Chinese text.


slmseg -d dict_file [option]... [corpus_file]...


slmseg is a tool for segmenting Chinese text into words using maximum matching algorithm. slmseg segments corpus_file, or standard input if no filename is specified, and write the segmented result to standard output.


Use dict_file as lexicon. A default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8.
Output Format, can be 'text' or 'bin'. default 'bin'. Normally, in text mode, word text are output, while in binary mode, binary short integer of the word-ids are written to stdout.
Sentence token id. Default 10. It will be written to output in binary mode after every sentence.
Show Id info. Under text output format mode, attach id after known words. If under binary mode, print id(s) in text.


Under binary mode, consecutive id of 0 are merged into one 0. Under text mode, no space are inserted between unknown-words.


Originally written by Phill.Zhang <>. Currently maintained by Kov.Chai <>.


mmseg(1), ids2ngram (1).

2019-12-06 perl v5.30.0