Scroll to navigation

SLMSEG(1) User Contributed Perl Documentation SLMSEG(1)

NAME

slmseg - maximum matching segment Chinese text.

SYNOPSIS

slmseg -d dict_file [option]... [corpus_file]...

DESCRIPTION

slmseg is a tool for segmenting Chinese text into words using maximum matching algorithm. slmseg segments corpus_file, or standard input if no filename is specified, and write the segmented result to standard output.

OPTIONS

Use dict_file as lexicon. A default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8.
Output Format, can be 'text' or 'bin'. default 'bin'. Normally, in text mode, word text are output, while in binary mode, binary short integer of the word-ids are written to stdout.
Sentence token id. Default 10. It will be written to output in binary mode after every sentence.
Show Id info. Under text output format mode, attach id after known words. If under binary mode, print id(s) in text.

NOTES

Under binary mode, consecutive id of 0 are merged into one 0. Under text mode, no space are inserted between unknown-words.

AUTHOR

Originally written by Phill.Zhang <phill.zhang@sun.com>. Currently maintained by Kov.Chai <tchaikov@gmail.com>.

SEE ALSO

mmseg(1), ids2ngram (1).

2024-02-10 perl v5.38.2