table of contents
Plucene::Analysis::LetterTokenizer(3pm) | User Contributed Perl Documentation | Plucene::Analysis::LetterTokenizer(3pm) |
NAME¶
Plucene::Analysis::LetterTokenizer - Letter tokenizerSYNOPSIS¶
# isa Plucene::Analysis::CharTokenizer
DESCRIPTION¶
This is the letter tokenizer class, which divides text at non-letters. Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces2011-08-14 | perl v5.12.4 |