.\" Automatically generated by Pod::Man 2.28 (Pod::Simple 3.28) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{ . if \nF \{ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "Lingua::Stem::Snowball 3pm" .TH Lingua::Stem::Snowball 3pm "2014-08-15" "perl v5.20.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Lingua::Stem::Snowball \- Perl interface to Snowball stemmers. .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 1 \& my @words = qw( horse hooves ); \& \& # OO interface: \& my $stemmer = Lingua::Stem::Snowball\->new( lang => \*(Aqen\*(Aq ); \& $stemmer\->stem_in_place( \e@words ); # qw( hors hoov ) \& \& # Functional interface: \& my @stems = stem( \*(Aqen\*(Aq, \e@words ); .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" Stemming reduces related words to a common root form \*(-- for instance, \*(L"horse\*(R", \&\*(L"horses\*(R", and \*(L"horsing\*(R" all become \*(L"hors\*(R". Most commonly, stemming is deployed as part of a search application, allowing searches for a given term to match documents which contain other forms of that term. .PP This module is very similar to Lingua::Stem \*(-- however, Lingua::Stem is pure Perl, while Lingua::Stem::Snowball is an \s-1XS\s0 module which provides a Perl interface to the C version of the Snowball stemmers. (). .SS "Supported Languages" .IX Subsection "Supported Languages" The following stemmers are available (as of Lingua::Stem::Snowball 0.95): .PP .Vb 10 \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| \& | Language | ISO code | default encoding | also available | \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| \& | Danish | da | ISO\-8859\-1 | UTF\-8 | \& | Dutch | nl | ISO\-8859\-1 | UTF\-8 | \& | English | en | ISO\-8859\-1 | UTF\-8 | \& | Finnish | fi | ISO\-8859\-1 | UTF\-8 | \& | French | fr | ISO\-8859\-1 | UTF\-8 | \& | German | de | ISO\-8859\-1 | UTF\-8 | \& | Hungarian | hu | ISO\-8859\-1 | UTF\-8 | \& | Italian | it | ISO\-8859\-1 | UTF\-8 | \& | Norwegian | no | ISO\-8859\-1 | UTF\-8 | \& | Portuguese | pt | ISO\-8859\-1 | UTF\-8 | \& | Romanian | ro | ISO\-8859\-2 | UTF\-8 | \& | Russian | ru | KOI8\-R | UTF\-8 | \& | Spanish | es | ISO\-8859\-1 | UTF\-8 | \& | Swedish | sv | ISO\-8859\-1 | UTF\-8 | \& | Turkish | tr | UTF\-8 | | \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| .Ve .SS "Benchmarks" .IX Subsection "Benchmarks" Here is a comparison of Lingua::Stem::Snowball and Lingua::Stem, using The Works of Edgar Allen Poe, volumes 1\-5 (via Project Gutenberg) as source material. It was produced on a 3.2GHz Pentium 4 running FreeBSD 5.3 and Perl 5.8.7. (The benchmarking script is included in this distribution: devel/benchmark_stemmers.plx.) .PP .Vb 10 \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| \& | total words: 454285 | unique words: 22748 | \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| \& | module | config | avg secs | rate | \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| \& | Lingua::Stem 0.81 | no cache | 2.029 | 223881 | \& | Lingua::Stem 0.81 | cache level 2 | 1.280 | 355025 | \& | Lingua::Stem::Snowball 0.94 | stem | 1.426 | 318636 | \& | Lingua::Stem::Snowball 0.94 | stem_in_place | 0.641 | 708495 | \& |\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-| .Ve .SH "METHODS / FUNCTIONS" .IX Header "METHODS / FUNCTIONS" .SS "new" .IX Subsection "new" .Vb 5 \& my $stemmer = Lingua::Stem::Snowball\->new( \& lang => \*(Aqes\*(Aq, \& encoding => \*(AqUTF\-8\*(Aq, \& ); \& die $@ if $@; .Ve .PP Create a Lingua::Stem::Snowball object. \fInew()\fR accepts the following hash style parameters: .IP "\(bu" 4 \&\fBlang\fR: An \s-1ISO\s0 code taken from the table of supported languages, above. .IP "\(bu" 4 \&\fBencoding\fR: A supported character encoding. .PP Be careful with the values you supply to \fInew()\fR. If \f(CW\*(C`lang\*(C'\fR is invalid, Lingua::Stem::Snowball does not throw an exception, but instead sets $@. Also, if you supply an invalid combination of values for \f(CW\*(C`lang\*(C'\fR and \&\f(CW\*(C`encoding\*(C'\fR, Lingua::Stem::Snowball will not warn you, but the behavior will change: \fIstem()\fR will always return undef, and \fIstem_in_place()\fR will be a no-op. .SS "stem" .IX Subsection "stem" .Vb 2 \& @stemmed = $stemmer\->stem( WORDS, [IS_STEMMED] ); \& @stemmed = stem( ISO_CODE, WORDS, [LOCALE], [IS_STEMMED] ); .Ve .PP Return lowercased and stemmed output. \s-1WORDS\s0 may be either an array of words or a single scalar word. .PP In a scalar context, \fIstem()\fR returns the first item in the array of stems: .PP .Vb 2 \& $stem = $stemmer\->stem($word); \& $first_stem = $stemmer\->stem(\e@words); # probably wrong .Ve .PP \&\s-1LOCALE\s0 has no effect; it is only there as a placeholder for backwards compatibility (see Changes). \s-1IS_STEMMED\s0 must be a reference to a scalar; if it is supplied, it will be set to 1 if the output differs from the input in some way, 0 otherwise. .SS "stem_in_place" .IX Subsection "stem_in_place" .Vb 1 \& $stemmer\->stem_in_place(\e@words); .Ve .PP This is a high-performance, streamlined version of \fIstem()\fR (in fact, \fIstem()\fR calls \fIstem_in_place()\fR internally). It has no return value, instead modifying each item in an existing array of words. The words must already be in lower case. .SS "lang" .IX Subsection "lang" .Vb 2 \& my $lang = $stemmer\->lang; \& $stemmer\->lang($iso_language_code); .Ve .PP Accessor/mutator for the lang parameter. If there is no stemmer for the supplied \s-1ISO\s0 code, the language is not changed (but $@ is set). .SS "encoding" .IX Subsection "encoding" .Vb 2 \& my $encoding = $stemmer\->encoding; \& $stemmer\->encoding($encoding); .Ve .PP Accessor/mutator for the encoding parameter. .SS "stemmers" .IX Subsection "stemmers" .Vb 2 \& my @iso_codes = stemmers(); \& my @iso_codes = $stemmer\->stemmers(); .Ve .PP Returns a list of all valid language codes. .SH "REQUESTS & BUGS" .IX Header "REQUESTS & BUGS" Please report any requests, suggestions or bugs via the \s-1RT\s0 bug-tracking system at http://rt.cpan.org/ or email to bug\-Lingua\-Stem\-Snowball@rt.cpan.org. .PP http://rt.cpan.org/NoAuth/Bugs.html?Dist=Lingua\-Stem\-Snowball is the \s-1RT\s0 queue for Lingua::Stem::Snowball. Please check to see if your bug has already been reported. .SH "AUTHORS" .IX Header "AUTHORS" Lingua::Stem::Snowball was originally developed to provide access to stemming algorithms for the OpenFTS (full text search engine) project (), by Oleg Bartunov, and Teodor Sigaev, . .PP Currently maintained by Marvin Humphrey . Previously maintained by Fabien Potencier . .SH "COPYRIGHT AND LICENSE" .IX Header "COPYRIGHT AND LICENSE" Perl bindings copyright 2004\-2008 by Marvin Humphrey, Fabien Potencier, Oleg Bartunov and Teodor Sigaev. .PP This software may be freely copied and distributed under the same terms and conditions as Perl. .PP Snowball files and stemmers are covered by the \s-1BSD\s0 license. .SH "SEE ALSO" .IX Header "SEE ALSO" , Lingua::Stem.