.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.40) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "Unicode::LineBreak 3pm" .TH Unicode::LineBreak 3pm "2020-11-09" "perl v5.32.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Unicode::LineBreak \- UAX #14 Unicode Line Breaking Algorithm .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 3 \& use Unicode::LineBreak; \& $lb = Unicode::LineBreak\->new(); \& $broken = $lb\->break($string); .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" Unicode::LineBreak performs Line Breaking Algorithm described in Unicode Standard Annex #14 [\s-1UAX\s0 #14]. East_Asian_Width informative property defined by Annex #11 [\s-1UAX\s0 #11] will be concerned to determine breaking positions. .SS "Terminology" .IX Subsection "Terminology" Following terms are used for convenience. .PP \&\fBMandatory break\fR is obligatory line breaking behavior defined by core rules and performed regardless of surrounding characters. \&\fBArbitrary break\fR is line breaking behavior allowed by core rules and chosen by user to perform it. Arbitrary break includes \fBdirect break\fR and \fBindirect break\fR defined by [\s-1UAX\s0 #14]. .PP \&\fBAlphabetic characters\fR are characters usually no line breaks are allowed between pairs of them, except that other characters provide break oppotunities. \&\fBIdeographic characters\fR are characters that usually allow line breaks both before and after themselves. [\s-1UAX\s0 #14] classifies most of alphabetic to \s-1AL\s0 and most of ideographic to \s-1ID\s0 (These terms are inaccurate from the point of view by grammatology). On several scripts, breaking positions are not obvious by each characters therefore heuristic based on dictionary is used. .PP \&\fBNumber of columns\fR of a string is not always equal to the number of characters it contains: Each of characters is either \fBwide\fR, \fBnarrow\fR or nonspacing; they occupy 2, 1 or 0 columns, respectively. Several characters may be both wide and narrow by the contexts they are used. Characters may have more various widths by customization. .SH "PUBLIC INTERFACE" .IX Header "PUBLIC INTERFACE" .SS "Line Breaking" .IX Subsection "Line Breaking" .IP "new ([\s-1KEY\s0 => \s-1VALUE, ...\s0])" 4 .IX Item "new ([KEY => VALUE, ...])" \&\fIConstructor\fR. About \s-1KEY\s0 => \s-1VALUE\s0 pairs see \*(L"Options\*(R". .IP "break (\s-1STRING\s0)" 4 .IX Item "break (STRING)" \&\fIInstance method\fR. Break Unicode string \s-1STRING\s0 and returns it. In array context, returns array of lines contained in the result. .IP "break_partial (\s-1STRING\s0)" 4 .IX Item "break_partial (STRING)" \&\fIInstance method\fR. Same as \fBbreak()\fR but accepts incremental inputs. Give \f(CW\*(C`undef\*(C'\fR as \s-1STRING\s0 argument to specify that input was completed. .IP "config (\s-1KEY\s0)" 4 .IX Item "config (KEY)" .PD 0 .IP "config (\s-1KEY\s0 => \s-1VALUE, ...\s0)" 4 .IX Item "config (KEY => VALUE, ...)" .PD \&\fIInstance method\fR. Get or update configuration. About \s-1KEY\s0 => \s-1VALUE\s0 pairs see \*(L"Options\*(R". .IP "copy" 4 .IX Item "copy" \&\fICopy constructor\fR. Create a copy of object instance. .SS "Getting Informations" .IX Subsection "Getting Informations" .IP "breakingRule (\s-1BEFORESTR, AFTERSTR\s0)" 4 .IX Item "breakingRule (BEFORESTR, AFTERSTR)" \&\fIInstance method\fR. Get possible line breaking behavior between strings \s-1BEFORESTR\s0 and \s-1AFTERSTR.\s0 See \*(L"Constants\*(R" for returned value. .Sp \&\fBNote\fR: This method gives just approximate description of line breaking behavior. Use \fBbreak()\fR and so on to wrap actual texts. .IP "context ([Charset => \s-1CHARSET\s0], [Language => \s-1LANGUAGE\s0])" 4 .IX Item "context ([Charset => CHARSET], [Language => LANGUAGE])" \&\fIFunction\fR. Get language/region context used by character set \s-1CHARSET\s0 or language \s-1LANGUAGE.\s0 .SS "Options" .IX Subsection "Options" \&\*(L"new\*(R" and \*(L"config\*(R" methods accept following pairs. Some of them affect number of columns ([\fBE\fR]), grapheme cluster segmentation ([\fBG\fR]) (see also Unicode::GCString) or line breaking behavior ([\fBL\fR]). .ie n .IP "BreakIndent => ""YES"" | ""NO""" 4 .el .IP "BreakIndent => \f(CW``YES''\fR | \f(CW``NO''\fR" 4 .IX Item "BreakIndent => ""YES"" | ""NO""" [\fBL\fR] Always allows break after SPACEs at beginning of line, a.k.a. indent. [\s-1UAX\s0 #14] does not take account of such usage of \s-1SPACE.\s0 Default is \f(CW"YES"\fR. .Sp \&\fBNote\fR: This option was introduced at release 1.011. .IP "CharMax => \s-1NUMBER\s0" 4 .IX Item "CharMax => NUMBER" [\fBL\fR] Possible maximum number of characters in one line, not counting trailing SPACEs and newline sequence. Note that number of characters generally doesn't represent length of line. Default is \f(CW998\fR. \&\f(CW0\fR means unlimited (as of release 2012.01). .IP "ColMin => \s-1NUMBER\s0" 4 .IX Item "ColMin => NUMBER" [\fBL\fR] Minimum number of columns which line broken arbitrarily may include, not counting trailing spaces and newline sequences. Default is \f(CW0\fR. .IP "ColMax => \s-1NUMBER\s0" 4 .IX Item "ColMax => NUMBER" [\fBL\fR] Maximum number of columns line may include not counting trailing spaces and newline sequence. In other words, maximum length of line. Default is \f(CW76\fR. .PP See also \*(L"Urgent\*(R" option and \*(L"User-Defined Breaking Behaviors\*(R". .ie n .IP "ComplexBreaking => ""YES"" | ""NO""" 4 .el .IP "ComplexBreaking => \f(CW``YES''\fR | \f(CW``NO''\fR" 4 .IX Item "ComplexBreaking => ""YES"" | ""NO""" [\fBL\fR] Performs heuristic breaking on South East Asian complex context. Default is, if word segmentation for South East Asian writing systems is enabled, \f(CW"YES"\fR. .IP "Context => \s-1CONTEXT\s0" 4 .IX Item "Context => CONTEXT" [\fBE\fR][\fBL\fR] Specify language/region context. Currently available contexts are \f(CW"EASTASIAN"\fR and \f(CW"NONEASTASIAN"\fR. Default context is \f(CW"NONEASTASIAN"\fR. .Sp In \f(CW"EASTASIAN"\fR context, characters with East_Asian_Width property ambiguous (A) are treated as \*(L"wide\*(R" and with Line Breaking Class \s-1AI\s0 as ideographic (\s-1ID\s0). .Sp In \f(CW"NONEASTASIAN"\fR context, characters with East_Asian_Width property ambiguous (A) are treated as \*(L"narrow\*(R" and with Line Breaking Class \s-1AI\s0 as alphabetic (\s-1AL\s0). .ie n .IP "EAWidth => ""["" \s-1ORD\s0 ""=>"" \s-1PROPERTY\s0 ""]""" 4 .el .IP "EAWidth => \f(CW[\fR \s-1ORD\s0 \f(CW=>\fR \s-1PROPERTY\s0 \f(CW]\fR" 4 .IX Item "EAWidth => [ ORD => PROPERTY ]" .PD 0 .ie n .IP "EAWidth => ""undef""" 4 .el .IP "EAWidth => \f(CWundef\fR" 4 .IX Item "EAWidth => undef" .PD [\fBE\fR] Tailor classification of East_Asian_Width property. \&\s-1ORD\s0 is \s-1UCS\s0 scalar value of character or array reference of them. \&\s-1PROPERTY\s0 is one of East_Asian_Width property values and extended values (See \*(L"Constants\*(R"). This option may be specified multiple times. If \f(CW\*(C`undef\*(C'\fR is specified, all tailoring assigned before will be canceled. .Sp By default, no tailorings are available. See also \*(L"Tailoring Character Properties\*(R". .IP "Format => \s-1METHOD\s0" 4 .IX Item "Format => METHOD" [\fBL\fR] Specify the method to format broken lines. .RS 4 .ie n .IP """SIMPLE""" 4 .el .IP "\f(CW``SIMPLE''\fR" 4 .IX Item """SIMPLE""" Default method. Just only insert newline at arbitrary breaking positions. .ie n .IP """NEWLINE""" 4 .el .IP "\f(CW``NEWLINE''\fR" 4 .IX Item """NEWLINE""" Insert or replace newline sequences with that specified by \*(L"Newline\*(R" option, remove SPACEs leading newline sequences or end-of-text. Then append newline at end of text if it does not exist. .ie n .IP """TRIM""" 4 .el .IP "\f(CW``TRIM''\fR" 4 .IX Item """TRIM""" Insert newline at arbitrary breaking positions. Remove SPACEs leading newline sequences. .ie n .IP """undef""" 4 .el .IP "\f(CWundef\fR" 4 .IX Item "undef" Do nothing, even inserting any newlines. .IP "Subroutine reference" 4 .IX Item "Subroutine reference" See \*(L"Formatting Lines\*(R". .RE .RS 4 .RE .ie n .IP "HangulAsAL => ""YES"" | ""NO""" 4 .el .IP "HangulAsAL => \f(CW``YES''\fR | \f(CW``NO''\fR" 4 .IX Item "HangulAsAL => ""YES"" | ""NO""" [\fBL\fR] Treat hangul syllables and conjoining jamos as alphabetic characters (\s-1AL\s0). Default is \f(CW"NO"\fR. .ie n .IP "LBClass => ""["" \s-1ORD\s0 ""=>"" \s-1CLASS\s0 ""]""" 4 .el .IP "LBClass => \f(CW[\fR \s-1ORD\s0 \f(CW=>\fR \s-1CLASS\s0 \f(CW]\fR" 4 .IX Item "LBClass => [ ORD => CLASS ]" .PD 0 .ie n .IP "LBClass => ""undef""" 4 .el .IP "LBClass => \f(CWundef\fR" 4 .IX Item "LBClass => undef" .PD [\fBG\fR][\fBL\fR] Tailor classification of line breaking property. \&\s-1ORD\s0 is \s-1UCS\s0 scalar value of character or array reference of them. \&\s-1CLASS\s0 is one of line breaking classes (See \*(L"Constants\*(R"). This option may be specified multiple times. If \f(CW\*(C`undef\*(C'\fR is specified, all tailoring assigned before will be canceled. .Sp By default, no tailorings are available. See also \*(L"Tailoring Character Properties\*(R". .ie n .IP "LegacyCM => ""YES"" | ""NO""" 4 .el .IP "LegacyCM => \f(CW``YES''\fR | \f(CW``NO''\fR" 4 .IX Item "LegacyCM => ""YES"" | ""NO""" [\fBG\fR][\fBL\fR] Treat combining characters lead by a \s-1SPACE\s0 as an isolated combining character (\s-1ID\s0). As of Unicode 5.0, such use of \s-1SPACE\s0 is not recommended. Default is \f(CW"YES"\fR. .IP "Newline => \s-1STRING\s0" 4 .IX Item "Newline => STRING" [\fBL\fR] Unicode string to be used for newline sequence. Default is \f(CW"\en"\fR. .IP "Prep => \s-1METHOD\s0" 4 .IX Item "Prep => METHOD" [\fBL\fR] Add user-defined line breaking behavior(s). This option may be specified multiple times. Following methods are available. .RS 4 .ie n .IP """NONBREAKURI""" 4 .el .IP "\f(CW``NONBREAKURI''\fR" 4 .IX Item """NONBREAKURI""" Won't break URIs. .ie n .IP """BREAKURI""" 4 .el .IP "\f(CW``BREAKURI''\fR" 4 .IX Item """BREAKURI""" Break URIs according to a rule suitable for printed materials. For more details see [\s-1CMOS\s0], sections 6.17 and 17.11. .ie n .IP """["" \s-1REGEX, SUBREF\s0 ""]""" 4 .el .IP "\f(CW[\fR \s-1REGEX, SUBREF\s0 \f(CW]\fR" 4 .IX Item "[ REGEX, SUBREF ]" The sequences matching regular expression \s-1REGEX\s0 will be broken by subroutine referred by \s-1SUBREF.\s0 For more details see \*(L"User-Defined Breaking Behaviors\*(R". .ie n .IP """undef""" 4 .el .IP "\f(CWundef\fR" 4 .IX Item "undef" Cancel all methods assigned before. .RE .RS 4 .RE .IP "Sizing => \s-1METHOD\s0" 4 .IX Item "Sizing => METHOD" [\fBL\fR] Specify method to calculate size of string. Following options are available. .RS 4 .ie n .IP """UAX11""" 4 .el .IP "\f(CW``UAX11''\fR" 4 .IX Item """UAX11""" Default method. Sizes are computed by columns of each characters according to built-in character database. .ie n .IP """undef""" 4 .el .IP "\f(CWundef\fR" 4 .IX Item "undef" Number of grapheme clusters (see Unicode::GCString) contained in the string. .IP "Subroutine reference" 4 .IX Item "Subroutine reference" See \*(L"Calculating String Size\*(R". .RE .RS 4 .Sp See also \*(L"ColMax\*(R", \*(L"ColMin\*(R" and \*(L"EAWidth\*(R" options. .RE .IP "Urgent => \s-1METHOD\s0" 4 .IX Item "Urgent => METHOD" [\fBL\fR] Specify method to handle excessing lines. Following options are available. .RS 4 .ie n .IP """CROAK""" 4 .el .IP "\f(CW``CROAK''\fR" 4 .IX Item """CROAK""" Print error message and die. .ie n .IP """FORCE""" 4 .el .IP "\f(CW``FORCE''\fR" 4 .IX Item """FORCE""" Force breaking excessing fragment. .ie n .IP """undef""" 4 .el .IP "\f(CWundef\fR" 4 .IX Item "undef" Default method. Won't break excessing fragment. .IP "Subroutine reference" 4 .IX Item "Subroutine reference" See \*(L"User-Defined Breaking Behaviors\*(R". .RE .RS 4 .RE .ie n .IP "ViramaAsJoiner => ""YES"" | ""NO""" 4 .el .IP "ViramaAsJoiner => \f(CW``YES''\fR | \f(CW``NO''\fR" 4 .IX Item "ViramaAsJoiner => ""YES"" | ""NO""" [\fBG\fR] Virama sign (\*(L"halant\*(R" in Hindi, \*(L"coeng\*(R" in Khmer) and its succeeding letter are not broken. Default is \f(CW"YES"\fR. \&\fBNote\fR: This option was introduced by release 2012.001_29. On previous releases, it was fixed to \f(CW"NO"\fR. \&\*(L"Default\*(R" grapheme cluster defined by [\s-1UAX\s0 #29] does not include this feature. .SS "Constants" .IX Subsection "Constants" .ie n .IP """EA_Na"", ""EA_N"", ""EA_A"", ""EA_W"", ""EA_H"", ""EA_F""" 4 .el .IP "\f(CWEA_Na\fR, \f(CWEA_N\fR, \f(CWEA_A\fR, \f(CWEA_W\fR, \f(CWEA_H\fR, \f(CWEA_F\fR" 4 .IX Item "EA_Na, EA_N, EA_A, EA_W, EA_H, EA_F" Index values to specify six East_Asian_Width property values defined by [\s-1UAX\s0 #11]: narrow (Na), neutral (N), ambiguous (A), wide (W), halfwidth (H) and fullwidth (F). .ie n .IP """EA_Z""" 4 .el .IP "\f(CWEA_Z\fR" 4 .IX Item "EA_Z" Index value to specify nonspacing characters. .Sp \&\fBNote\fR: This \*(L"nonspacing\*(R" value is extension by this module, not a part of [\s-1UAX\s0 #11]. .ie n .IP """LB_BK"", ""LB_CR"", ""LB_LF"", ""LB_NL"", ""LB_SP"", ""LB_OP"", ""LB_CL"", ""LB_CP"", ""LB_QU"", ""LB_GL"", ""LB_NS"", ""LB_EX"", ""LB_SY"", ""LB_IS"", ""LB_PR"", ""LB_PO"", ""LB_NU"", ""LB_AL"", ""LB_HL"", ""LB_ID"", ""LB_IN"", ""LB_HY"", ""LB_BA"", ""LB_BB"", ""LB_B2"", ""LB_CB"", ""LB_ZW"", ""LB_CM"", ""LB_WJ"", ""LB_H2"", ""LB_H3"", ""LB_JL"", ""LB_JV"", ""LB_JT"", ""LB_SG"", ""LB_AI"", ""LB_CJ"", ""LB_SA"", ""LB_XX"", ""LB_RI""" 4 .el .IP "\f(CWLB_BK\fR, \f(CWLB_CR\fR, \f(CWLB_LF\fR, \f(CWLB_NL\fR, \f(CWLB_SP\fR, \f(CWLB_OP\fR, \f(CWLB_CL\fR, \f(CWLB_CP\fR, \f(CWLB_QU\fR, \f(CWLB_GL\fR, \f(CWLB_NS\fR, \f(CWLB_EX\fR, \f(CWLB_SY\fR, \f(CWLB_IS\fR, \f(CWLB_PR\fR, \f(CWLB_PO\fR, \f(CWLB_NU\fR, \f(CWLB_AL\fR, \f(CWLB_HL\fR, \f(CWLB_ID\fR, \f(CWLB_IN\fR, \f(CWLB_HY\fR, \f(CWLB_BA\fR, \f(CWLB_BB\fR, \f(CWLB_B2\fR, \f(CWLB_CB\fR, \f(CWLB_ZW\fR, \f(CWLB_CM\fR, \f(CWLB_WJ\fR, \f(CWLB_H2\fR, \f(CWLB_H3\fR, \f(CWLB_JL\fR, \f(CWLB_JV\fR, \f(CWLB_JT\fR, \f(CWLB_SG\fR, \f(CWLB_AI\fR, \f(CWLB_CJ\fR, \f(CWLB_SA\fR, \f(CWLB_XX\fR, \f(CWLB_RI\fR" 4 .IX Item "LB_BK, LB_CR, LB_LF, LB_NL, LB_SP, LB_OP, LB_CL, LB_CP, LB_QU, LB_GL, LB_NS, LB_EX, LB_SY, LB_IS, LB_PR, LB_PO, LB_NU, LB_AL, LB_HL, LB_ID, LB_IN, LB_HY, LB_BA, LB_BB, LB_B2, LB_CB, LB_ZW, LB_CM, LB_WJ, LB_H2, LB_H3, LB_JL, LB_JV, LB_JT, LB_SG, LB_AI, LB_CJ, LB_SA, LB_XX, LB_RI" Index values to specify 40 line breaking property values (classes) defined by [\s-1UAX\s0 #14]. .Sp \&\fBNote\fR: Property value \s-1CP\s0 was introduced by Unicode 5.2.0. Property values \s-1HL\s0 and \s-1CJ\s0 were introduced by Unicode 6.1.0. Property value \s-1RI\s0 was introduced by Unicode 6.2.0. .ie n .IP """MANDATORY"", ""DIRECT"", ""INDIRECT"", ""PROHIBITED""" 4 .el .IP "\f(CWMANDATORY\fR, \f(CWDIRECT\fR, \f(CWINDIRECT\fR, \f(CWPROHIBITED\fR" 4 .IX Item "MANDATORY, DIRECT, INDIRECT, PROHIBITED" Four values to specify line breaking behaviors: Mandatory break; Both direct break and indirect break are allowed; Indirect break is allowed but direct break is prohibited; Prohibited break. .ie n .IP """Unicode::LineBreak::SouthEastAsian::supported""" 4 .el .IP "\f(CWUnicode::LineBreak::SouthEastAsian::supported\fR" 4 .IX Item "Unicode::LineBreak::SouthEastAsian::supported" Flag to determin if word segmentation for South East Asian writing systems is enabled. If this feature was enabled, a non-empty string is set. Otherwise, \f(CW\*(C`undef\*(C'\fR is set. .Sp \&\fBN.B.\fR: Current release supports Thai script of modern Thai language only. .ie n .IP """UNICODE_VERSION""" 4 .el .IP "\f(CWUNICODE_VERSION\fR" 4 .IX Item "UNICODE_VERSION" A string to specify version of Unicode standard this module refers. .SH "CUSTOMIZATION" .IX Header "CUSTOMIZATION" .SS "Formatting Lines" .IX Subsection "Formatting Lines" If you specify subroutine reference as a value of \*(L"Format\*(R" option, it should accept three arguments: .PP .Vb 1 \& $MODIFIED = &subroutine(SELF, EVENT, STR); .Ve .PP \&\s-1SELF\s0 is a Unicode::LineBreak object, \&\s-1EVENT\s0 is a string to determine the context that subroutine was called in, and \s-1STR\s0 is a fragment of Unicode string leading or trailing breaking position. .PP .Vb 12 \& EVENT |When Fired |Value of STR \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \& "sot" |Beginning of text |Fragment of first line \& "sop" |After mandatory break|Fragment of next line \& "sol" |After arbitrary break|Fragment on sequel of line \& "" |Just before any |Complete line without trailing \& |breaks |SPACEs \& "eol" |Arbitrary break |SPACEs leading breaking position \& "eop" |Mandatory break |Newline and its leading SPACEs \& "eot" |End of text |SPACEs (and newline) at end of \& | |text \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .Ve .PP Subroutine should return modified text fragment or may return \&\f(CW\*(C`undef\*(C'\fR to express that no modification occurred. Note that modification in the context of \f(CW"sot"\fR, \f(CW"sop"\fR or \f(CW"sol"\fR may affect decision of successive breaking positions while in the others won't. .PP \&\fBNote\fR: String arguments are actually sequences of grapheme clusters. See Unicode::GCString. .PP For example, following code folds lines removing trailing spaces: .PP .Vb 8 \& sub fmt { \& if ($_[1] =~ /^eo/) { \& return "\en"; \& } \& return undef; \& } \& my $lb = Unicode::LineBreak\->new(Format => \e&fmt); \& $output = $lb\->break($text); .Ve .SS "User-Defined Breaking Behaviors" .IX Subsection "User-Defined Breaking Behaviors" When a line generated by arbitrary break is expected to be beyond measure of either CharMax, ColMax or ColMin, \fBurgent break\fR may be performed on successive string. If you specify subroutine reference as a value of \*(L"Urgent\*(R" option, it should accept two arguments: .PP .Vb 1 \& @BROKEN = &subroutine(SELF, STR); .Ve .PP \&\s-1SELF\s0 is a Unicode::LineBreak object and \s-1STR\s0 is a Unicode string to be broken. .PP Subroutine should return an array of broken string \s-1STR.\s0 .PP \&\fBNote\fR: String argument is actually a sequence of grapheme clusters. See Unicode::GCString. .PP For example, following code inserts hyphen to the name of several chemical substances (such as Titin) so that it may be folded: .PP .Vb 5 \& sub hyphenize { \& return map {$_ =~ s/yl$/yl\-/; $_} split /(\ew+?yl(?=\ew))/, $_[1]; \& } \& my $lb = Unicode::LineBreak\->new(Urgent => \e&hyphenize); \& $output = $lb\->break("Methionylthreonylthreonylglutaminylarginyl..."); .Ve .PP If you specify [\s-1REGEX, SUBREF\s0] array reference as any of \*(L"Prep\*(R" option, subroutine should accept two arguments: .PP .Vb 1 \& @BROKEN = &subroutine(SELF, STR); .Ve .PP \&\s-1SELF\s0 is a Unicode::LineBreak object and \&\s-1STR\s0 is a Unicode string matched with \s-1REGEX.\s0 .PP Subroutine should return an array of broken string \s-1STR.\s0 .PP For example, following code will break \s-1HTTP\s0 URLs using [\s-1CMOS\s0] rule. .PP .Vb 10 \& my $url = qr{http://[\ex21\-\ex7E]+}i; \& sub breakurl { \& my $self = shift; \& my $str = shift; \& return split m{(?<=[/]) (?=[^/]) | \& (?<=[^\-.]) (?=[\-~.,_?\e#%=&]) | \& (?<=[=&]) (?=.)}x, $str; \& } \& my $lb = Unicode::LineBreak\->new(Prep => [$url, \e&breakurl]); \& $output = $lb\->break($string); .Ve .PP \fIPreserving State\fR .IX Subsection "Preserving State" .PP Unicode::LineBreak object can behave as hash reference. Any items may be preserved throughout its life. .PP For example, following code will separate paragraphs with empty lines. .PP .Vb 4 \& sub paraformat { \& my $self = shift; \& my $action = shift; \& my $str = shift; \& \& if ($action eq \*(Aqsot\*(Aq or $action eq \*(Aqsop\*(Aq) { \& $self\->{\*(Aqline\*(Aq} = \*(Aq\*(Aq; \& } elsif ($action eq \*(Aq\*(Aq) { \& $self\->{\*(Aqline\*(Aq} = $str; \& } elsif ($action eq \*(Aqeol\*(Aq) { \& return "\en"; \& } elsif ($action eq \*(Aqeop\*(Aq) { \& if (length $self\->{\*(Aqline\*(Aq}) { \& return "\en\en"; \& } else { \& return "\en"; \& } \& } elsif ($action eq \*(Aqeot\*(Aq) { \& return "\en"; \& } \& return undef; \& } \& my $lb = Unicode::LineBreak\->new(Format => \e¶format); \& $output = $lb\->break($string); .Ve .SS "Calculating String Size" .IX Subsection "Calculating String Size" If you specify subroutine reference as a value of \*(L"Sizing\*(R" option, it will be called with five arguments: .PP .Vb 1 \& $COLS = &subroutine(SELF, LEN, PRE, SPC, STR); .Ve .PP \&\s-1SELF\s0 is a Unicode::LineBreak object, \s-1LEN\s0 is size of preceding string, \&\s-1PRE\s0 is preceding Unicode string, \s-1SPC\s0 is additional SPACEs and \s-1STR\s0 is a Unicode string to be processed. .PP Subroutine should return calculated number of columns of \f(CW\*(C`PRE.SPC.STR\*(C'\fR. The number of columns may not be an integer: Unit of the number may be freely chosen, however, it should be same as those of \*(L"ColMin\*(R" and \*(L"ColMax\*(R" option. .PP \&\fBNote\fR: String arguments are actually sequences of grapheme clusters. See Unicode::GCString. .PP For example, following code processes lines with tab stops by each eight columns. .PP .Vb 2 \& sub tabbedsizing { \& my ($self, $cols, $pre, $spc, $str) = @_; \& \& my $spcstr = $spc.$str; \& while ($spcstr\->lbc == LB_SP) { \& my $c = $spcstr\->item(0); \& if ($c eq "\et") { \& $cols += 8 \- $cols % 8; \& } else { \& $cols += $c\->columns; \& } \& $spcstr = $spcstr\->substr(1); \& } \& $cols += $spcstr\->columns; \& return $cols; \& }; \& my $lb = Unicode::LineBreak\->new(LBClass => [ord("\et") => LB_SP], \& Sizing => \e&tabbedsizing); \& $output = $lb\->break($string); .Ve .SS "Tailoring Character Properties" .IX Subsection "Tailoring Character Properties" Character properties may be tailored by \*(L"LBClass\*(R" and \*(L"EAWidth\*(R" options. Some constants are defined for convenience of tailoring. .PP \fILine Breaking Properties\fR .IX Subsection "Line Breaking Properties" .PP Non-starters of Kana-like Characters .IX Subsection "Non-starters of Kana-like Characters" .PP By default, several hiragana, katakana and characters corresponding to kana are treated as non-starters (\s-1NS\s0 or \s-1CJ\s0). When the following pair(s) are specified for value of \*(L"LBClass\*(R" option, these characters are treated as normal ideographic characters (\s-1ID\s0). .ie n .IP """KANA_NONSTARTERS() => LB_ID""" 4 .el .IP "\f(CWKANA_NONSTARTERS() => LB_ID\fR" 4 .IX Item "KANA_NONSTARTERS() => LB_ID" All of characters below. .ie n .IP """IDEOGRAPHIC_ITERATION_MARKS() => LB_ID""" 4 .el .IP "\f(CWIDEOGRAPHIC_ITERATION_MARKS() => LB_ID\fR" 4 .IX Item "IDEOGRAPHIC_ITERATION_MARKS() => LB_ID" Ideographic iteration marks. U+3005 \s-1IDEOGRAPHIC ITERATION MARK, U+303B VERTICAL IDEOGRAPHIC ITERATION MARK, U+309D HIRAGANA ITERATION MARK, U+309E HIRAGANA VOICED ITERATION MARK, U+30FD KATAKANA ITERATION MARK\s0 and U+30FE \s-1KATAKANA VOICED ITERATION MARK.\s0 .Sp N.B. Some of them are neither hiragana nor katakana. .ie n .IP """KANA_SMALL_LETTERS() => LB_ID""" 4 .el .IP "\f(CWKANA_SMALL_LETTERS() => LB_ID\fR" 4 .IX Item "KANA_SMALL_LETTERS() => LB_ID" .PD 0 .ie n .IP """KANA_PROLONGED_SOUND_MARKS() => LB_ID""" 4 .el .IP "\f(CWKANA_PROLONGED_SOUND_MARKS() => LB_ID\fR" 4 .IX Item "KANA_PROLONGED_SOUND_MARKS() => LB_ID" .PD Hiragana or katakana small letters: Hiragana small letters U+3041 A, U+3043 I, U+3045 U, U+3047 E, U+3049 O, U+3063 \s-1TU, U+3083 YA, U+3085 YU, U+3087 YO, U+308E WA, U+3095 KA, U+3096 KE.\s0 Katakana small letters U+30A1 A, U+30A3 I, U+30A5 U, U+30A7 E, U+30A9 O, U+30C3 \s-1TU, U+30E3 YA, U+30E5 YU, U+30E7 YO, U+30EE WA, U+30F5 KA, U+30F6 KE.\s0 Katakana phonetic extensions U+31F0 \s-1KU\s0 \- U+31FF \s-1RO.\s0 Halfwidth katakana small letters U+FF67 A \- U+FF6F \s-1TU.\s0 .Sp Hiragana or katakana prolonged sound marks: U+30FC KATAKANA-HIRAGANA \s-1PROLONGED SOUND MARK\s0 and U+FF70 \s-1HALFWIDTH\s0 KATAKANA-HIRAGANA \s-1PROLONGED SOUND MARK.\s0 .Sp N.B. These letters are optionally treated either as non-starter or as normal ideographic. See [\s-1JIS X 4051\s0] 6.1.1, [\s-1JLREQ\s0] 3.1.7 or [\s-1UAX14\s0]. .Sp N.B. U+3095, U+3096, U+30F5, U+30F6 are considered to be neither hiragana nor katakana. .ie n .IP """MASU_MARK() => LB_ID""" 4 .el .IP "\f(CWMASU_MARK() => LB_ID\fR" 4 .IX Item "MASU_MARK() => LB_ID" U+303C \s-1MASU MARK.\s0 .Sp N.B. Although this character is not kana, it is usually regarded as abbreviation to sequence of hiragana ま す or katakana マ ス, \s-1MA\s0 and \s-1SU.\s0 .Sp N.B. This character is classified as non-starter (\s-1NS\s0) by [\s-1UAX\s0 #14] and as the class corresponding to \s-1ID\s0 by [\s-1JIS X 4051\s0] and [\s-1JLREQ\s0]. .PP Ambiguous Quotation Marks .IX Subsection "Ambiguous Quotation Marks" .PP By default, some punctuations are ambiguous quotation marks (\s-1QU\s0). .ie n .IP """BACKWARD_QUOTES() => LB_OP, FORWARD_QUOTES() => LB_CL""" 4 .el .IP "\f(CWBACKWARD_QUOTES() => LB_OP, FORWARD_QUOTES() => LB_CL\fR" 4 .IX Item "BACKWARD_QUOTES() => LB_OP, FORWARD_QUOTES() => LB_CL" Some languages (Dutch, English, Italian, Portuguese, Spanish, Turkish and most East Asian) use rotated\-9\-style punctuations (‘ “) as opening and 9\-style punctuations (’ ”) as closing quotation marks. .ie n .IP """FORWARD_QUOTES() => LB_OP, BACKWARD_QUOTES() => LB_CL""" 4 .el .IP "\f(CWFORWARD_QUOTES() => LB_OP, BACKWARD_QUOTES() => LB_CL\fR" 4 .IX Item "FORWARD_QUOTES() => LB_OP, BACKWARD_QUOTES() => LB_CL" Some others (Czech, German and Slovak) use 9\-style punctuations (’ ”) as opening and rotated\-9\-style punctuations (‘ “) as closing quotation marks. .ie n .IP """BACKWARD_GUILLEMETS() => LB_OP, FORWARD_GUILLEMETS() => LB_CL""" 4 .el .IP "\f(CWBACKWARD_GUILLEMETS() => LB_OP, FORWARD_GUILLEMETS() => LB_CL\fR" 4 .IX Item "BACKWARD_GUILLEMETS() => LB_OP, FORWARD_GUILLEMETS() => LB_CL" French, Greek, Russian etc. use left-pointing guillemets (« ‹) as opening and right-pointing guillemets (» ›) as closing quotation marks. .ie n .IP """FORWARD_GUILLEMETS() => LB_OP, BACKWARD_GUILLEMETS() => LB_CL""" 4 .el .IP "\f(CWFORWARD_GUILLEMETS() => LB_OP, BACKWARD_GUILLEMETS() => LB_CL\fR" 4 .IX Item "FORWARD_GUILLEMETS() => LB_OP, BACKWARD_GUILLEMETS() => LB_CL" German and Slovak use right-pointing guillemets (» ›) as opening and left-pointing guillemets (« ‹) as closing quotation marks. .PP Danish, Finnish, Norwegian and Swedish use 9\-style or right-pointing punctuations (’ ” » ›) as both opening and closing quotation marks. .PP \s-1IDEOGRAPHIC SPACE\s0 .IX Subsection "IDEOGRAPHIC SPACE" .ie n .IP """IDEOGRAPHIC_SPACE() => LB_BA""" 4 .el .IP "\f(CWIDEOGRAPHIC_SPACE() => LB_BA\fR" 4 .IX Item "IDEOGRAPHIC_SPACE() => LB_BA" U+3000 \s-1IDEOGRAPHIC SPACE\s0 won't be placed at beginning of line. This is default behavior. .ie n .IP """IDEOGRAPHIC_SPACE() => LB_ID""" 4 .el .IP "\f(CWIDEOGRAPHIC_SPACE() => LB_ID\fR" 4 .IX Item "IDEOGRAPHIC_SPACE() => LB_ID" \&\s-1IDEOGRAPHIC SPACE\s0 can be placed at beginning of line. This was default behavior by Unicode 6.2 and earlier. .ie n .IP """IDEOGRAPHIC_SPACE() => LB_SP""" 4 .el .IP "\f(CWIDEOGRAPHIC_SPACE() => LB_SP\fR" 4 .IX Item "IDEOGRAPHIC_SPACE() => LB_SP" \&\s-1IDEOGRAPHIC SPACE\s0 won't be placed at beginning of line, and will protrude from end of line. .PP \fIEast_Asian_Width Properties\fR .IX Subsection "East_Asian_Width Properties" .PP Some particular letters of Latin, Greek and Cyrillic scripts have ambiguous (A) East_Asian_Width property. Thus, these characters are treated as wide in \f(CW"EASTASIAN"\fR context. Specifying \f(CW\*(C`EAWidth => [ AMBIGUOUS_\*(C'\fR*\f(CW\*(C`() => EA_N ]\*(C'\fR, those characters are always treated as narrow. .ie n .IP """AMBIGUOUS_ALPHABETICS() => EA_N""" 4 .el .IP "\f(CWAMBIGUOUS_ALPHABETICS() => EA_N\fR" 4 .IX Item "AMBIGUOUS_ALPHABETICS() => EA_N" Treat all of characters below as East_Asian_Width neutral (N). .ie n .IP """AMBIGUOUS_CYRILLIC() => EA_N""" 4 .el .IP "\f(CWAMBIGUOUS_CYRILLIC() => EA_N\fR" 4 .IX Item "AMBIGUOUS_CYRILLIC() => EA_N" .PD 0 .ie n .IP """AMBIGUOUS_GREEK() => EA_N""" 4 .el .IP "\f(CWAMBIGUOUS_GREEK() => EA_N\fR" 4 .IX Item "AMBIGUOUS_GREEK() => EA_N" .ie n .IP """AMBIGUOUS_LATIN() => EA_N""" 4 .el .IP "\f(CWAMBIGUOUS_LATIN() => EA_N\fR" 4 .IX Item "AMBIGUOUS_LATIN() => EA_N" .PD Treate letters having ambiguous (A) width of Cyrillic, Greek and Latin scripts as neutral (N). .PP On the other hand, despite several characters were occasionally rendered as wide characters by number of implementations for East Asian character sets, they are given narrow (Na) East_Asian_Width property just because they have fullwidth (F) compatibility characters. Specifying \f(CW\*(C`EAWidth\*(C'\fR as below, those characters are treated as ambiguous \&\-\-\- wide on \f(CW"EASTASIAN"\fR context. .ie n .IP """QUESTIONABLE_NARROW_SIGNS() => EA_A""" 4 .el .IP "\f(CWQUESTIONABLE_NARROW_SIGNS() => EA_A\fR" 4 .IX Item "QUESTIONABLE_NARROW_SIGNS() => EA_A" U+00A2 \s-1CENT SIGN, U+00A3 POUND SIGN, U+00A5 YEN SIGN\s0 (or yuan sign), U+00A6 \s-1BROKEN BAR, U+00AC NOT SIGN, U+00AF MACRON.\s0 .SS "Configuration File" .IX Subsection "Configuration File" Built-in defaults of option parameters for \*(L"new\*(R" and \*(L"config\*(R" method can be overridden by configuration files: \&\fIUnicode/LineBreak/Defaults.pm\fR. For more details read \fIUnicode/LineBreak/Defaults.pm.sample\fR. .SH "BUGS" .IX Header "BUGS" Please report bugs or buggy behaviors to developer. .PP \&\s-1CPAN\s0 Request Tracker: . .SH "VERSION" .IX Header "VERSION" Consult \f(CW$VERSION\fR variable. .SS "Incompatible Changes" .IX Subsection "Incompatible Changes" .IP "Release 2012.06" 4 .IX Item "Release 2012.06" .RS 4 .PD 0 .IP "\(bu" 4 .PD \&\fBeawidth()\fR method was deprecated. \&\*(L"columns\*(R" in Unicode::GCString may be used instead. .IP "\(bu" 4 \&\fBlbclass()\fR method was deprecated. Use \*(L"lbc\*(R" in Unicode::GCString or \*(L"lbcext\*(R" in Unicode::GCString. .RE .RS 4 .RE .SS "Conformance to Standards" .IX Subsection "Conformance to Standards" Character properties this module is based on are defined by Unicode Standard version 8.0.0. .PP This module is intended to implement \s-1UAX14\-C2.\s0 .SH "IMPLEMENTATION NOTES" .IX Header "IMPLEMENTATION NOTES" .IP "\(bu" 4 Some ideographic characters may be treated either as \s-1NS\s0 or as \s-1ID\s0 by choice. .IP "\(bu" 4 Hangul syllables and conjoining jamos may be treated as either \s-1ID\s0 or \s-1AL\s0 by choice. .IP "\(bu" 4 Characters assigned to \s-1AI\s0 may be resolved to either \s-1AL\s0 or \s-1ID\s0 by choice. .IP "\(bu" 4 Character(s) assigned to \s-1CB\s0 are not resolved. .IP "\(bu" 4 Characters assigned to \s-1CJ\s0 are always resolved to \s-1NS.\s0 More flexible tailoring mechanism is provided. .IP "\(bu" 4 When word segmentation for South East Asian writing systems is not supported, characters assigned to \s-1SA\s0 are resolved to \s-1AL,\s0 except that characters that have Grapheme_Cluster_Break property value Extend or SpacingMark be resolved to \s-1CM.\s0 .IP "\(bu" 4 Characters assigned to \s-1SG\s0 or \s-1XX\s0 are resolved to \s-1AL.\s0 .IP "\(bu" 4 Code points of following \s-1UCS\s0 ranges are given fixed property values even if they have not been assigned any characters. .Sp .Vb 10 \& Ranges | UAX #14 | UAX #11 | Description \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \& U+20A0..U+20CF | PR [*1] | N [*2] | Currency symbols \& U+3400..U+4DBF | ID | W | CJK ideographs \& U+4E00..U+9FFF | ID | W | CJK ideographs \& U+D800..U+DFFF | AL (SG) | N | Surrogates \& U+E000..U+F8FF | AL (XX) | F or N (A) | Private use \& U+F900..U+FAFF | ID | W | CJK ideographs \& U+20000..U+2FFFD | ID | W | CJK ideographs \& U+30000..U+3FFFD | ID | W | Old hanzi \& U+F0000..U+FFFFD | AL (XX) | F or N (A) | Private use \& U+100000..U+10FFFD | AL (XX) | F or N (A) | Private use \& Other unassigned | AL (XX) | N | Unassigned, \& | | | reserved or \& | | | noncharacters \& \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \& [*1] Except U+20A7 PESETA SIGN (PO), \& U+20B6 LIVRE TOURNOIS SIGN (PO), U+20BB NORDIC MARK SIGN (PO) \& and U+20BE LARI SIGN (PO). \& [*2] Except U+20A9 WON SIGN (H) and U+20AC EURO SIGN \& (F or N (A)). .Ve .IP "\(bu" 4 Characters belonging to General Category Mn, Me, Cc, Cf, Zl or Zp are treated as nonspacing by this module. .SH "REFERENCES" .IX Header "REFERENCES" .IP "[\s-1CMOS\s0]" 4 .IX Item "[CMOS]" \&\fIThe Chicago Manual of Style\fR, 15th edition. University of Chicago Press, 2003. .IP "[\s-1JIS X 4051\s0]" 4 .IX Item "[JIS X 4051]" \&\s-1JIS X 4051:2004\s0 \&\fI日本語文書の組版方法\fR (\fIFormatting Rules for Japanese Documents\fR). Japanese Standards Association, 2004. .IP "[\s-1JLREQ\s0]" 4 .IX Item "[JLREQ]" Anan, Yasuhiro et al. \&\fIRequirements for Japanese Text Layout\fR, W3C Working Group Note 3 April 2012. . .IP "[\s-1UAX\s0 #11]" 4 .IX Item "[UAX #11]" A. Freytag (ed.) (2008\-2009). \&\fIUnicode Standard Annex #11: East Asian Width\fR, Revisions 17\-19. . .IP "[\s-1UAX\s0 #14]" 4 .IX Item "[UAX #14]" A. Freytag and A. Heninger (eds.) (2008\-2015). \&\fIUnicode Standard Annex #14: Unicode Line Breaking Algorithm\fR, Revisions 22\-35. . .IP "[\s-1UAX\s0 #29]" 4 .IX Item "[UAX #29]" Mark Davis (ed.) (2009\-2013). \&\fIUnicode Standard Annex #29: Unicode Text Segmentation\fR, Revisions 15\-23. . .SH "SEE ALSO" .IX Header "SEE ALSO" Text::LineFold, Text::Wrap, Unicode::GCString. .SH "AUTHOR" .IX Header "AUTHOR" Copyright (C) 2009\-2018 Hatuka*nezumi \- \s-1IKEDA\s0 Soji . .PP This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.