.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.40)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "LaTeXML::Core::Token 3pm"
.TH LaTeXML::Core::Token 3pm "2021-05-20" "perl v5.32.1" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
"LaTeXML::Core::Token" \- representation of a Token:
a pair of character and category code (catcode);
It extends LaTeXML::Common::Object.
.SS "Exported functions"
.IX Subsection "Exported functions"
.ie n .IP """$catcode = CC_ESCAPE;""" 4
.el .IP "\f(CW$catcode = CC_ESCAPE;\fR" 4
.IX Item "$catcode = CC_ESCAPE;"
Constants for the category codes:
.Sp
.Vb 4
\&  CC_BEGIN, CC_END, CC_MATH, CC_ALIGN, CC_EOL,
\&  CC_PARAM, CC_SUPER, CC_SUB, CC_IGNORE,
\&  CC_SPACE, CC_LETTER, CC_OTHER, CC_ACTIVE,
\&  CC_COMMENT, CC_INVALID, CC_CS.
.Ve
.Sp
[The last 2 are (apparent) extensions,
with catcodes 16 and 17, respectively].
.ie n .IP """$token = Token($string,$cc);""" 4
.el .IP "\f(CW$token = Token($string,$cc);\fR" 4
.IX Item "$token = Token($string,$cc);"
Creates a LaTeXML::Core::Token with the given content and catcode.
The following shorthand versions are also exported for convenience:
.Sp
.Vb 4
\&  T_BEGIN, T_END, T_MATH, T_ALIGN, T_PARAM,
\&  T_SUB, T_SUPER, T_SPACE, T_LETTER($letter),
\&  T_OTHER($char), T_ACTIVE($char),
\&  T_COMMENT($comment), T_CS($cs)
.Ve
.ie n .IP """@tokens = Explode($string);""" 4
.el .IP "\f(CW@tokens = Explode($string);\fR" 4
.IX Item "@tokens = Explode($string);"
Returns a list of the tokens corresponding to the characters in \f(CW$string\fR.
All tokens have catcode \s-1CC_OTHER,\s0 except for spaces which have catcode \s-1CC_SPACE.\s0
.ie n .IP """@tokens = ExplodeText($string);""" 4
.el .IP "\f(CW@tokens = ExplodeText($string);\fR" 4
.IX Item "@tokens = ExplodeText($string);"
Returns a list of the tokens corresponding to the characters in \f(CW$string\fR.
All (roman) letters have catcode \s-1CC_LETTER,\s0 all others have catcode \s-1CC_OTHER,\s0
except for spaces which have catcode \s-1CC_SPACE.\s0
.ie n .IP """UnTeX($object);""" 4
.el .IP "\f(CWUnTeX($object);\fR" 4
.IX Item "UnTeX($object);"
Converts \f(CW$object\fR to a string containing TeX that created it (or could have).
Note that this is not necessarily the original TeX code; expansions
or other substitutions may have taken place.
.SS "Methods"
.IX Subsection "Methods"
.ie n .IP """@tokens = $object\->unlist;""" 4
.el .IP "\f(CW@tokens = $object\->unlist;\fR" 4
.IX Item "@tokens = $object->unlist;"
Return a list of the tokens making up this \f(CW$object\fR.
.ie n .IP """$string = $object\->toString;""" 4
.el .IP "\f(CW$string = $object\->toString;\fR" 4
.IX Item "$string = $object->toString;"
Return a string representing \f(CW$object\fR.
.ie n .IP """$string = $token\->getCSName;""" 4
.el .IP "\f(CW$string = $token\->getCSName;\fR" 4
.IX Item "$string = $token->getCSName;"
Return the string or character part of the \f(CW$token\fR; for the special category
codes, returns the standard string (eg. \f(CW\*(C`T_BEGIN\->getCSName\*(C'\fR returns \*(L"{\*(R").
.ie n .IP """$string = $token\->getString;""" 4
.el .IP "\f(CW$string = $token\->getString;\fR" 4
.IX Item "$string = $token->getString;"
Return the string or character part of the \f(CW$token\fR.
.ie n .IP """$code = $token\->getCharcode;""" 4
.el .IP "\f(CW$code = $token\->getCharcode;\fR" 4
.IX Item "$code = $token->getCharcode;"
Return the character code of the character part of the \f(CW$token\fR,
or 256 if it is a control sequence.
.ie n .IP """$code = $token\->getCatcode;""" 4
.el .IP "\f(CW$code = $token\->getCatcode;\fR" 4
.IX Item "$code = $token->getCatcode;"
Return the catcode of the \f(CW$token\fR.
.SH "AUTHOR"
.IX Header "AUTHOR"
pBruce Miller <bruce.miller@nist.gov>
.SH "COPYRIGHT"
.IX Header "COPYRIGHT"
Public domain software, produced as part of work done by the
United States Government & not subject to copyright in the \s-1US.\s0