NAME¶
"LaTeXML::Token" - representation of a token, and
"LaTeXML::Tokens", representing lists of tokens.
DESCRIPTION¶
This module defines Tokens ("LaTeXML::Token",
"LaTeXML::Tokens") that get created during tokenization and
expansion.
A "LaTeXML::Token" represents a TeX token which is a pair of a
character or string and a category code. A "LaTeXML::Tokens" is a
list of tokens (and also implements the API of a LaTeXML::Mouth so that tokens
can be read from a list).
Common methods¶
The following methods apply to all objects.
- "@tokens = $object->unlist;"
- Return a list of the tokens making up this $object.
- "$string = $object->toString;"
- Return a string representing $object.
Token methods¶
The following methods are specific to "LaTeXML::Token".
- "$string = $token->getCSName;"
- Return the string or character part of the $token; for the
special category codes, returns the standard string (eg.
"T_BEGIN-"getCSName> returns "{").
- "$string = $token->getString;"
- Return the string or character part of the $token.
- "$code = $token->getCharcode;"
- Return the character code of the character part of the
$token, or 256 if it is a control sequence.
- "$code = $token->getCatcode;"
- Return the catcode of the $token.
Tokens methods¶
The following methods are specific to "LaTeXML::Tokens".
- "$tokenscopy = $tokens->clone;"
- Return a shallow copy of the $tokens. This is useful before
reading from a "LaTeXML::Tokens".
- "$token = $tokens->readToken;"
- Returns (and remove) the next token from $tokens. This is
part of the public API of LaTeXML::Mouth so that a
"LaTeXML::Tokens" can serve as a LaTeXML::Mouth.
AUTHOR¶
Bruce Miller <bruce.miller@nist.gov>
COPYRIGHT¶
Public domain software, produced as part of work done by the United States
Government & not subject to copyright in the US.