.\" Automatically generated by Pod::Man 2.25 (Pod::Simple 3.16) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .ie \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .el \{\ . de IX .. .\} .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "Scalar::String 3pm" .TH Scalar::String 3pm "2011-11-15" "perl v5.14.2" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Scalar::String \- string aspects of scalars .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 2 \& use Scalar::String \& qw(sclstr_is_upgraded sclstr_is_downgraded); \& \& if(sclstr_is_upgraded($value)) { ... \& if(sclstr_is_downgraded($value)) { ... \& \& use Scalar::String qw( \& sclstr_upgrade_inplace sclstr_upgraded \& sclstr_downgrade_inplace sclstr_downgraded \& ); \& \& sclstr_upgrade_inplace($value); \& $value = sclstr_upgraded($value); \& sclstr_downgrade_inplace($value); \& $value = sclstr_downgraded($value); .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" This module is about the string part of plain Perl scalars. A scalar has a string value, which is notionally a sequence of Unicode codepoints, but may be internally encoded in either \s-1ISO\-8859\-1\s0 or \s-1UTF\-8\s0. In places, and more so in older versions of Perl, the internal encoding shows through. To fully understand Perl strings it is necessary to understand these implementation details. .PP This module provides functions to classify a string by encoding and to encode a string in a desired way. .PP This module is implemented in \s-1XS\s0, with a pure Perl backup version for systems that can't handle \s-1XS\s0. .SH "STRING ENCODING" .IX Header "STRING ENCODING" \&\s-1ISO\-8859\-1\s0 is a simple 8\-bit character encoding, which represents the first 256 Unicode characters (codepoints 0x00 to 0xff) in one octet each. This is how strings were historically represented in Perl. A string represented this way is referred to as \*(L"downgraded\*(R". .PP \&\s-1UTF\-8\s0 is a variable-width character encoding, which represents all possible Unicode codepoints in differing numbers of octets. A design feature of \s-1UTF\-8\s0 is that \s-1ASCII\s0 characters (codepoints 0x00 to 0x7f) are each represented in a single octet, identically to their \s-1ISO\-8859\-1\s0 encoding. Perl has its own variant of \s-1UTF\-8\s0, which can handle a wider range of codepoints than Unicode formally allows. A string represented in this variant \s-1UTF\-8\s0 is referred to as \*(L"upgraded\*(R". .PP A Perl string is physically represented as a string of octets along with a flag that says whether the string is downgraded or upgraded. At this level, to determine the Unicode codepoints that are represented requires examining both parts of the representation. If the string contains only \&\s-1ASCII\s0 characters then the octet sequence is identical in either encoding, but Perl still maintains an encoding flag on such a string. A string is always either downgraded or upgraded; it is never both or neither. .PP When handling string input, it is good form to operate only on the Unicode characters represented by the string, ignoring the manner in which they are encoded. Basic string operations such as concatenation work this way (except for a bug in perl 5.6.0), so simple code written in pure Perl is generally safe on this front. Pieces of character-based code can pass around strings among themselves, and always get consistent behaviour, without worrying about the way in which the characters are encoded. .PP However, due to an historical accident, a lot of C code that interfaces with Perl looks at the octets used to represent a string without also examining the encoding flag. Such code gives inconsistent behaviour for the same character sequence represented in the different ways. In perl 5.6, many pure Perl operations (such as regular expression matching) also work this way, though some of them can be induced to work correctly by using the utf8 pragma. In perl 5.8, regular expression matching is character-based by default, but many I/O functions (such as \f(CW\*(C`open\*(C'\fR) are still octet-based. .PP Where code that operates on the octets of a string must be used by code that operates on characters, the latter needs to pay attention to the encoding of its strings. Commonly, the octet-based code expects its input to be represented in a particular encoding, in which case the character-based code must oblige by forcing strings to that encoding before they are passed in. There are other usage patterns too. .PP You will be least confused if you think about a Perl string as a character sequence plus an encoding flag. You should normally operate on the character sequence and not care about the encoding flag. Occasionally you must pay attention to the flag in addition to the characters. Unless you are writing C code, you should try not to think about a string the other way round, as an octet sequence plus encoding flag. .SH "FUNCTIONS" .IX Header "FUNCTIONS" Each \*(L"sclstr_\*(R" function takes one or more scalar string arguments to operate on. These arguments must be strings; giving non-string arguments will cause mayhem. See \*(L"is_string\*(R" in Params::Classify for a way to check for stringness. Only the string value of the scalar is used; the numeric value is completely ignored, so dualvars are not a problem. .SS "Classification" .IX Subsection "Classification" .IP "sclstr_is_upgraded(\s-1VALUE\s0)" 4 .IX Item "sclstr_is_upgraded(VALUE)" Returns a truth value indicating whether the provided string \fI\s-1VALUE\s0\fR is in upgraded form. .IP "sclstr_is_downgraded(\s-1VALUE\s0)" 4 .IX Item "sclstr_is_downgraded(VALUE)" Returns a truth value indicating whether the provided string \fI\s-1VALUE\s0\fR is in downgraded form. .SS "Regrading" .IX Subsection "Regrading" .IP "sclstr_upgrade_inplace(\s-1VALUE\s0)" 4 .IX Item "sclstr_upgrade_inplace(VALUE)" Modifies the string \fI\s-1VALUE\s0\fR in-place, so that it is in upgraded form, regardless of how it was encoded before. The character sequence that it represents is unchanged. .Sp A cleaner interface to this operation is the non-mutating \&\*(L"sclstr_upgraded\*(R". .IP "sclstr_upgraded(\s-1VALUE\s0)" 4 .IX Item "sclstr_upgraded(VALUE)" Returns a string that represents the same character sequence as the string \&\fI\s-1VALUE\s0\fR, and is in upgraded form (regardless of how \fI\s-1VALUE\s0\fR is encoded). .IP "sclstr_downgrade_inplace(VALUE[, \s-1FAIL_OK\s0])" 4 .IX Item "sclstr_downgrade_inplace(VALUE[, FAIL_OK])" Modifies the string \fI\s-1VALUE\s0\fR in-place, so that it is in downgraded form, regardless of how it was encoded before. The character sequence that it represents is unchanged. If the string cannot be downgraded, because it contains a non\-ISO\-8859\-1 character, then by default the function \f(CW\*(C`die\*(C'\fRs, but if \fI\s-1FAIL_OK\s0\fR is present and true then it will return leaving \fI\s-1VALUE\s0\fR unmodified. .Sp A cleaner interface to this operation is the non-mutating \&\*(L"sclstr_downgraded\*(R". .IP "sclstr_downgraded(VALUE[, \s-1FAIL_OK\s0])" 4 .IX Item "sclstr_downgraded(VALUE[, FAIL_OK])" Returns a string that represents the same character sequence as the string \fI\s-1VALUE\s0\fR, and is in downgraded form (regardless of how \fI\s-1VALUE\s0\fR is encoded). If the string cannot be represented in downgraded form, because it contains a non\-ISO\-8859\-1 character, then by default the function \f(CW\*(C`die\*(C'\fRs, but if \fI\s-1FAIL_OK\s0\fR is present and true then it will return \fI\s-1VALUE\s0\fR in its original upgraded form. .SH "SEE ALSO" .IX Header "SEE ALSO" utf8 .SH "AUTHOR" .IX Header "AUTHOR" Andrew Main (Zefram) .SH "COPYRIGHT" .IX Header "COPYRIGHT" Copyright (C) 2009, 2010, 2011 Andrew Main (Zefram) .SH "LICENSE" .IX Header "LICENSE" This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.