Scroll to navigation

Locales(3pm) User Contributed Perl Documentation Locales(3pm)

NAME

Locales - Methods for getting localized CLDR language/territory names (and a subset of other data)

VERSION

This document describes Locales version 0.33

SYNOPSIS

    use Locales;
    my $locale = Locales->new('en_gb');
    print $locale->get_locale(); # 'en_gb'
    print $locale->get_language(); # 'en'
    print $locale->get_territory(); # 'gb'
    print $locale->get_language_from_code('fr'); # 'French'
    print $locale->get_code_from_language('French'); # 'fr'
    print $locale->get_territory_from_code('us'); # 'United States'
    print $locale->get_code_from_territory('Australia'); # 'au'

DESCRIPTION

Locales lets you create an object for a certain locale that lets you access certain data harvested directly from CLDR.

<http://cldr.unicode.org/index/downloads>

Currently the data/methods include translated locale names and territory names.

For simplicity Locales does not work with or know about Variants or Scripts. It only knows about languages and territories.

Also it does not contain all the data contained in CLDR. For example, DateTime’s localization already has all the calender/date/time info from CLDR. Other information has not had any demand yet.

For consistency all data is written in utf-8. No conversion should be necessary if you are (wisely) using utf-8 as your character set everywhere (See <http://drmuey.com\/?do=page&id=57> for more info on that.).

Note: You probably [don't need to/should not] use utf8 in regards to the data contained herein.

Based on CLDR

This module is based on CLDR v2.0.

You can learn about the Unicode Common Locale Data Repository at <http://cldr.unicode.org/>.

Supported Locale Criteria

The locale tags that can be objectified fit this criteria:

As noted in “I am using a locale code that I know exists in the CLDR but I can't use it anywhere in Locales” in "BEFORE YOU SUBMIT A BUG REPORT".
As noted in “I am using a locale code that I know exists in the CLDR but I can't use it anywhere in Locales” in "BEFORE YOU SUBMIT A BUG REPORT".
As noted in the "DESCRIPTION".
As described in "Soft Locales".

Soft Locales

As “soft locale” is a language-territory locale that does not fit the "Supported Locale Criteria" directly but its super does and the territory is known.

For example “es-MX” does not fit the criteria but “es” does and “MX” is a valid territory code.

To byte or not to byte, that is the question. Ok, I’ll byte …

The CLDR data is in bytes, specifically utf-8.

By default this module simply passes along the strings as bytes, which works fine in applications that don’t operate in character mode (i.e. that use utf8 byte strings instead of Unicode strings**).

If you want Unicode strings instead you can do so by bringing in the module this way:

    use Locales unicode => 1;

[**] What is the difference between Unicode strings and utf-8 bytes strings you ask? See String::UnicodeUTF8 for more info.

INTERFACE

new()

Takes one argument, the locale tag whose CLDR data you want to use.

No argument defaults to 'en'.

It is an argument based singleton so you can call it more than once with out it having to rebuild the object every time.

It returns false if a locale given is not available. $@ should have been set at that point by eval.

    my $en = Locales->new('en') or die $@;

Object methods

Misc methods

Takes no arguments.

Returns the version of the CLDR any data it uses comes from. Can also be called as a class method or function.

Takes no arguments.

Returns the normalized locale of the object, this is the same as the argument to new()

Takes no arguments.

Returns the language portion of the object’s locale.

Takes no arguments.

Returns the territory portion of the object’s locale if any (e.g. 'en_au'), undef if there is none (e.g. 'it').

Takes no arguments.

Returns the locale that the object is based on in the case that the given locale (i.e. "get_locale()") is a soft locale.

Note: If you do not want to have soft locale objects you should simply not call new() if it is soft:

    - my $loc = Locales->new($tag) || die $@;
    + my $loc = (Locales::tag_is_soft_locale($tag) ? undef : Locales->new($tag)) || die $@;
    

This could be added to the constructor but for now I don't want to make it more complicated only to support something that seems odd. If you have a use case submit an rt w/ details. Thanks!

Note: As of v0.17 you probably want "get_formatted_decimal()" instead of numf().

Takes one optional boolean argument.

Returns 1 if the object’ss locale’s number format is comma for thousand separator, period for decimal.

Returns 2 if the object’s locale’s number format is period for thousand separator, comma for decimal.

Otherwise it returns a reference to a 3 element array containing this CLDR data: number format, separator character, decimal character.

The boolean argument, when true will do it’s best to determine and return a 1 or a 2.

Territory methods

Take no arguments.

Returns an unsorted list of known territory codes.

Take no arguments.

Returns an unsorted list of the display names for each known territory code.

Take no arguments.

Returns a copy of the lookup hash of the display names for each known territory code.

Takes one argument, the locale code whose territory name you want to find. Defaults to the territory of the of object’s locale, if any.

Returns the name of the given tag’s territory or, if not found, the territory portion (if any), returns false otherwise.

An optional second argument, when true, will force it to return the normalized tag if nothing else can be figured out.

Takes one argument, the territory name whose locale you want to find.

Returns the locale tag if found, false otherwise.

Alias for get_territory_from_code()
Alias for get_code_from_territory()

Language Methods

Take no arguments.

Returns an unsorted list of known language codes.

Take no arguments.

Returns an unsorted list of the display names for each known language code.

Take no arguments.

Returns a copy of the lookup hash of the display names for each known language code.

Takes one argument, the locale code whose language name you want to find. Defaults to the object’s locale.

Returns the name of the given tag’s language, returns false otherwise.

An optional second argument, when true, will force it to return a properly formatted CLDR format display based on if we know the language and/or territory if nothing else can be figured out.

Takes one argument, the language name whose locale you want to find.

Returns the locale tag if found, false otherwise.

Like get_language_from_code() except it returns the name in the given locale’s native language.
Like get_language_from_code() except it returns the character orientation identifier for the given locale. (defaulting to the locale of the object if non is given)

Typically it will be the string “left-to-right” or “right-to-left”.

See <http://unicode.org/repos/cldr-tmp/trunk/diff/by_type/misc.layout.html> for more information.

Same as get_character_orientation_from_code() except it should use less-overhead. Can be called as a function also so you can use it without creating an object.
Like get_character_orientation_from_code() except it returns the locale display pattern for the given locale. (defaulting to the locale of the object if non is given)

Typically it will be something like '{0} ({1})'

See <http://unicode.org/repos/cldr-tmp/trunk/diff/by_type/names.localeDisplayPattern.html> for more information.

Same as get_locale_display_pattern_from_code() except it should use less-overhead. Can be called as a function also so you can use it without creating an object.
Returns the decimal point symbol for the object’s locale. Takes no arguments.

For formatting numbers use get_formatted_decimal().

Returns the integer grouping symbol for the object’s locale. Takes no arguments.

For formatting numbers use get_formatted_decimal().

Returns a fallback list of locales in the order they apply based on the object’s locale.

The basic list will be: object’s locale, object’s super if any, the object’s CLDR fallback if any, “special lookup” if any, 'en'

    my @list = $fr_ca->get_fallback_list();
    # fr_ca fr en
    

"special lookup" is a code ref that can be passed in as the first optional arg.

It is given the object’s locale when called and should return a list of locale tags (they will be normalized).

    my @list = $fr_ca->get_fallback_list(sub { return $_[0] =~ m/fr/ ? qw(i_yoda i_love_rhi) : () } );
    # fr_ca fr i_yoda i_love_rhi en
    
Takes a number and returns the plural category that the number fits under for the object’s locale.

You can also add an array of items to return instead of the category name. For the details on what arguments a given local needs see Locales::DB::Docs::PluralForms.

The array should be the same length of the list of plural form categories for the locale. See get_plural_form_categories().

The exception to that is when you specify the optional "“Special Zero” Argument" in Locales::DB::Docs::PluralForms.

For example, 'en' has the plural categories “one” and “other”, so it'd work like this:

    my $cat = $en->get_plural_form(0); # 'other'
    my $str = $en->get_plural_form(0,'I am 1','I am other'); # I am other
    my $str = $en->get_plural_form(0,'I am 1','I am other','I am nothing'); # I am nothing
    my $cat = $en->get_plural_form(1); # 'one'
    my $str = $en->get_plural_form(1,'I am 1','I am other'); # I am 1
    my $str = $en->get_plural_form(1,'I am 1','I am other','I am nothing');  #I am 1
    my $cat = $en->get_plural_form(2); # 'other'
    my $str = $en->get_plural_form(2,'I am 1','I am other'); # I am other
    my $str = $en->get_plural_form(2,'I am 1','I am other','I am nothing'); # I am other
    

In array context the second value is a boolean for if the return value is the "“Special Zero” Argument" in Locales::DB::Docs::PluralForms or not.

This boolean value only has meaning when called with the additional array of items to return instead of the category name.

This method can carp() a few things:

"Could not determine plural logic."
The locale does not have plural logic data.
"The number of given values (%d) does not match the number of categories (%d)."
You passed too many or too few values after the initial numeric argument.

You'll only see this if $locales_object->{'verbose'} is set to true.

"The category (%s) is not used by this locale."
The locale’s plural rules come up with a category that is not applicable to the locale. Default to “other” at this point.
Returns an array of the CLDR plural rule category names that this locale uses.

Their order corresponds to the position of the corresponding value that get_plural_form() uses.

Takes no arguments, returns a boolean.

It is true if the locale uses the "“Special Zero” Argument" in Locales::DB::Docs::PluralForms.

False if it does not.

Takes no arguments.

Returns the number of plural categories applicable to the object’s locale.

Does not factor in support (or not) of the special zeroth category.

Stringify an "and" list of items as defined in the CLDR for the object’s locale.

Note: get_list_or() will be done once CLDR defines the OR-list data <http://unicode.org/cldr/trac/ticket/4051>.

    $en->get_list_and() # nothing
    $en->get_list_and(1) # 1
    $en->get_list_and(1,2) # 1 and 2
    $en->get_list_and(1,2,3) # 1, 2, and 3
    $en->get_list_and(1,2,3,4) # 1, 2, 3, and 3
    $es->get_list_and() # nothing
    $es->get_list_and(1) # 1
    $es->get_list_and(1,2) # 1 y 2
    $es->get_list_and(1,2,3) # 1, 2 y 3
    $es->get_list_and(1,2,3,4) # 1, 2, 3 y 3
    

To help disambiguate ambiguous arguments (none, undef, “”, all space/non-break-space) you can use $loc->{'misc'}{'list_quote_mode'}.

The default value is “none”.

Possible values:

“all”
quote() all values.
“some”
quote() only ambiguous values (none (as if it was “”), undef, “”, all space/non-break-space).
“none”
do not quote() any values

If another value is given or the entry does not exist you'll get “none” behavior. If it is set to an unknown value you'll get a carp() of “$self->{misc}{list_quote_mode} is set to an unknown value”.

Stringify an "or" list of items as defined in the CLDR for the object’s locale.

This is a stub until CLDR defines the OR-list data <http://unicode.org/cldr/trac/ticket/4051>.

Until then it is essentially the same as "get_list_and()"except it uses English rules/grammer for or lists.

Uses $loc->{'misc'}{'list_quote_mode'} the same way get_list_and() does.

Formats the given string per the initial ellipsis pattern.

Truncating for length is the caller’s responsibility since knowing how to do that correctly depends on what the string is (e.g. plain text needs to factor in the encoding or we might corrupt the text, HTML might have a broken or unclosed tag, ANSI might be unclosed or truncated, etc) so it is outside of the scope of the CLDR.

    …foo
    
Formats the given string per the medial ellipsis pattern.

Truncating for length is the caller’s responsibility since knowing how to do that correctly depends on what the string is (e.g. plain text needs to factor in the encoding or we might corrupt the text, HTML might have a broken or unclosed tag, ANSI might be unclosed or truncated, etc) so it is outside of the scope of the CLDR.

    foo…bar
    
Formats the given string per the medial ellipsis pattern.

Truncating for length is the caller’s responsibility since knowing how to do that correctly depends on what the string is (e.g. plain text needs to factor in the encoding or we might corrupt the text, HTML might have a broken or includes tag, ANSI might be unclosed or truncated, etc) so it is outside of the scope of the CLDR.

    foo…
    
Quotes the argument with the CLDR delimiters quotation_start and quotation_end.
Quotes the argument with the CLDR delimiters alternate_quotation_start and alternate_quotation_end.
Return the given number as a string formatted per the locale’s CLDR decimal format pattern.

An optional second argument defines a maximum length of decimal places (default is 6 perl %f, max is 14, if you have a need for a larger max please open an rt w/ context and we may make the max settable in the object)

    $fr->get_formatted_decimal("1234567890.12345"); # 1 234 567 890,12345
    $fr->get_formatted_decimal("1234567890.12345",4); # 1 234 567 890,123
    $fr->get_formatted_decimal("1234567890.12345",3); # 1 234 567 890,1235
    

Perl number stringification caveats:

     $l->get_formatted_decimal(99999999999999999983222787.1234); # returns 1e+26 since that is how it comes into the function
     $l->get_formatted_decimal("99999999999999999983222787.1234"); # 99,999,999,999,999,999,983,222,787.1234
    
     $l->get_formatted_decimal(10000000001.12345678911234,12); # 10,000,000,001.1235 since it is already truncated  when it comes into the function
     $l->get_formatted_decimal("10000000001.12345678911234",12); # 10,000,000,001.123456789112
    
That is OK though, since this isn't intended to be used in math and you are already aware of how large integers and decimals act oddly on computers right?
This is why we have a hard limit of 14 decimal places, to enforce some sense of sanity. You might consider only using the max decimal places argument to make it less than 6 digits long.

This method can carp() a few (hopefully self explanatory) things regarding CLDR number format syntax errors:

"Format had more than 2 pos/neg sections. Using default pattern."
"Format should have one decimal section. Using default pattern."
"Format is empty. Using default pattern."
Alias for get_language_from_code()
Alias for get_code_from_language()

Utility functions

These are some functions used internally that you might find useful.

Takes a single argument, the locale tag to normalize.

Returns the normalized tag.

   print Locales::normalize_tag("  en-GB\n "); # 'en_gb'
    
Like normalize_tag() except the return value should be suitable for DateTime::Locale

   print Locales::normalize_tag_for_datetime_locale("  en-GB\n "); # 'en_GB'
    
Like normalize_tag() except the return value should be suitable for IETF.

This is not a comprehensive IETF formatter, it is intended (for now at least) for the subset of tags Locales.pm uses.

   print Locales::normalize_tag_for_ietf("  en_gb\n "); # 'en-GB'
    
Takes a single argument, the locale tag to split into language and territory parts.

Returns the resulting array of 1 or 2 normalized (but not validated) items.

   my ($language, $territory) = Locales::split_tag("  en-GB\n "); # ('en','gb')
   my ($language, $territory) = Locales::split_tag('fr'); # ('fr');
   my ($language, $territory) = Locales::split_tag('sr_Cyrl_YU'); # ('sr','cyrl_yu'), yes 'cyrl_yu' is invalid here since Locales doesn't work with the Script variants, good catch
    
Takes a single argument, the locale tag string to transform into "i" notation.

Returns the resulting normalized locale tag.

The standard tag for strings/tags without a standard is an "i" notation tag.

For example, the language "Yoda Speak" does not have an ISO code. You'd have to use i_yoda_speak.

    # assuming $string = "Yoda Speak"; you'd get into the if(), assuming it was 'Spanish' or 'es'
    if (!$en->get_language_from_code($string) && !$en->get_code_from_language($string) ) {
        # it is not a code or a language (at least in the language of $en) so lets create a tag for it:
        _create_locale_files( Locales::get_i_tag_for_string($string) ); # i_yoda_speak
    }
    else {
        # if it is a language name then we fetch the code otherwise, at this point, we know it is a code, so return a normailized version
        _create_locale_files( $en->get_code_from_language($yoda) || Locales::normalize_tag($yoda) );
    }
    
Takes a single argument, the locale tag you want to check to see if it is <soft locale|/Soft Locales> or not.

If it is it returns the super portion that an object would be based on. If it is not it returns false.

Returns true if the given tag can be loaded as a Locales object via new(). False otherwise.
Returns true if the given tag is a known territory. False otherwise.
Takes no arguments. Returns an unsorted list of codes that can be loaded as a Locales object via new().
Takes no arguments. Returns a list of locale tags that are not actually locales. e.g. 'mul' means “Multiple Languages”.
Takes a locale tag as the argument and returns true if it is a non-locale code (See "Locales::non_locale_list()"), false otherwise.
Takes no arguments. Returns a list of locale tags that are typically aliases of 'en'.
Takes a locale tag as the argument and returns true if it is typically an alias of 'en' (See "Locales::typical_en_alias_list()"), false otherwise.
Takes a single argument, the phrase string normalize in the same way the names are stored in each locale’s lookup hash.

Returns the resulting normalized string.

This is used internally to normalize a given name in the same manner the name-to-code hash keys are normalized.

If said normalization is ever improved then using this function will ensure everything is normalized consistently.

That allows $en->get_code_from_language($name) to map to 'afa' if given these various variations of $arg:

  "Afro-Asiatic Language"
  "afroasiatic\tLanguage"
  "AFRO-Asiatic Language"
  "  Afro_Asiatic    Language"
  "afro.Asiatic Language\n"
    
Returns a list of plural categories that CLDR uses.

With no argument, the order is what is appropriate for some noun quantifying localization methods.

With a true argument, the order is the order it makes sense to check their corresponding rules in.

This is used under the hood to facilitate get_plural_form(). That being the case there probably isn't much use for it to be used directly.

This takes the plural rule string as found in the CLDR XML and returns an eval()able perl code version of it.

It will carp "Unknown plural rule syntax" and return; if it does not understand what you sent.

A second, optional, argument is the value to return if the rule matches.

If you eval the returned string you'll have a code reference that returns true (or whatever you give it) if the rule matched the given number or not:

    my $perly = Locales::plural_rule_string_to_code("n is 42 or n mod 42 is not 7");
    my $check = eval $perly;
    my $plural_category = $check->(42);
    
This is used under the hood to facilitate get_plural_form(). That being the case there probably isn't much use for it to be used directly.

This takes a hashref that contains rules, puts them in the hash, and returns an overall code ref. Its pretty internal so if you really need the details have a gander at the source.

Same as Locales::plural_rule_string_to_code() except it returns javascript code instead of perl code.

Used internally when building this distribution’s share/misc_info contents.

DIAGNOSTICS

Throws no warning or errors of it’s own. If any function or method returns false then the arguments given (or not given) were invalid/not found.

Deviations from this are documented per function/method.

CONFIGURATION AND ENVIRONMENT

Locales requires no configuration files or environment variables.

DEPENDENCIES

None.

INCOMPATIBILITIES

None reported.

TODO

  - CLDR builder TODOs
  - more CLDR version/misc-info fetchers
  - generally improve get_code_from_* lookups
  - tests that misc info doesn't get odd structs from XML instead of a string
  - ? install share/ via L<File::ShareDir> mechanism ?
  - vet share/ && document better

DEPRECATED MODULES/INTERFACE

The original, non CLDR based, '::Base' based modules/interface in this distribution were deprecated in version 0.06.

These modules were removed in version 0.15.

BUGS AND FEATURES

Please report any bugs or feature requests (and a pull request for bonus points)
through the issue tracker at <https://github.com/drmuey/p5-Locales/issues>.

Please report any bugs or feature requests regarding CLDR data as per <http://cldr.unicode.org/index/bug-reports>.

BEFORE YOU SUBMIT A BUG REPORT

Please read TODO, DESCRIPTION, and the information below thoroughly to see if your thought is already addressed.

  • A non-English object returns English names.

    Data that is not defined in a locale’s CLDR data falls back to English.

    Please report the missing data to the CLDR as per <http://cldr.unicode.org/index/bug-reports>.

  • I am using a locale code that I know exists in the CLDR but I can't use it anywhere in Locales

    Only locales and territory codes that 'en' knows about are used. Only locales that have their own data set in CLDR are able to be objectified.

    Additions or updates can be request as per <http://cldr.unicode.org/index/bug-reports>.

  • A name is misformatted, incorrect, etc.

    The data is automatically harvested from CLDR. So if there is really a problem you'll have to report the problem to them. (as per <http://cldr.unicode.org/index/bug-reports>)

    Here are some things to check before submitting a report:

Corrupt text
  • Is your charset correct?

    For example, viewing UTF-8 characters on a latin1 web page will result in garbled characters.

  • It still looks corrupt!

    Some locale’s require special fonts to be installed on your system to view them properly.

    For example Bengali (bn) is like this. As per <http://www.unicode.org/help/display_problems.html> if you install the proper font it renders correctly.

Incorrect data or formatting
  • Is it really inaccurate?

    It could simply be an incomplete understanding of the context of the data, for example:

    In English we capitalize proper names (e.g. French).

    In other languages it may be perfectly acceptable for a language or territory name to not start with upper case letters.

    In that case a report about names not being capitalized like we do in English would be unwarranted.

  • Is it really mis-formatted?

    Sometimes something might look strange to us and we'd be tempted to report the problem. Keep in mind though that sometimes locale nuances can cause things to render in a way that non-native speakers may not understand.

    For example Arabic’s (ar) right-to-left text direction can seem strange when mixed with latin text. It's simply not wrong. You may be able to improve it by using the direction data to render it better (e.g. CSS or HTML attributes if the output is HTML).

    Also, CLDR pattern formats can differ per locale.

    In cases like this a report would be unwarranted.

AUTHOR

Daniel Muey "<http://drmuey.com/cpan_contact.pl>"

LICENCE AND COPYRIGHT

Copyright (c) 2009, Daniel Muey "<http://drmuey.com/cpan_contact.pl>". All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

DISCLAIMER OF WARRANTY

BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

2022-11-19 perl v5.36.0