language

package
v0.14.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 26, 2024 License: BSD-3-Clause Imports: 7 Imported by: 37

Documentation

Overview

Package language implements BCP 47 language tags and related functionality.

The most important function of package language is to match a list of user-preferred languages to a list of supported languages. It alleviates the developer of dealing with the complexity of this process and provides the user with the best experience (see https://blog.golang.org/matchlang).

Matching preferred against supported languages

A Matcher for an application that supports English, Australian English, Danish, and standard Mandarin can be created as follows: 

var matcher = language.NewMatcher([]language.Tag{
    language.English,   // The first language is used as fallback.
    language.MustParse("en-AU"),
    language.Danish,
    language.Chinese,
})

This list of supported languages is typically implied by the languages for which there exists translations of the user interface.

User-preferred languages usually come as a comma-separated list of BCP 47 language tags. The MatchString finds best matches for such strings: 

handler(w http.ResponseWriter, r *http.Request) {
    lang, _ := r.Cookie("lang")
    accept := r.Header.Get("Accept-Language")
    tag, _ := language.MatchStrings(matcher, lang.String(), accept)

    // tag should now be used for the initialization of any
    // locale-specific service.
}

The Matcher's Match method can be used to match Tags directly.

Matchers are aware of the intricacies of equivalence between languages, such as deprecated subtags, legacy tags, macro languages, mutual intelligibility between scripts and languages, and transparently passing BCP 47 user configuration. For instance, it will know that a reader of Bokmål Danish can read Norwegian and will know that Cantonese ("yue") is a good match for "zh-HK".

Using match results

To guarantee a consistent user experience to the user it is important to use the same language tag for the selection of any locale-specific services. For example, it is utterly confusing to substitute spelled-out numbers or dates in one language in text of another language. More subtly confusing is using the wrong sorting order or casing algorithm for a certain language.

All the packages in x/text that provide locale-specific services (e.g. collate, cases) should be initialized with the tag that was obtained at the start of an interaction with the user.

Note that Tag that is returned by Match and MatchString may differ from any of the supported languages, as it may contain carried over settings from the user tags. This may be inconvenient when your application has some additional locale-specific data for your supported languages. Match and MatchString both return the index of the matched supported tag to simplify associating such data with the matched tag.

Canonicalization

If one uses the Matcher to compare languages one does not need to worry about canonicalization.

The meaning of a Tag varies per application. The language package therefore delays canonicalization and preserves information as much as possible. The Matcher, however, will always take into account that two different tags may represent the same language.

By default, only legacy and deprecated tags are converted into their canonical equivalent. All other information is preserved. This approach makes the confidence scores more accurate and allows matchers to distinguish between variants that are otherwise lost.

As a consequence, two tags that should be treated as identical according to BCP 47 or CLDR, like "en-Latn" and "en", will be represented differently. The Matcher handles such distinctions, though, and is aware of the equivalence relations. The CanonType type can be used to alter the canonicalization form.

References

BCP 47 - Tags for Identifying Languages http://tools.ietf.org/html/bcp47

Index

Examples

Constants

View Source 
const (
	// Replace deprecated base languages with their preferred replacements.
	DeprecatedBase CanonType = 1 << iota
	// Replace deprecated scripts with their preferred replacements.
	DeprecatedScript
	// Replace deprecated regions with their preferred replacements.
	DeprecatedRegion
	// Remove redundant scripts.
	SuppressScript
	// Normalize legacy encodings. This includes legacy languages defined in
	// CLDR as well as bibliographic codes defined in ISO-639.
	Legacy
	// Map the dominant language of a macro language group to the macro language
	// subtag. For example cmn -> zh.
	Macro
	// The CLDR flag should be used if full compatibility with CLDR is required.
	// There are a few cases where language.Tag may differ from CLDR. To follow all
	// of CLDR's suggestions, use All|CLDR.
	CLDR

	// Raw can be used to Compose or Parse without Canonicalization.
	Raw CanonType = 0

	// Replace all deprecated tags with their preferred replacements.
	Deprecated = DeprecatedBase | DeprecatedScript | DeprecatedRegion

	// All canonicalizations recommended by BCP 47.
	BCP47 = Deprecated | SuppressScript

	// All canonicalizations.
	All = BCP47 | Legacy | Macro

	// Default is the canonicalization used by Parse, Make and Compose. To
	// preserve as much information as possible, canonicalizations that remove
	// potentially valuable information are not included. The Matcher is
	// designed to recognize similar tags that would be the same if
	// they were canonicalized using All.
	Default = Deprecated | Legacy
)
View Source 
const CLDRVersion = "32"

CLDRVersion is the CLDR version from which the tables in this package are derived.

View Source 
const NumCompactTags = compact.NumCompactTags

NumCompactTags is the number of compact tags. The maximum tag is NumCompactTags-1.

Variables

View Source 
var ErrMissingLikelyTagsData = errors.New("missing likely tags data")

ErrMissingLikelyTagsData indicates no information was available to compute likely values of missing tags.

Functions

func CompactIndex 

func CompactIndex(t Tag) (index int, exact bool)

CompactIndex returns an index, where 0 <= index < NumCompactTags, for tags for which data exists in the text repository.The index will change over time and should not be stored in persistent storage. If t does not match a compact index, exact will be false and the compact index will be returned for the first match after repeatedly taking the Parent of t.

func Compare 

func Compare(a Tag, others ...Tag) (equal bool)

Compare returns true if the first tag is equivalent to one of the remaining tags given

Compare was added by Go-Enjin

Types

type Base 

type Base struct {
	// contains filtered or unexported fields
}

Base is an ISO 639 language code, used for encoding the base language of a language tag.

func MustParseBase 

func MustParseBase(s string) Base

MustParseBase is like ParseBase, but panics if the given base cannot be parsed. It simplifies safe initialization of Base values.

func ParseBase 

func ParseBase(s string) (Base, error)

ParseBase parses a 2- or 3-letter ISO 639 code. It returns a ValueError if s is a well-formed but unknown language identifier or another error if another error occurred.

func (Base) ISO3 

func (b Base) ISO3() string

ISO3 returns the ISO 639-3 language code.

func (Base) IsPrivateUse 

func (b Base) IsPrivateUse() bool

IsPrivateUse reports whether this language code is reserved for private use.

func (Base) String 

func (b Base) String() string

String returns the BCP 47 representation of the base language.

type CanonType 

type CanonType int

CanonType can be used to enable or disable various types of canonicalization.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	p := func(id string) {
		fmt.Printf("Default(%s) -> %s\n", id, language.Make(id))
		fmt.Printf("BCP47(%s) -> %s\n", id, language.BCP47.Make(id))
		fmt.Printf("Macro(%s) -> %s\n", id, language.Macro.Make(id))
		fmt.Printf("All(%s) -> %s\n", id, language.All.Make(id))
	}
	p("en-Latn")
	p("sh")
	p("zh-cmn")
	p("bjd")
	p("iw-Latn-fonipa-u-cu-usd")
}

Output:

Default(en-Latn) -> en-Latn
BCP47(en-Latn) -> en
Macro(en-Latn) -> en-Latn
All(en-Latn) -> en
Default(sh) -> sr-Latn
BCP47(sh) -> sh
Macro(sh) -> sh
All(sh) -> sr-Latn
Default(zh-cmn) -> cmn
BCP47(zh-cmn) -> cmn
Macro(zh-cmn) -> zh
All(zh-cmn) -> zh
Default(bjd) -> drl
BCP47(bjd) -> drl
Macro(bjd) -> bjd
All(bjd) -> drl
Default(iw-Latn-fonipa-u-cu-usd) -> he-Latn-fonipa-u-cu-usd
BCP47(iw-Latn-fonipa-u-cu-usd) -> he-Latn-fonipa-u-cu-usd
Macro(iw-Latn-fonipa-u-cu-usd) -> iw-Latn-fonipa-u-cu-usd
All(iw-Latn-fonipa-u-cu-usd) -> he-Latn-fonipa-u-cu-usd

func (CanonType) Canonicalize 

func (c CanonType) Canonicalize(t Tag) (Tag, error)

Canonicalize returns the canonicalized equivalent of the tag.

func (CanonType) Compose 

func (c CanonType) Compose(part ...interface{}) (t Tag, err error)

Compose creates a Tag from individual parts, which may be of type Tag, Base, Script, Region, Variant, []Variant, Extension, []Extension or error. If a Base, Script or Region or slice of type Variant or Extension is passed more than once, the latter will overwrite the former. Variants and Extensions are accumulated, but if two extensions of the same type are passed, the latter will replace the former. For -u extensions, though, the key-type pairs are added, where later values overwrite older ones. A Tag overwrites all former values and typically only makes sense as the first argument. The resulting tag is returned after canonicalizing using CanonType c. If one or more errors are encountered, one of the errors is returned.

func (CanonType) Make 

func (c CanonType) Make(s string) Tag

Make is a convenience wrapper for c.Parse that omits the error. In case of an error, a sensible default is returned.

func (CanonType) MustParse 

func (c CanonType) MustParse(s string) Tag

MustParse is like Parse, but panics if the given BCP 47 tag cannot be parsed. It simplifies safe initialization of Tag values.

func (CanonType) Parse 

func (c CanonType) Parse(s string) (t Tag, err error)

Parse parses the given BCP 47 string and returns a valid Tag. If parsing failed it returns an error and any part of the tag that could be parsed. If parsing succeeded but an unknown value was found, it returns ValueError. The Tag returned in this case is just stripped of the unknown value. All other values are preserved. It accepts tags in the BCP 47 format and extensions to this standard defined in https://www.unicode.org/reports/tr35/#Unicode_Language_and_Locale_Identifiers. The resulting tag is canonicalized using the canonicalization type c.

type Confidence 

type Confidence int

Confidence indicates the level of certainty for a given return value. For example, Serbian may be written in Cyrillic or Latin script. The confidence level indicates whether a value was explicitly specified, whether it is typically the only possible value, or whether there is an ambiguity. 

const (
	No    Confidence = iota // full confidence that there was no match
	Low                     // most likely value picked out of a set of alternatives
	High                    // value is generally assumed to be the correct match
	Exact                   // exact match or explicitly specified value
)

func Comprehends 

func Comprehends(speaker, alternative Tag) Confidence

Comprehends reports the confidence score for a speaker of a given language to being able to comprehend the written form of an alternative language.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	// Various levels of comprehensibility.
	fmt.Println(language.Comprehends(language.English, language.English))
	fmt.Println(language.Comprehends(language.AmericanEnglish, language.BritishEnglish))

	// An explicit Und results in no match.
	fmt.Println(language.Comprehends(language.English, language.Und))

	fmt.Println("----")

	// There is usually no mutual comprehensibility between different scripts.
	fmt.Println(language.Comprehends(language.Make("en-Dsrt"), language.English))

	// One exception is for Traditional versus Simplified Chinese, albeit with
	// a low confidence.
	fmt.Println(language.Comprehends(language.TraditionalChinese, language.SimplifiedChinese))

	fmt.Println("----")

	// A Swiss German speaker will often understand High German.
	fmt.Println(language.Comprehends(language.Make("gsw"), language.Make("de")))

	// The converse is not generally the case.
	fmt.Println(language.Comprehends(language.Make("de"), language.Make("gsw")))

}

Output:

Exact
High
No
----
No
Low
----
High
No

func (Confidence) String 

func (c Confidence) String() string

type Coverage 

type Coverage interface {
	// Tags returns the list of supported tags.
	Tags() []Tag

	// BaseLanguages returns the list of supported base languages.
	BaseLanguages() []Base

	// Scripts returns the list of supported scripts.
	Scripts() []Script

	// Regions returns the list of supported regions.
	Regions() []Region
}

The Coverage interface is used to define the level of coverage of an internationalization service. Note that not all types are supported by all services. As lists may be generated on the fly, it is recommended that users of a Coverage cache the results. 

var (
	// Supported defines a Coverage that lists all supported subtags. Tags
	// always returns nil.
	Supported Coverage = allSubtags{}
)

func NewCoverage 

func NewCoverage(list ...interface{}) Coverage

NewCoverage returns a Coverage for the given lists. It is typically used by packages providing internationalization services to define their level of coverage. A list may be of type []T or func() []T, where T is either Tag, Base, Script or Region. The returned Coverage derives the value for Bases from Tags if no func or slice for []Base is specified. For other unspecified types the returned Coverage will return nil for the respective methods.

type Extension 

type Extension struct {
	// contains filtered or unexported fields
}

Extension is a single BCP 47 extension.

func ParseExtension 

func ParseExtension(s string) (e Extension, err error)

ParseExtension parses s as an extension and returns it on success.

func (Extension) String 

func (e Extension) String() string

String returns the string representation of the extension, including the type tag.

func (Extension) Tokens 

func (e Extension) Tokens() []string

Tokens returns the list of tokens of e.

func (Extension) Type 

func (e Extension) Type() byte

Type returns the one-byte extension type of e. It returns 0 for the zero exception.

type MatchOption 

type MatchOption func(*matcher)

A MatchOption configures a Matcher.

func PreferSameScript 

func PreferSameScript(preferSame bool) MatchOption

PreferSameScript will, in the absence of a match, result in the first preferred tag with the same script as a supported tag to match this supported tag. The default is currently true, but this may change in the future.

type Matcher 

type Matcher interface {
	Match(t ...Tag) (tag Tag, index int, c Confidence)
}

Matcher is the interface that wraps the Match method.

Match returns the best match for any of the given tags, along with a unique index associated with the returned tag and a confidence score.

Example

ExampleMatcher_bestMatch gives some examples of getting the best match of a set of tags to any of the tags of given set. 

package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	// This is the set of tags from which we want to pick the best match. These
	// can be, for example, the supported languages for some package.
	tags := []language.Tag{
		language.English,             // en
		language.BritishEnglish,      // en-GB
		language.French,              // fr
		language.Afrikaans,           // af
		language.BrazilianPortuguese, // pt-BR
		language.EuropeanPortuguese,  // pt-PT
		language.SimplifiedChinese,   // zh-Hans
		language.Raw.Make("iw-IL"),   // Hebrew from Israel
		language.Raw.Make("iw"),      // Hebrew
		language.Raw.Make("he"),      // Hebrew
	}
	m := language.NewMatcher(tags)

	// A simple match.
	fmt.Println(m.Match(language.Make("fr")))

	// Australian English is closer to British English than American English.
	// The resulting match is "en-GB-u-rg-auzzzz". The first language listed,
	// "en-GB", is the matched language. Next is the region override prefix
	// "-u-rg-", the region override "au", and the region override suffix "zzzz".
	// The region override is for things like currency, dates, and measurement
	// systems.
	fmt.Println(m.Match(language.Make("en-AU")))

	// Default to the first tag passed to the Matcher if there is no match.
	fmt.Println(m.Match(language.Make("ar")))

	// Get the default tag.
	fmt.Println(m.Match())

	fmt.Println("----")

	// We match SimplifiedChinese, but with Low confidence.
	fmt.Println(m.Match(language.TraditionalChinese))

	// British English is closer to Australian English than Traditional Chinese
	// to Simplified Chinese.
	fmt.Println(m.Match(language.TraditionalChinese, language.Make("en-AU")))

	fmt.Println("----")

	// In case a multiple variants of a language are available, the most spoken
	// variant is typically returned.
	fmt.Println(m.Match(language.Portuguese))

	// Pick the first value passed to Match in case of a tie.
	fmt.Println(m.Match(language.Dutch, language.Make("fr-BE"), language.Make("af-NA")))
	fmt.Println(m.Match(language.Dutch, language.Make("af-NA"), language.Make("fr-BE")))

	fmt.Println("----")

	// If a Matcher is initialized with a language and its deprecated version,
	// it will distinguish between them.
	fmt.Println(m.Match(language.Raw.Make("iw")))

	// However, for non-exact matches, it will treat deprecated versions as
	// equivalent and consider other factors first.
	fmt.Println(m.Match(language.Raw.Make("he-IL")))

	fmt.Println("----")

	// User settings passed to the Unicode extension are ignored for matching
	// and preserved in the returned tag.
	fmt.Println(m.Match(language.Make("de-u-co-phonebk"), language.Make("fr-u-cu-frf")))

	// Even if the matching language is different.
	fmt.Println(m.Match(language.Make("de-u-co-phonebk"), language.Make("br-u-cu-frf")))

	// If there is no matching language, the options of the first preferred tag are used.
	fmt.Println(m.Match(language.Make("de-u-co-phonebk")))

}

Output:

fr 2 Exact
en-GB-u-rg-auzzzz 1 High
en 0 No
en 0 No
----
zh-Hans 6 Low
en-GB-u-rg-auzzzz 1 High
----
pt-BR 4 Exact
fr-u-rg-bezzzz 2 High
af-u-rg-nazzzz 3 High
----
iw-IL 7 Exact
he-u-rg-ilzzzz 9 Exact
----
fr-u-cu-frf 2 Exact
fr-u-cu-frf 2 High
en-u-co-phonebk 0 No

func NewMatcher 

func NewMatcher(t []Tag, options ...MatchOption) Matcher

NewMatcher returns a Matcher that matches an ordered list of preferred tags against a list of supported tags based on written intelligibility, closeness of dialect, equivalence of subtags and various other rules. It is initialized with the list of supported tags. The first element is used as the default value in case no match is found.

Its Match method matches the first of the given Tags to reach a certain confidence threshold. The tags passed to Match should therefore be specified in order of preference. Extensions are ignored for matching.

The index returned by the Match method corresponds to the index of the matched tag in t, but is augmented with the Unicode extension ('u')of the corresponding preferred tag. This allows user locale options to be passed transparently.

type Region 

type Region struct {
	// contains filtered or unexported fields
}

Region is an ISO 3166-1 or UN M.49 code for representing countries and regions.

func EncodeM49 

func EncodeM49(r int) (Region, error)

EncodeM49 returns the Region for the given UN M.49 code. It returns an error if r is not a valid code.

func MustParseRegion 

func MustParseRegion(s string) Region

MustParseRegion is like ParseRegion, but panics if the given region cannot be parsed. It simplifies safe initialization of Region values.

func ParseRegion 

func ParseRegion(s string) (Region, error)

ParseRegion parses a 2- or 3-letter ISO 3166-1 or a UN M.49 code. It returns a ValueError if s is a well-formed but unknown region identifier or another error if another error occurred.

func (Region) Canonicalize 

func (r Region) Canonicalize() Region

Canonicalize returns the region or a possible replacement if the region is deprecated. It will not return a replacement for deprecated regions that are split into multiple regions.

func (Region) Contains 

func (r Region) Contains(c Region) bool

Contains returns whether Region c is contained by Region r. It returns true if c == r.

func (Region) ISO3 

func (r Region) ISO3() string

ISO3 returns the 3-letter ISO code of r. Note that not all regions have a 3-letter ISO code. In such cases this method returns "ZZZ".

func (Region) IsCountry 

func (r Region) IsCountry() bool

IsCountry returns whether this region is a country or autonomous area. This includes non-standard definitions from CLDR.

func (Region) IsGroup 

func (r Region) IsGroup() bool

IsGroup returns whether this region defines a collection of regions. This includes non-standard definitions from CLDR.

func (Region) IsPrivateUse 

func (r Region) IsPrivateUse() bool

IsPrivateUse reports whether r has the ISO 3166 User-assigned status. This may include private-use tags that are assigned by CLDR and used in this implementation. So IsPrivateUse and IsCountry can be simultaneously true.

func (Region) M49 

func (r Region) M49() int

M49 returns the UN M.49 encoding of r, or 0 if this encoding is not defined for r.

func (Region) String 

func (r Region) String() string

String returns the BCP 47 representation for the region. It returns "ZZ" for an unspecified region.

func (Region) TLD 

func (r Region) TLD() (Region, error)

TLD returns the country code top-level domain (ccTLD). UK is returned for GB. In all other cases it returns either the region itself or an error.

This method may return an error for a region for which there exists a canonical form with a ccTLD. To get that ccTLD canonicalize r first. The region will already be canonicalized it was obtained from a Tag that was obtained using any of the default methods.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	us := language.MustParseRegion("US")
	gb := language.MustParseRegion("GB")
	uk := language.MustParseRegion("UK")
	bu := language.MustParseRegion("BU")

	fmt.Println(us.TLD())
	fmt.Println(gb.TLD())
	fmt.Println(uk.TLD())
	fmt.Println(bu.TLD())

	fmt.Println(us.Canonicalize().TLD())
	fmt.Println(gb.Canonicalize().TLD())
	fmt.Println(uk.Canonicalize().TLD())
	fmt.Println(bu.Canonicalize().TLD())
}

Output:

US <nil>
UK <nil>
UK <nil>
ZZ language: region is not a valid ccTLD
US <nil>
UK <nil>
UK <nil>
MM <nil>

type Script 

type Script struct {
	// contains filtered or unexported fields
}

Script is a 4-letter ISO 15924 code for representing scripts. It is idiomatically represented in title case.

func MustParseScript 

func MustParseScript(s string) Script

MustParseScript is like ParseScript, but panics if the given script cannot be parsed. It simplifies safe initialization of Script values.

func ParseScript 

func ParseScript(s string) (Script, error)

ParseScript parses a 4-letter ISO 15924 code. It returns a ValueError if s is a well-formed but unknown script identifier or another error if another error occurred.

func (Script) IsPrivateUse 

func (s Script) IsPrivateUse() bool

IsPrivateUse reports whether this script code is reserved for private use.

func (Script) String 

func (s Script) String() string

String returns the script code in title case. It returns "Zzzz" for an unspecified script.

type Tag 

type Tag compact.Tag

Tag represents a BCP 47 language tag. It is used to specify an instance of a specific language or locale. All language tag values are guaranteed to be well-formed.

Example (Values) 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	us := language.MustParseRegion("US")
	en := language.MustParseBase("en")

	lang, _, region := language.AmericanEnglish.Raw()
	fmt.Println(lang == en, region == us)

	lang, _, region = language.BritishEnglish.Raw()
	fmt.Println(lang == en, region == us)

	// Tags can be compared for exact equivalence using '=='.
	en_us, _ := language.Compose(en, us)
	fmt.Println(en_us == language.AmericanEnglish)

}

Output:

true true
true false
true

var (
	Und Tag = Tag{}

	Afrikaans            Tag = Tag(compact.Afrikaans)
	Amharic              Tag = Tag(compact.Amharic)
	Arabic               Tag = Tag(compact.Arabic)
	ModernStandardArabic Tag = Tag(compact.ModernStandardArabic)
	Azerbaijani          Tag = Tag(compact.Azerbaijani)
	Bulgarian            Tag = Tag(compact.Bulgarian)
	Bengali              Tag = Tag(compact.Bengali)
	Catalan              Tag = Tag(compact.Catalan)
	Czech                Tag = Tag(compact.Czech)
	Danish               Tag = Tag(compact.Danish)
	German               Tag = Tag(compact.German)
	Greek                Tag = Tag(compact.Greek)
	English              Tag = Tag(compact.English)
	AmericanEnglish      Tag = Tag(compact.AmericanEnglish)
	BritishEnglish       Tag = Tag(compact.BritishEnglish)
	Spanish              Tag = Tag(compact.Spanish)
	EuropeanSpanish      Tag = Tag(compact.EuropeanSpanish)
	LatinAmericanSpanish Tag = Tag(compact.LatinAmericanSpanish)
	Estonian             Tag = Tag(compact.Estonian)
	Persian              Tag = Tag(compact.Persian)
	Finnish              Tag = Tag(compact.Finnish)
	Filipino             Tag = Tag(compact.Filipino)
	French               Tag = Tag(compact.French)
	CanadianFrench       Tag = Tag(compact.CanadianFrench)
	Gujarati             Tag = Tag(compact.Gujarati)
	Hebrew               Tag = Tag(compact.Hebrew)
	Hindi                Tag = Tag(compact.Hindi)
	Croatian             Tag = Tag(compact.Croatian)
	Hungarian            Tag = Tag(compact.Hungarian)
	Armenian             Tag = Tag(compact.Armenian)
	Indonesian           Tag = Tag(compact.Indonesian)
	Icelandic            Tag = Tag(compact.Icelandic)
	Italian              Tag = Tag(compact.Italian)
	Japanese             Tag = Tag(compact.Japanese)
	Georgian             Tag = Tag(compact.Georgian)
	Kazakh               Tag = Tag(compact.Kazakh)
	Khmer                Tag = Tag(compact.Khmer)
	Kannada              Tag = Tag(compact.Kannada)
	Korean               Tag = Tag(compact.Korean)
	Kirghiz              Tag = Tag(compact.Kirghiz)
	Lao                  Tag = Tag(compact.Lao)
	Lithuanian           Tag = Tag(compact.Lithuanian)
	Latvian              Tag = Tag(compact.Latvian)
	Macedonian           Tag = Tag(compact.Macedonian)
	Malayalam            Tag = Tag(compact.Malayalam)
	Mongolian            Tag = Tag(compact.Mongolian)
	Marathi              Tag = Tag(compact.Marathi)
	Malay                Tag = Tag(compact.Malay)
	Burmese              Tag = Tag(compact.Burmese)
	Nepali               Tag = Tag(compact.Nepali)
	Dutch                Tag = Tag(compact.Dutch)
	Norwegian            Tag = Tag(compact.Norwegian)
	Punjabi              Tag = Tag(compact.Punjabi)
	Polish               Tag = Tag(compact.Polish)
	Portuguese           Tag = Tag(compact.Portuguese)
	BrazilianPortuguese  Tag = Tag(compact.BrazilianPortuguese)
	EuropeanPortuguese   Tag = Tag(compact.EuropeanPortuguese)
	Romanian             Tag = Tag(compact.Romanian)
	Russian              Tag = Tag(compact.Russian)
	Sinhala              Tag = Tag(compact.Sinhala)
	Slovak               Tag = Tag(compact.Slovak)
	Slovenian            Tag = Tag(compact.Slovenian)
	Albanian             Tag = Tag(compact.Albanian)
	Serbian              Tag = Tag(compact.Serbian)
	SerbianLatin         Tag = Tag(compact.SerbianLatin)
	Swedish              Tag = Tag(compact.Swedish)
	Swahili              Tag = Tag(compact.Swahili)
	Tamil                Tag = Tag(compact.Tamil)
	Telugu               Tag = Tag(compact.Telugu)
	Thai                 Tag = Tag(compact.Thai)
	Turkish              Tag = Tag(compact.Turkish)
	Ukrainian            Tag = Tag(compact.Ukrainian)
	Urdu                 Tag = Tag(compact.Urdu)
	Uzbek                Tag = Tag(compact.Uzbek)
	Vietnamese           Tag = Tag(compact.Vietnamese)
	Chinese              Tag = Tag(compact.Chinese)
	SimplifiedChinese    Tag = Tag(compact.SimplifiedChinese)
	TraditionalChinese   Tag = Tag(compact.TraditionalChinese)
	Zulu                 Tag = Tag(compact.Zulu)
)

func Compose 

func Compose(part ...interface{}) (t Tag, err error)

Compose creates a Tag from individual parts, which may be of type Tag, Base, Script, Region, Variant, []Variant, Extension, []Extension or error. If a Base, Script or Region or slice of type Variant or Extension is passed more than once, the latter will overwrite the former. Variants and Extensions are accumulated, but if two extensions of the same type are passed, the latter will replace the former. For -u extensions, though, the key-type pairs are added, where later values overwrite older ones. A Tag overwrites all former values and typically only makes sense as the first argument. The resulting tag is returned after canonicalizing using the Default CanonType. If one or more errors are encountered, one of the errors is returned.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	nl, _ := language.ParseBase("nl")
	us, _ := language.ParseRegion("US")
	de := language.Make("de-1901-u-co-phonebk")
	jp := language.Make("ja-JP")
	fi := language.Make("fi-x-ing")

	u, _ := language.ParseExtension("u-nu-arabic")
	x, _ := language.ParseExtension("x-piglatin")

	// Combine a base language and region.
	fmt.Println(language.Compose(nl, us))
	// Combine a base language and extension.
	fmt.Println(language.Compose(nl, x))
	// Replace the region.
	fmt.Println(language.Compose(jp, us))
	// Combine several tags.
	fmt.Println(language.Compose(us, nl, u))

	// Replace the base language of a tag.
	fmt.Println(language.Compose(de, nl))
	fmt.Println(language.Compose(de, nl, u))
	// Remove the base language.
	fmt.Println(language.Compose(de, language.Base{}))
	// Remove all variants.
	fmt.Println(language.Compose(de, []language.Variant{}))
	// Remove all extensions.
	fmt.Println(language.Compose(de, []language.Extension{}))
	fmt.Println(language.Compose(fi, []language.Extension{}))
	// Remove all variants and extensions.
	fmt.Println(language.Compose(de.Raw()))

	// An error is gobbled or returned if non-nil.
	fmt.Println(language.Compose(language.ParseRegion("ZA")))
	fmt.Println(language.Compose(language.ParseRegion("HH")))

	// Compose uses the same Default canonicalization as Make.
	fmt.Println(language.Compose(language.Raw.Parse("en-Latn-UK")))

	// Call compose on a different CanonType for different results.
	fmt.Println(language.All.Compose(language.Raw.Parse("en-Latn-UK")))

}

Output:

nl-US <nil>
nl-x-piglatin <nil>
ja-US <nil>
nl-US-u-nu-arabic <nil>
nl-1901-u-co-phonebk <nil>
nl-1901-u-co-phonebk-nu-arabic <nil>
und-1901-u-co-phonebk <nil>
de-u-co-phonebk <nil>
de-1901 <nil>
fi <nil>
de <nil>
und-ZA <nil>
und language: subtag "HH" is well-formed but unknown
en-Latn-GB <nil>
en-GB <nil>

func Make 

func Make(s string) Tag

Make is a convenience wrapper for Parse that omits the error. In case of an error, a sensible default is returned.

func MatchStrings 

func MatchStrings(m Matcher, lang ...string) (tag Tag, index int)

MatchStrings parses and matches the given strings until one of them matches the language in the Matcher. A string may be an Accept-Language header as handled by ParseAcceptLanguage. The default language is returned if no other language matched.

Example 
package main

import (
	"fmt"
	"net/http"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	// languages supported by this service:
	matcher := language.NewMatcher([]language.Tag{
		language.English, language.Dutch, language.German,
	})

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		lang, _ := r.Cookie("lang")
		tag, _ := language.MatchStrings(matcher, lang.String(), r.Header.Get("Accept-Language"))

		fmt.Println("User language:", tag)
	})
}

Output:

func MustParse 

func MustParse(s string) Tag

MustParse is like Parse, but panics if the given BCP 47 tag cannot be parsed. It simplifies safe initialization of Tag values.

func Parse 

func Parse(s string) (t Tag, err error)

Parse parses the given BCP 47 string and returns a valid Tag. If parsing failed it returns an error and any part of the tag that could be parsed. If parsing succeeded but an unknown value was found, it returns ValueError. The Tag returned in this case is just stripped of the unknown value. All other values are preserved. It accepts tags in the BCP 47 format and extensions to this standard defined in https://www.unicode.org/reports/tr35/#Unicode_Language_and_Locale_Identifiers. The resulting tag is canonicalized using the default canonicalization type.

Example (Errors) 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	for _, s := range []string{"Foo", "Bar", "Foobar"} {
		_, err := language.Parse(s)
		if err != nil {
			if inv, ok := err.(language.ValueError); ok {
				fmt.Println(inv.Subtag())
			} else {
				fmt.Println(s)
			}
		}
	}
	for _, s := range []string{"en", "aa-Uuuu", "AC", "ac-u"} {
		_, err := language.Parse(s)
		switch e := err.(type) {
		case language.ValueError:
			fmt.Printf("%s: culprit %q\n", s, e.Subtag())
		case nil:
			// No error.
		default:
			// A syntax error.
			fmt.Printf("%s: ill-formed\n", s)
		}
	}
}

Output:

foo
Foobar
aa-Uuuu: culprit "Uuuu"
AC: culprit "ac"
ac-u: ill-formed

func ParseAcceptLanguage 

func ParseAcceptLanguage(s string) (tag []Tag, q []float32, err error)

ParseAcceptLanguage parses the contents of an Accept-Language header as defined in http://www.ietf.org/rfc/rfc2616.txt and returns a list of Tags and a list of corresponding quality weights. It is more permissive than RFC 2616 and may return non-nil slices even if the input is not valid. The Tags will be sorted by highest weight first and then by first occurrence. Tags with a weight of zero will be dropped. An error will be returned if the input could not be parsed.

Example 
package main

import (
	"fmt"
	"net/http"
	"strings"

	"github.com/go-corelibs/x-text/language"
)

// matcher is a language.Matcher configured for all supported languages.
var matcher = language.NewMatcher([]language.Tag{
	language.BritishEnglish,
	language.Norwegian,
	language.German,
})

// handler is an http.HandlerFunc.
func handler(w http.ResponseWriter, r *http.Request) {
	t, q, err := language.ParseAcceptLanguage(r.Header.Get("Accept-Language"))
	// We ignore the error: the default language will be selected for t == nil.
	tag, _, _ := matcher.Match(t...)
	fmt.Printf("%17v (t: %6v; q: %3v; err: %v)\n", tag, t, q, err)
}

func main() {
	for _, al := range []string{
		"nn;q=0.3, en-us;q=0.8, en,",
		"gsw, en;q=0.7, en-US;q=0.8",
		"gsw, nl, da",
		"invalid",
	} {
		// Create dummy request with Accept-Language set and pass it to handler.
		r, _ := http.NewRequest("GET", "example.com", strings.NewReader("Hello"))
		r.Header.Set("Accept-Language", al)
		handler(nil, r)
	}

}

Output:

            en-GB (t: [    en  en-US     nn]; q: [  1 0.8 0.3]; err: <nil>)
en-GB-u-rg-uszzzz (t: [   gsw  en-US     en]; q: [  1 0.8 0.7]; err: <nil>)
               de (t: [   gsw     nl     da]; q: [  1   1   1]; err: <nil>)
            en-GB (t: []; q: []; err: language: tag is not well-formed)

func (Tag) Base 

func (t Tag) Base() (Base, Confidence)

Base returns the base language of the language tag. If the base language is unspecified, an attempt will be made to infer it from the context. It uses a variant of CLDR's Add Likely Subtags algorithm. This is subject to change.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	fmt.Println(language.Make("und").Base())
	fmt.Println(language.Make("und-US").Base())
	fmt.Println(language.Make("und-NL").Base())
	fmt.Println(language.Make("und-419").Base()) // Latin America
	fmt.Println(language.Make("und-ZZ").Base())
}

Output:

en Low
en High
nl High
es Low
en Low

func (Tag) Extension 

func (t Tag) Extension(x byte) (ext Extension, ok bool)

Extension returns the extension of type x for tag t. It will return false for ok if t does not have the requested extension. The returned extension will be invalid in this case.

func (Tag) Extensions 

func (t Tag) Extensions() []Extension

Extensions returns all extensions of t.

func (Tag) IsRoot 

func (t Tag) IsRoot() bool

IsRoot returns true if t is equal to language "und".

func (*Tag) MarshalBinary 

func (t *Tag) MarshalBinary() (data []byte, err error)

MarshalBinary was added by Go-Enjin

func (Tag) MarshalText 

func (t Tag) MarshalText() (text []byte, err error)

MarshalText implements encoding.TextMarshaler.

func (Tag) Parent 

func (t Tag) Parent() Tag

Parent returns the CLDR parent of t. In CLDR, missing fields in data for a specific language are substituted with fields from the parent language. The parent for a language may change for newer versions of CLDR.

Parent returns a tag for a less specific language that is mutually intelligible or Und if there is no such language. This may not be the same as simply stripping the last BCP 47 subtag. For instance, the parent of "zh-TW" is "zh-Hant", and the parent of "zh-Hant" is "und".

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	p := func(tag string) {
		fmt.Printf("parent(%v): %v\n", tag, language.Make(tag).Parent())
	}
	p("zh-CN")

	// Australian English inherits from World English.
	p("en-AU")

	// If the tag has a different maximized script from its parent, a tag with
	// this maximized script is inserted. This allows different language tags
	// which have the same base language and script in common to inherit from
	// a common set of settings.
	p("zh-HK")

	// If the maximized script of the parent is not identical, CLDR will skip
	// inheriting from it, as it means there will not be many entries in common
	// and inheriting from it is nonsensical.
	p("zh-Hant")

	// The parent of a tag with variants and extensions is the tag with all
	// variants and extensions removed.
	p("de-1994-u-co-phonebk")

	// Remove default script.
	p("de-Latn-LU")

}

Output:

parent(zh-CN): zh
parent(en-AU): en-001
parent(zh-HK): zh-Hant
parent(zh-Hant): und
parent(de-1994-u-co-phonebk): de
parent(de-Latn-LU): de

func (Tag) Raw 

func (t Tag) Raw() (b Base, s Script, r Region)

Raw returns the raw base language, script and region, without making an attempt to infer their values.

func (Tag) Region 

func (t Tag) Region() (Region, Confidence)

Region returns the region for the language tag. If it was not explicitly given, it will infer a most likely candidate from the context. It uses a variant of CLDR's Add Likely Subtags algorithm. This is subject to change.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	ru := language.Make("ru")
	en := language.Make("en")
	fmt.Println(ru.Region())
	fmt.Println(en.Region())
}

Output:

RU Low
US Low

func (Tag) Script 

func (t Tag) Script() (Script, Confidence)

Script infers the script for the language tag. If it was not explicitly given, it will infer a most likely candidate. If more than one script is commonly used for a language, the most likely one is returned with a low confidence indication. For example, it returns (Cyrl, Low) for Serbian. If a script cannot be inferred (Zzzz, No) is returned. We do not use Zyyy (undetermined) as one would suspect from the IANA registry for BCP 47. In a Unicode context Zyyy marks common characters (like 1, 2, 3, '.', etc.) and is therefore more like multiple scripts. See https://www.unicode.org/reports/tr24/#Values for more details. Zzzz is also used for unknown value in CLDR. (Zzzz, Exact) is returned if Zzzz was explicitly specified. Note that an inferred script is never guaranteed to be the correct one. Latin is almost exclusively used for Afrikaans, but Arabic has been used for some texts in the past. Also, the script that is commonly used may change over time. It uses a variant of CLDR's Add Likely Subtags algorithm. This is subject to change.

Example 
package main

import (
	"fmt"

	"github.com/go-corelibs/x-text/language"
)

func main() {
	en := language.Make("en")
	sr := language.Make("sr")
	sr_Latn := language.Make("sr_Latn")
	fmt.Println(en.Script())
	fmt.Println(sr.Script())
	// Was a script explicitly specified?
	_, c := sr.Script()
	fmt.Println(c == language.Exact)
	_, c = sr_Latn.Script()
	fmt.Println(c == language.Exact)
}

Output:

Latn High
Cyrl Low
false
true

func (Tag) SetTypeForKey 

func (t Tag) SetTypeForKey(key, value string) (Tag, error)

SetTypeForKey returns a new Tag with the key set to type, where key and type are of the allowed values defined for the Unicode locale extension ('u') in https://www.unicode.org/reports/tr35/#Unicode_Language_and_Locale_Identifiers. An empty value removes an existing pair with the same key.

func (Tag) String 

func (t Tag) String() string

String returns the canonical string representation of the language tag.

func (Tag) TypeForKey 

func (t Tag) TypeForKey(key string) string

TypeForKey returns the type associated with the given key, where key and type are of the allowed values defined for the Unicode locale extension ('u') in https://www.unicode.org/reports/tr35/#Unicode_Language_and_Locale_Identifiers. TypeForKey will traverse the inheritance chain to get the correct value.

If there are multiple types associated with a key, only the first will be returned. If there is no type associated with a key, it returns the empty string.

func (*Tag) UnmarshalBinary 

func (t *Tag) UnmarshalBinary(data []byte) (err error)

MarshalBinary was added by Go-Enjin

func (*Tag) UnmarshalText 

func (t *Tag) UnmarshalText(text []byte) error

UnmarshalText implements encoding.TextUnmarshaler.

func (Tag) Variants 

func (t Tag) Variants() []Variant

Variants returns the variants specified explicitly for this language tag. or nil if no variant was specified.

type ValueError 

type ValueError interface {
	error

	// Subtag returns the subtag for which the error occurred.
	Subtag() string
}

ValueError is returned by any of the parsing functions when the input is well-formed but the respective subtag is not recognized as a valid value.

type Variant 

type Variant struct {
	// contains filtered or unexported fields
}

Variant represents a registered variant of a language as defined by BCP 47.

func ParseVariant 

func ParseVariant(s string) (Variant, error)

ParseVariant parses and returns a Variant. An error is returned if s is not a valid variant.

func (Variant) String 

func (v Variant) String() string

String returns the string representation of the variant.

Source Files

View all Source files

Directories

Path Synopsis
Package display provides display names for languages, scripts and regions in a requested language.
 Package display provides display names for languages, scripts and regions in a requested language.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.