subtags

Documentation

Overview

Package subtags can be used to cache and query the IANA language subtags registry.

Index

• type Entry
• type LanguageSubtags
  • func New(opts ...Option) *LanguageSubtags
  • func NewWithLoad(opts ...Option) (*LanguageSubtags, error)
  • func (c *LanguageSubtags) GetExtLanguage(in string) *Entry
  • func (c *LanguageSubtags) GetGrandfathered(in string) *Entry
  • func (c *LanguageSubtags) GetLanguage(in string) *Entry
  • func (c *LanguageSubtags) GetRedundant(in string) *Entry
  • func (c *LanguageSubtags) GetRegion(in string) *Entry
  • func (c *LanguageSubtags) GetScript(in string) *Entry
  • func (c *LanguageSubtags) GetVariant(in string) *Entry
  • func (c *LanguageSubtags) Grandfathered(in string) (bool, *Entry)
  • func (c *LanguageSubtags) IsGrandfathered(in string) bool
  • func (c *LanguageSubtags) IsRedundant(in string) bool
  • func (c *LanguageSubtags) Load() error
  • func (c *LanguageSubtags) Redundant(in string) (bool, *Entry)
  • func (c *LanguageSubtags) ReplaceGrandfathered(in string) (found bool, replaced bool, out string, gf *Entry)
  • func (c *LanguageSubtags) Verify(in string, opts ...VerifyOption) (bool, *LanguageTag, error)
  • func (c *LanguageSubtags) VerifyTag(l *LanguageTag, opts ...VerifyOption) (bool, *LanguageTag, error)
• type LanguageTag
  • func Parse(in string) *LanguageTag
  • func ParseStrict(in string) *LanguageTag
  • func (l *LanguageTag) BCP47() *LanguageTag
  • func (l *LanguageTag) FixRedundancy(c *LanguageSubtags) (bool, *LanguageTag)
  • func (l *LanguageTag) String() string
• type Option
  • func WithFile(path string) Option
  • func WithLogger(log *slog.Logger) Option
  • func WithURL(url string) Option
• type VerifyOption
  • func ForceBCP47() VerifyOption
  • func ReplaceEmptyLanguage() VerifyOption
  • func ReplaceGrandfathered() VerifyOption
  • func ReplaceRedundant() VerifyOption
  • func StrictParsing() VerifyOption
  • func SuppressScript() VerifyOption
  • func WithLog(verbose bool) VerifyOption

Types

type Entry

type Entry struct {
	Added          *time.Time `json:"added,omitempty"`
	Deprecated     *time.Time `json:"deprecated,omitempty"`
	Tag            string     `json:"tag,omitempty"`
	Subtag         string     `json:"subtag,omitempty"`
	SuppressScript string     `json:"suppressScript,omitempty"`
	Scope          string     `json:"scope,omitempty"`
	PreferredValue string     `json:"preferredValue,omitempty"`
	MacroLanguage  string     `json:"macroLanguage,omitempty"`
	Comments       string     `json:"comments,omitempty"`
	Descriptions   []string   `json:"descriptions,omitempty"`
	Prefixes       []string   `json:"prefixes,omitempty"`
	// contains filtered or unexported fields
}

Entry represents an IANA language subtag entry.

type LanguageSubtags

type LanguageSubtags struct {
	// contains filtered or unexported fields
}

LanguageSubtags is a cache for entries from the IANA language subtag registry.

func New

func New(opts ...Option) *LanguageSubtags

New instantiates a new (empty) cache for working with the IANA language subtag registry.

func NewWithLoad

func NewWithLoad(opts ...Option) (*LanguageSubtags, error)

NewWithLoad instantiates a new cache for working with the IANA language subtag registry, and loads the data from the given file or (default) URL.

func (*LanguageSubtags) GetExtLanguage

func (c *LanguageSubtags) GetExtLanguage(in string) *Entry

GetExtLanguage returns the extended language entry if the given code exists, or nil otherwise.

Comparison is done case insensitive.

func (*LanguageSubtags) GetGrandfathered

func (c *LanguageSubtags) GetGrandfathered(in string) *Entry

GetGrandfathered returns the grandfathered entry if the given code exists, or nil otherwise.

All underscore characters are converted to a hyphen sign before comparing. Comparison is done case insensitive.

func (*LanguageSubtags) GetLanguage

func (c *LanguageSubtags) GetLanguage(in string) *Entry

GetLanguage returns the language entry if the given code exists, or nil otherwise.

Comparison is done case insensitive.

func (*LanguageSubtags) GetRedundant

func (c *LanguageSubtags) GetRedundant(in string) *Entry

GetRedundant returns the redundant entry if the given code exists, or nil otherwise.

All underscore characters are converted to a hyphen sign before comparing. Comparison is done case insensitive.

func (*LanguageSubtags) GetRegion

func (c *LanguageSubtags) GetRegion(in string) *Entry

GetRegion returns the region entry if the given code exists, or nil otherwise.

Comparison is done case insensitive.

func (*LanguageSubtags) GetScript

func (c *LanguageSubtags) GetScript(in string) *Entry

GetScript returns the script entry if the given code exists, or nil otherwise.

Comparison is done case insensitive.

func (*LanguageSubtags) GetVariant

func (c *LanguageSubtags) GetVariant(in string) *Entry

GetVariant returns the variant entry if the given code exists, or nil otherwise.

Comparison is done case insensitive.

func (*LanguageSubtags) Grandfathered

func (c *LanguageSubtags) Grandfathered(in string) (bool, *Entry)

Grandfathered returns true if the language tag is equal to or starts with a grandfathered tag.

It will also return the corresponding entry from the IANA language subtag registry. This allows you to figure out what to do with the input:

1. PreferredValue contains a value -> replace the grandfathered tag with this value.
2. Deprecated contains a date in the past -> tag is no longer valid.
3. View the Comments -> it may give further instructions.
4. In all other cases, consult your favorite search engine :)

The comparison is done case insensitive.

The returned entry is the cached version. DO NOT modify it!

func (*LanguageSubtags) IsGrandfathered

func (c *LanguageSubtags) IsGrandfathered(in string) bool

IsGrandfathered returns true if the language tag is equal to or starts with a grandfathered tag.

The comparison is done case insensitive.

func (*LanguageSubtags) IsRedundant

func (c *LanguageSubtags) IsRedundant(in string) bool

IsRedundant returns true if the language tag is equal to or starts with a grandfathered tag.

The comparison is done case insensitive.

func (*LanguageSubtags) Load

func (c *LanguageSubtags) Load() error

Load (re)loads the cache from the given file (`WithFile` option) or from the internet using http.

The default URL is: https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry In the (unlikely) event IANA decides to move the registry to another URL, you can use the `WithURL` option.

Use the `WithFile` option if you want to have full control of the IANA registry data yourself.

The options need to be provided when the cache is instantiated with `New()`, and cannot be changed afterwards.

func (*LanguageSubtags) Redundant

func (c *LanguageSubtags) Redundant(in string) (bool, *Entry)

Redundant returns true if the language tag is equal to or starts with a redundant tag.

It will also return the corresponding entry from the IANA language subtag registry. This allows you to figure out what to do with the input. If PreferredValue has a value, it can be used as a replacement. In other cases the primary language subtag may be sufficient, but check with your favorite search engine to make sure!

The comparison is done case insensitive.

The returned entry is the cached version. DO NOT modify it!

func (*LanguageSubtags) ReplaceGrandfathered

func (c *LanguageSubtags) ReplaceGrandfathered(in string) (found bool, replaced bool, out string, gf *Entry)

ReplaceGrandfathered checks if the given language tag is equal to or starts with a grandfathered tag.

If it is, it will attempt to replace the grandfathered part with the recommended replacement.

`found` is true if the input tag is found to be grandfathered. Note that if `found` is true, it does not necessarily mean the tag has been fixed.

`replaced` is true if the grandfathered part could be replaced as recommended. In all other cases it will be false.

`out` will contain the modified input if it was found to be grandfathered and a replacement was possible. In all other cases it will be same as the input string.

`gf` will contain the grandfathered entry if it was found, or nil otherwise. This is the cached version. DO NOT modify it!

If `found` is true but `replaced` is false, it is up to you to figure out how to "fix" the language tag. It is quite possible this will require manual intervention.

func (*LanguageSubtags) Verify

func (c *LanguageSubtags) Verify(in string, opts ...VerifyOption) (bool, *LanguageTag, error)

Verify verifies the given tag against the IANA language subtag registry.

The function returns true if the tag was successfully parsed and verified. It returns false and an error containing a description of all problems.

The function also returns the parsed tag as a structure, even if it failed verification.

func (*LanguageSubtags) VerifyTag

func (c *LanguageSubtags) VerifyTag(l *LanguageTag, opts ...VerifyOption) (bool, *LanguageTag, error)

VerifyTag verifies the given language tag structure against the IANA language subtag registry.

The function returns true if the tag was successfully parsed and verified. It returns false and an error containing a description of all problems.

The function also returns the potentially modified input, even if it failed verification.

type LanguageTag

type LanguageTag struct {
	Input        string          `json:"input"`
	Language     string          `json:"language,omitempty"`
	ExtLanguages []string        `json:"extLanguages,omitempty"`
	Script       string          `json:"script,omitempty"`
	Region       string          `json:"region,omitempty"`
	Variants     []string        `json:"variants,omitempty"`
	Extensions   map[byte]string `json:"extensions,omitempty"`
	Errors       []error         `json:"errors,omitempty"`
	// contains filtered or unexported fields
}

LanguageTag contains the elements of a parsed language tag.

func Parse

func Parse(in string) *LanguageTag

Parse parses a language tag.

It does not validate the individual fields. Use the `Verify` or `VerifyTag` functions for validation.

A badly formed tag may fail to render some subtags correctly.

The `Errors` slice in the returned structure will contain any problems encountered during parsing.

Even though this function will parse badly formed grandfathered tags correctly, it is advisable to check and fix grandfathered tags before using this function. It will prevent the complexity of trying to solve the grandfather process from the returned set of subtags.

The returned language tag structure has a function (`FixRedundancy`) to fix redundant tags after parsing.

Use the `String` function of the returned tag to "rebuild" it. Use the `BCP47` function to canonicalize the subtags.

func ParseStrict

func ParseStrict(in string) *LanguageTag

ParseStrict works exactly like `Parse`, but it will fail on badly formed grandfathered tags.

func (*LanguageTag) BCP47

func (l *LanguageTag) BCP47() *LanguageTag

BCP47 returns a newly instantiated language tag as a canonical copy of the original.

This means that all subtags will be converted to the preferred casing as defined in BCP47. The input and any extensions will be copied verbatim. Any errors reported in the original are not copied.

func (*LanguageTag) FixRedundancy

func (l *LanguageTag) FixRedundancy(c *LanguageSubtags) (bool, *LanguageTag)

FixRedundancy can be used to remove redundant subtags, according to the rules in the IANA language subtag registry.

The function returns true if a redundant subtag was removed. If a subtag was removed, it also returns a newly instantiated language tag with the redundant subtags removed. E.g. "zh-yue" will become "yue", etc.

Redundant tags that do not have a preferred value are left untouched, as they are already well-formed.

func (*LanguageTag) String

func (l *LanguageTag) String() string

String implements the Stringer interface.

This function "rebuilds" the tag from the various subtags and extensions. It uses a dash ("-") as the separator character between subtags/extensions. If there are extensions, they will be added in alphabetical order of extension key, except for an 'x' extension which will always be added last.

type Option

type Option func(subtags *LanguageSubtags)

Option defines the function signature for options to instantiate a new language subtag cache.

func WithFile

func WithFile(path string) Option

WithFile defines a file to use instead of retrieving the IANA registry through the http protocol.

Use ths option if you want to have full control of the IANA registry data yourself.

func WithLogger

func WithLogger(log *slog.Logger) Option

WithLogger adds a logging option to the cache.

func WithURL

func WithURL(url string) Option

WithURL defines the URL to use when retrieving the IANA registry from the internet.

When not provided, the default is: https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry

type VerifyOption

type VerifyOption func(v *verifier)

VerifyOption is the function signature for options when verifying a tag.

func ForceBCP47

func ForceBCP47() VerifyOption

ForceBCP47 forces canonicalization of the subtags as defined by BCP47.

func ReplaceEmptyLanguage

func ReplaceEmptyLanguage() VerifyOption

ReplaceEmptyLanguage turns on checking and replacing an empty language subtag with 'und'.

This allows for verification of special tags without a language.

func ReplaceGrandfathered

func ReplaceGrandfathered() VerifyOption

ReplaceGrandfathered turns on testing and replacing of grandfathered tags.

This option has no effect on the `VerifyTag` function.

func ReplaceRedundant

func ReplaceRedundant() VerifyOption

ReplaceRedundant turns on testing and replacing of redundant tags.

func StrictParsing

func StrictParsing() VerifyOption

StrictParsing turns on strict parsing of the input tag.

See `ParseStrict` for more details.

This option has no effect on the `VerifyTag` function.

func SuppressScript

func SuppressScript() VerifyOption

SuppressScript turns on testing and removal of unnecessary scripts.

If the language subtag matches an entry the IANA language subtag registry, and this entry contains a `SuppressScript` value, the script subtag will be removed if it matches this value.

func WithLog

func WithLog(verbose bool) VerifyOption

WithLog turns on logging for the verification process.

This option has no effect if the cache was not initialized with a logger.