charset

Documentation

Overview

Package charset provides common text encodings for HTML documents.

The mapping from encoding labels to encodings is defined at https://encoding.spec.whatwg.org/.

Index

• func DetermineEncoding(content []byte, contentType string) (e encoding.Encoding, name string, certain bool)
• func Lookup(label string) (e encoding.Encoding, name string)
• func NewReader(r io.Reader, contentType string) (io.Reader, error)
• func NewReaderLabel(label string, input io.Reader) (io.Reader, error)

Functions

func DetermineEncoding

func DetermineEncoding(content []byte, contentType string) (e encoding.Encoding, name string, certain bool)

DetermineEncoding determines the encoding of an HTML document by examining up to the first 1024 bytes of content and the declared Content-Type.

See http://www.whatwg.org/specs/web-apps/current-work/multipage/parsing.html#determining-the-character-encoding

func Lookup

func Lookup(label string) (e encoding.Encoding, name string)

Lookup returns the encoding with the specified label, and its canonical name. It returns nil and the empty string if label is not one of the standard encodings for HTML. Matching is case-insensitive and ignores leading and trailing whitespace. Encoders will use HTML escape sequences for runes that are not supported by the character set.

func NewReader

func NewReader(r io.Reader, contentType string) (io.Reader, error)

NewReader returns an io.Reader that converts the content of r to UTF-8. It calls DetermineEncoding to find out what r's encoding is.

func NewReaderLabel

func NewReaderLabel(label string, input io.Reader) (io.Reader, error)

NewReaderLabel returns a reader that converts from the specified charset to UTF-8. It uses Lookup to find the encoding that corresponds to label, and returns an error if Lookup returns nil. It is suitable for use as encoding/xml.Decoder's CharsetReader function.