utfmgr

Documentation

Overview

Package utfmgr Created by RTT. Author: teocci@yandex.com on 2021-Oct-22

Package utfmgr Created by RTT. Author: teocci@yandex.com on 2021-Oct-22

Package utfmgr Created by RTT. Author: teocci@yandex.com on 2021-Oct-22

Index

• Constants
• func BytesReader(b []byte, d EncodingHint) io.Reader
• func CloseFile() func(f UTFReadCloser)
• func ReadFile(name string, d EncodingHint) ([]byte, error)
• type EncodingHint
• type UTFReadCloser
  • func NewReader(r io.Reader, d EncodingHint) UTFReadCloser
  • func OpenFile(name string, d EncodingHint) (UTFReadCloser, error)
• type UTFScanCloser
  • func NewScanner(name string, d EncodingHint) (UTFScanCloser, error)

Constants

const (
	// UTF8 indicates the specified encoding.
	UTF8 EncodingHint = iota
	// UTF16LE indicates the specified encoding.
	UTF16LE
	// UTF16BE indicates the specified encoding.
	UTF16BE
	// WINDOWS indicates that the file came from a MS-Windows system
	WINDOWS = UTF16LE
	// POSIX indicates that the file came from Unix or Unix-like systems
	POSIX = UTF8
	// HTML5 indicates that the file came from the web
	// This is recommended by the W3C for use in HTML 5:
	// "For compatibility with deployed content, the byte order
	// mark (also known as BOM) is considered more authoritative
	// than anything else." http://www.w3.org/TR/encoding/#specification-hooks
	HTML5 = UTF8
)

Functions

func BytesReader

func BytesReader(b []byte, d EncodingHint) io.Reader

BytesReader is a convenience function that takes a []byte and decodes them to UTF-8.

func CloseFile

func CloseFile() func(f UTFReadCloser)

func ReadFile

func ReadFile(name string, d EncodingHint) ([]byte, error)

ReadFile is the equivalent of ioutil.ReadFile()

Types

type EncodingHint

type EncodingHint int

EncodingHint indicates the file's encoding if there is no BOM.

type UTFReadCloser

type UTFReadCloser interface {
	Read(p []byte) (n int, err error)
	Close() error
}

UTFReadCloser describes a ReadCloser structure for this package.

func NewReader

func NewReader(r io.Reader, d EncodingHint) UTFReadCloser

NewReader wraps a Reader to decode Unicode to UTF-8 as it reads.

func OpenFile

func OpenFile(name string, d EncodingHint) (UTFReadCloser, error)

OpenFile is the equivalent of os.Open().

type UTFScanCloser

type UTFScanCloser interface {
	Buffer(buf []byte, max int)
	Bytes() []byte
	Err() error
	Scan() bool
	Split(split bufio.SplitFunc)
	Text() string
	Close() error
}

UTFScanCloser describes a new utfmgr ScanCloser structure. It's similar to ReadCloser, but with a scanner instead of a reader.

func NewScanner

func NewScanner(name string, d EncodingHint) (UTFScanCloser, error)

NewScanner is a convenience function that takes a filename and returns a scanner.