bom

README

BOM

Functions to work with BOM.

BOM is a byte order mark used in Unicode-based encodings.

Documentation

Index

• Constants
• func GetBOMForEncodingBOCU1(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingGB18030(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingSCSU(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF1(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF16BE(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF16LE(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF32BE(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF32LE(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF7(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF8(r *reader.Reader) (prefix []byte, err error)
• func GetBOMForEncodingUTF_EBCDIC(r *reader.Reader) (prefix []byte, err error)
• func IsEncoding(rs rs.ReaderSeeker, enc Encoding) (isEncoding bool, err error)
• func SkipBOMPrefix(rs rs.ReaderSeeker, enc Encoding) (newRS rs.ReaderSeeker, err error)
• type Encoding
  • func GetEncoding(rs rs.ReaderSeeker, shouldIncludeUnknownEncoding bool) (result []Encoding, err error)
  • func PossibleEncodings() []Encoding
  • func (e Encoding) IsKnown() bool
  • func (e Encoding) IsUnknown() bool

Constants

const (
	ErrUnknownEncoding = "unknown encoding: %v"
	ErrBOMIsNotFound   = "byte order mark is not found"
)

const (
	EncodingUnknown    = Encoding(0)
	EncodingUTF8       = Encoding(1)
	EncodingUTF16BE    = Encoding(2)
	EncodingUTF16LE    = Encoding(3)
	EncodingUTF32BE    = Encoding(4)
	EncodingUTF32LE    = Encoding(5)
	EncodingUTF7       = Encoding(6)
	EncodingUTF1       = Encoding(8)
	EncodingUTF_EBCDIC = Encoding(9)
	EncodingSCSU       = Encoding(10)
	EncodingBOCU1      = Encoding(11)
	EncodingGB18030    = Encoding(12)
)

Functions

func GetBOMForEncodingBOCU1

func GetBOMForEncodingBOCU1(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingBOCU1 tries to read the BOM of BOCU-1 encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingGB18030

func GetBOMForEncodingGB18030(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingGB18030 tries to read the BOM of GB18030 encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingSCSU

func GetBOMForEncodingSCSU(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingSCSU tries to read the BOM of SCSU encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF1

func GetBOMForEncodingUTF1(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF1 tries to read the BOM of UTF-1 encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF16BE

func GetBOMForEncodingUTF16BE(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF16BE tries to read the BOM of UTF-16 [BE] encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF16LE

func GetBOMForEncodingUTF16LE(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF16LE tries to read the BOM of UTF-16 [LE] encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF32BE

func GetBOMForEncodingUTF32BE(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF32BE tries to read the BOM of UTF-32 [BE] encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF32LE

func GetBOMForEncodingUTF32LE(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF32LE tries to read the BOM of UTF-32 [LE] encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF7

func GetBOMForEncodingUTF7(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF7 tries to read the BOM of UTF-7 encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF8

func GetBOMForEncodingUTF8(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF8 tries to read the BOM of UTF-8 encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func GetBOMForEncodingUTF_EBCDIC

func GetBOMForEncodingUTF_EBCDIC(r *reader.Reader) (prefix []byte, err error)

GetBOMForEncodingUTF_EBCDIC tries to read the BOM of UTF-EBCDIC encoding. If the BOM is found, error is nil; otherwise it is not nil. The prefix which was read from the stream is always returned as the first returned value. The reader is not reset after the reading.

func IsEncoding

func IsEncoding(rs rs.ReaderSeeker, enc Encoding) (isEncoding bool, err error)

IsEncoding checks the beginning of the stream and compares it's BOM with the BOM of the specified encoding. Simply, it tries to check whether the stream has a BOM of the specified encoding or not. The reader is reset after the reading.

func SkipBOMPrefix

func SkipBOMPrefix(rs rs.ReaderSeeker, enc Encoding) (newRS rs.ReaderSeeker, err error)

SkipBOMPrefix tries to skip BOM prefix from the data. It reads the BOM and returns the reader.

Types

type Encoding

type Encoding byte

func GetEncoding

func GetEncoding(rs rs.ReaderSeeker, shouldIncludeUnknownEncoding bool) (result []Encoding, err error)

GetEncoding tries to get all possible encodings for the stream. Please note that some encodings have similar BOMs, that is why an array is returned instead of a single value. The reader is reset after the reading.

The 'shouldIncludeUnknownEncoding' flag switches the way of result telling. If the flag is true, the UnknownEncoding is always included into the result. This is a pedantic way of telling the result for those who want a scientific approach. If the flag is false, UnknownEncoding is not included into the result. This means that if no encoding is found, the result will be an empty array. This is a simplistic approach. Independently of the setting, the result array is always null (nil) on reader's error.

func PossibleEncodings

func PossibleEncodings() []Encoding

PossibleEncodings returns a list of possible encodings except the unknown encoding.

func (Encoding) IsKnown

func (e Encoding) IsKnown() bool

IsKnown tells whether the encoding is known or not.

func (Encoding) IsUnknown

func (e Encoding) IsUnknown() bool

IsUnknown tells whether the encoding is unknown or not.