codec

Documentation

Index

• Constants
• func CheckEOF(in IndexInput) error
• func CheckFooter(in ChecksumIndexInput) (cs int64, err error)
• func CheckHeader(in DataInput, codec string, minVersion, maxVersion int32) (v int32, err error)
• func CheckHeaderNoMagic(in DataInput, codec string, minVersion, maxVersion int32) (v int32, err error)
• func HeaderLength(codec string) int
• func NewIndexFormatTooNewError(in DataInput, version, minVersion, maxVersion int32) error
• func NewIndexFormatTooOldError(in DataInput, version, minVersion, maxVersion int32) error
• func RetrieveChecksum(in IndexInput) (int64, error)
• func WriteFooter(out IndexOutput) (err error)
• func WriteHeader(out DataOutput, codec string, version int) error
• type ChecksumIndexInput
• type DataInput
• type DataOutput
• type IndexInput
• type IndexOutput
• type PostingsConsumer
• type TermStats
  • func NewTermStats(docFreq int, totalTermFreq int64) *TermStats

Constants

const CODEC_MAGIC = 0x3fd76c17

Constant to identify the start of a codec header.

const FOOTER_LENGTH = 16

const FOOTER_MAGIC = ^CODEC_MAGIC

Constant to identify the start of a codec footer.

Functions

func CheckEOF

func CheckEOF(in IndexInput) error

Checks that the stream is positioned at the end, and returns error if it is not.

func CheckFooter

func CheckFooter(in ChecksumIndexInput) (cs int64, err error)

Validates the codec footer previously written by WriteFooter().

func CheckHeader

func CheckHeader(in DataInput, codec string, minVersion, maxVersion int32) (v int32, err error)

func CheckHeaderNoMagic

func CheckHeaderNoMagic(in DataInput, codec string, minVersion, maxVersion int32) (v int32, err error)

func HeaderLength

func HeaderLength(codec string) int

Computes the length of a codec header

func NewIndexFormatTooNewError

func NewIndexFormatTooNewError(in DataInput, version, minVersion, maxVersion int32) error

func NewIndexFormatTooOldError

func NewIndexFormatTooOldError(in DataInput, version, minVersion, maxVersion int32) error

func RetrieveChecksum

func RetrieveChecksum(in IndexInput) (int64, error)

Returns (but does not validate) the checksum previously written by CheckFooter.

func WriteFooter

func WriteFooter(out IndexOutput) (err error)

Writes a codec footer, which records both a checksum algorithm ID and a checksum. This footer can be parsed and validated with CheckFooter().

CodecFooter --> Magic,AlgorithmID,Checksum

• Magic --> uint32. This identifies the start of the footer. It is always FOOTER_MAGIC.
• AlgorithmID --> uing32. This indicates the checksum algorithm used. Currently this is always 0, for zlib-crc32.
• Checksum --> uint64. The actual checksum value for all previous bytes in the stream, including the bytes from Magic and AlgorithmID.

func WriteHeader

func WriteHeader(out DataOutput, codec string, version int) error

Writes a codc header, which records both a string to identify the file and a version number. This header can be parsed and validated with CheckHeader().

CodecHeader --> Magic,CodecName,Version

Magic --> uint32. This identifies the start of the header. It is
always CODEC_MAGIC.
CodecName --> string. This is a string to identify this file.
Version --> uint32. Records the version of the file.

Note that the length of a codec header depends only upon the name of the codec, so this length can be computed at any time with HeaderLength().

Types

type ChecksumIndexInput

type ChecksumIndexInput interface {
	IndexInput
	Checksum() int64
}

type DataInput

type DataInput interface {
	ReadInt() (int32, error)
	ReadString() (string, error)
}

type DataOutput

type DataOutput interface {
	WriteInt(n int32) error
	WriteString(s string) error
}

type IndexInput

type IndexInput interface {
	FilePointer() int64
	Seek(int64) error
	Length() int64
	ReadInt() (int32, error)
	ReadLong() (int64, error)
}

type IndexOutput

type IndexOutput interface {
	WriteInt(n int32) error
	WriteLong(n int64) error
	Checksum() int64
}

type PostingsConsumer

type PostingsConsumer interface {
	// Adds a new doc in this term. freq will be -1 when term
	// frequencies are omitted for the field.
	StartDoc(docId, freq int) error
	// Add a new position & payload, and start/end offset. A nil
	// payload means no payload; a non-nil payload with zero length
	// also means no payload. Caller may reuse the BytesRef for the
	// payload between calls (method must fully consume the payload).
	// startOffset and endOffset will be -1 when offsets are not indexed.
	AddPosition(position int, payload []byte, startOffset, endOffset int) error
	// Called when we are done adding positions & payloads for each doc.
	FinishDoc() error
}

Abstract API that consumes postings for an individual term.

The lifecycle is:

• PostingsConsumer is returned for each term by TermsConsumer.startTerm().
• startDoc() is called for each document where the term occurs, specifying id and term frequency for that document.
• If positions are enabled for the field, then addPosition() will be called for each occurrence in the document.
• finishDoc() is called when the producer is done adding positions to the document.

type TermStats

type TermStats struct {
	// How many documents have at least one occurrence of this term.
	DocFreq int
	// Total number of times this term occurs across all documents in
	// the field.
	TotalTermFreq int64
}

Holder for per-term statistics.

func NewTermStats

func NewTermStats(docFreq int, totalTermFreq int64) *TermStats