packed

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 24, 2020 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	/* Default amount of memory to use for bulk operations. */
	DEFAULT_BUFFER_SIZE = 1024 // 1K

	PACKED_CODEC_NAME                = "PackedInts"
	PACKED_VERSION_START             = 0
	PACKED_VERSION_BYTE_ALIGNED      = 1
	VERSION_MONOTONIC_WITHOUT_ZIGZAG = 2
	VERSION_CURRENT                  = VERSION_MONOTONIC_WITHOUT_ZIGZAG
)
View Source 
const (
	PACKED              = 0
	PACKED_SINGLE_BLOCK = 1
)
View Source 
const (
	PACKED64_BLOCK_SIZE = 64                      // 32 = int, 64 = long
	PACKED64_BLOCK_BITS = 6                       // The #bits representing BLOCK_SIZE
	PACKED64_MOD_MASK   = PACKED64_BLOCK_SIZE - 1 // x % BLOCK_SIZE
)
View Source 
const BLOCK_COUNT = 1
View Source 
const DEFAULT_PAGE_SIZE = 1024
View Source 
const INITIAL_PAGE_COUNT = 16
View Source 
const MAX_BLOCK_SIZE = 1 << 30
View Source 
const MAX_PAGE_SIZE = 1 << 20
View Source 
const MIN_BLOCK_SIZE = 1 << 6
View Source 
const MIN_PAGE_SIZE = 64

Variables

View Source 
var PACKED16_THREE_BLOCKS_MAX_SIZE = int32(math.MaxInt32 / 3)
View Source 
var PACKED8_THREE_BLOCKS_MAX_SIZE = int32(math.MaxInt32 / 3)
View Source 
var PackedInts = struct {
	FASTEST float32 // At most 700% memory overhead, always select a direct implementation.
	FAST    float32 // At most 50% memory overhead, always elect a reasonable fast implementation.
	DEFAULT float32 // At most 25% memory overhead.
	COMPACT float32 // No memory overhead at all, but hte returned implementation may be slow.
}{7, 0.5, 0.25, 0}

Simplistic compression for arrays of unsinged int64 values. Each value is >= 0 and <= a specified maximum value. The vlues are stored as packed ints, with each value consuming a fixed number of bits.

View Source 
var TrailingZeros = func() map[int]int {
	ans := make(map[int]int)
	var n = 1
	for i := 0; i < 32; i++ {
		ans[n] = i
		n <<= 1
	}
	return ans
}()

Functions

func BitsRequired 

func BitsRequired(maxValue int64) int

Returns how many bits are required to hold values up to and including maxValue NOTE: This method returns at least 1.

func CheckVersion 

func CheckVersion(version int32)

Ceck the validity of a version number

func Copy 

func Copy(src PackedIntsReader, srcPos int, dest Mutable, destPos, length, mem int)

Copy src[srcPos:srcPos+len] into dest[destPos:destPos+len] using at most mem bytes.

func MaxValue 

func MaxValue(bitsPerValue int) int64

Calculate the maximum unsigned long that can be expressed with the given number of bits

func UnsignedBitsRequired 

func UnsignedBitsRequired(bits int64) int

Returns how many bits are required to store bits, interpreted as an unsigned value. NOTE: This method returns at least 1.

Types

type BulkOperation 

type BulkOperation interface {
	LongBlockCount() int
	LongValueCount() int
	ByteBlockCount() int
	ByteValueCount() int

	EncodeIntToByte(values []int, blocks []byte, iterations int)
	// contains filtered or unexported methods
}

Efficient sequential read/write of packed integers.

type BulkOperationImpl 

type BulkOperationImpl struct {
	PackedIntsDecoder
}

type BulkOperationPacked 

type BulkOperationPacked struct {
	*BulkOperationImpl
	// contains filtered or unexported fields
}

Non-specialized BulkOperation for Packed format

func (*BulkOperationPacked) ByteBlockCount 

func (p *BulkOperationPacked) ByteBlockCount() int

func (*BulkOperationPacked) ByteValueCount 

func (p *BulkOperationPacked) ByteValueCount() int

func (*BulkOperationPacked) EncodeIntToByte 

func (p *BulkOperationPacked) EncodeIntToByte(values []int, blocks []byte, iterations int)

func (*BulkOperationPacked) LongBlockCount 

func (p *BulkOperationPacked) LongBlockCount() int

func (*BulkOperationPacked) LongValueCount 

func (p *BulkOperationPacked) LongValueCount() int

type BulkOperationPacked1 

type BulkOperationPacked1 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked10 

type BulkOperationPacked10 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked11 

type BulkOperationPacked11 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked12 

type BulkOperationPacked12 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked13 

type BulkOperationPacked13 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked14 

type BulkOperationPacked14 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked15 

type BulkOperationPacked15 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked16 

type BulkOperationPacked16 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked17 

type BulkOperationPacked17 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked18 

type BulkOperationPacked18 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked19 

type BulkOperationPacked19 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked2 

type BulkOperationPacked2 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked20 

type BulkOperationPacked20 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked21 

type BulkOperationPacked21 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked22 

type BulkOperationPacked22 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked23 

type BulkOperationPacked23 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked24 

type BulkOperationPacked24 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked3 

type BulkOperationPacked3 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked4 

type BulkOperationPacked4 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked5 

type BulkOperationPacked5 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked6 

type BulkOperationPacked6 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked7 

type BulkOperationPacked7 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked8 

type BulkOperationPacked8 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPacked9 

type BulkOperationPacked9 struct {
	*BulkOperationPacked
}

Efficient sequential read/write of packed integers.

type BulkOperationPackedSingleBlock 

type BulkOperationPackedSingleBlock struct {
	*BulkOperationImpl
	// contains filtered or unexported fields
}

Non-specialized BulkOperation for PACKED_SINGLE_BLOCK format

func (*BulkOperationPackedSingleBlock) ByteBlockCount 

func (p *BulkOperationPackedSingleBlock) ByteBlockCount() int

func (*BulkOperationPackedSingleBlock) ByteValueCount 

func (p *BulkOperationPackedSingleBlock) ByteValueCount() int

func (*BulkOperationPackedSingleBlock) EncodeIntToByte 

func (p *BulkOperationPackedSingleBlock) EncodeIntToByte(values []int,
	blocks []byte, iterations int)

func (*BulkOperationPackedSingleBlock) LongBlockCount 

func (p *BulkOperationPackedSingleBlock) LongBlockCount() int

func (*BulkOperationPackedSingleBlock) LongValueCount 

func (p *BulkOperationPackedSingleBlock) LongValueCount() int

type DataInput 

type DataInput interface {
	ReadByte() (byte, error)
	ReadBytes(buf []byte) error
	ReadShort() (int16, error)
	ReadInt() (int32, error)
	ReadVInt() (int32, error)
	ReadLong() (int64, error)
	ReadString() (string, error)
}

type DataOutput 

type DataOutput interface {
	WriteBytes(buf []byte) error
	WriteInt(int32) error
	WriteVInt(int32) error
	WriteString(string) error
}

type DeltaPackedLongValuesBuilderImpl 

type DeltaPackedLongValuesBuilderImpl struct {
	*PackedLongValuesBuilderImpl
	// contains filtered or unexported fields
}

func NewDeltaPackedLongValuesBuilder 

func NewDeltaPackedLongValuesBuilder(pageSize int,
	acceptableOverheadRatio float32) *DeltaPackedLongValuesBuilderImpl

type Direct16 

type Direct16 struct {
	*MutableImpl
	// contains filtered or unexported fields
}

Direct wrapping of 16-bits values to a backing array.

func (*Direct16) Clear 

func (d *Direct16) Clear()

func (Direct16) Format 

func (m Direct16) Format() PackedFormat

func (*Direct16) Get 

func (d *Direct16) Get(index int) int64

func (*Direct16) RamBytesUsed 

func (d *Direct16) RamBytesUsed() int64

func (Direct16) Save 

func (m Direct16) Save(out util.DataOutput) error

func (*Direct16) Set 

func (d *Direct16) Set(index int, value int64)

type Direct32 

type Direct32 struct {
	*MutableImpl
	// contains filtered or unexported fields
}

Direct wrapping of 32-bits values to a backing array.

func (*Direct32) Clear 

func (d *Direct32) Clear()

func (Direct32) Format 

func (m Direct32) Format() PackedFormat

func (*Direct32) Get 

func (d *Direct32) Get(index int) int64

func (*Direct32) RamBytesUsed 

func (d *Direct32) RamBytesUsed() int64

func (Direct32) Save 

func (m Direct32) Save(out util.DataOutput) error

func (*Direct32) Set 

func (d *Direct32) Set(index int, value int64)

type Direct64 

type Direct64 struct {
	*MutableImpl
	// contains filtered or unexported fields
}

Direct wrapping of 64-bits values to a backing array.

func (*Direct64) Clear 

func (d *Direct64) Clear()

func (Direct64) Format 

func (m Direct64) Format() PackedFormat

func (*Direct64) Get 

func (d *Direct64) Get(index int) int64

func (*Direct64) RamBytesUsed 

func (d *Direct64) RamBytesUsed() int64

func (Direct64) Save 

func (m Direct64) Save(out util.DataOutput) error

func (*Direct64) Set 

func (d *Direct64) Set(index int, value int64)

type Direct8 

type Direct8 struct {
	*MutableImpl
	// contains filtered or unexported fields
}

Direct wrapping of 8-bits values to a backing array.

func (*Direct8) Clear 

func (d *Direct8) Clear()

func (Direct8) Format 

func (m Direct8) Format() PackedFormat

func (*Direct8) Get 

func (d *Direct8) Get(index int) int64

func (*Direct8) RamBytesUsed 

func (d *Direct8) RamBytesUsed() int64

func (Direct8) Save 

func (m Direct8) Save(out util.DataOutput) error

func (*Direct8) Set 

func (d *Direct8) Set(index int, value int64)

type FormatAndBits 

type FormatAndBits struct {
	Format       PackedFormat
	BitsPerValue int
}

Simple class that holds a format and a number of bits per value.

func FastestFormatAndBits 

func FastestFormatAndBits(valueCount, bitsPerValue int,
	acceptableOverheadRatio float32) FormatAndBits

Try to find the Format and number of bits per value that would restore from disk the fastest reader whose overhead is less than acceptableOverheadRatio.

The acceptableOverheadRatio parameter makes sense for random-access Readers. In case you only plan to perform sequential access on this stream later on, you should probably use COMPACT.

If you don't know how many values you are going to write, use valueCount = -1.

func (FormatAndBits) String 

func (v FormatAndBits) String() string

type GrowableWriter 

type GrowableWriter struct {
	// contains filtered or unexported fields
}

func NewGrowableWriter 

func NewGrowableWriter(startBitsPerValue, valueCount int,
	acceptableOverheadRatio float32) *GrowableWriter

func (*GrowableWriter) BitsPerValue 

func (w *GrowableWriter) BitsPerValue() int

func (*GrowableWriter) Clear 

func (w *GrowableWriter) Clear()

func (GrowableWriter) Format 

func (m GrowableWriter) Format() PackedFormat

func (*GrowableWriter) Get 

func (w *GrowableWriter) Get(index int) int64

func (*GrowableWriter) RamBytesUsed 

func (w *GrowableWriter) RamBytesUsed() int64

func (*GrowableWriter) Save 

func (w *GrowableWriter) Save(out util.DataOutput) error

func (*GrowableWriter) Set 

func (w *GrowableWriter) Set(index int, value int64)

func (*GrowableWriter) Size 

func (w *GrowableWriter) Size() int

type Mutable 

type Mutable interface {
	PackedIntsReader
	// Returns the number of bits used to store any given value. Note:
	// this does not imply that memory usage is bpv * values() as
	// implementations are free to use non-space-optimal packing of
	// bits.
	BitsPerValue() int
	// Set the value at the given index in the array.
	Set(index int, value int64)

	Clear()
	// Save this mutable into out. Instantiating a reader from the
	// generated data will return a reader with the same number of bits
	// per value.
	Save(out util.DataOutput) error
	// contains filtered or unexported methods
}

A packed integer array that can be modified.

func MutableFor 

func MutableFor(valueCount, bitsPerValue int, acceptableOverheadRatio float32) Mutable

Create a packed integer slice with the given amount of values initialized to 0. The valueCount and the bitsPerValue cannot be changed after creation. All Mutables known by this factory are kept fully in RAM.

Positive values of acceptableOverheadRatio will trade space for speed by selecting a faster but potentially less memory-efficient implementation. An acceptableOverheadRatio of COMPACT will make sure that the most memory-efficient implementation is selected whereas FASTEST will make sure that the fastest implementation is selected.

func MutableForFormat 

func MutableForFormat(vc, bpv int, format PackedFormat) Mutable

Same as MutableFor() with a pre-computed number of bits per value and format.

type MutableImpl 

type MutableImpl struct {
	// contains filtered or unexported fields
}

func (*MutableImpl) BitsPerValue 

func (m *MutableImpl) BitsPerValue() int

func (MutableImpl) Clear 

func (m MutableImpl) Clear()

Sets all values to 0

func (MutableImpl) Format 

func (m MutableImpl) Format() PackedFormat

func (MutableImpl) Save 

func (m MutableImpl) Save(out util.DataOutput) error

func (*MutableImpl) Size 

func (m *MutableImpl) Size() int

type NilReader 

type NilReader struct {
	// contains filtered or unexported fields
}

A reader which has all its values equal to 0 (bitsPerValue = 0).

func (*NilReader) Get 

func (r *NilReader) Get(int) int64

func (*NilReader) RamBytesUsed 

func (r *NilReader) RamBytesUsed() int64

func (*NilReader) Size 

func (r *NilReader) Size() int

type Packed16ThreeBlocks 

type Packed16ThreeBlocks struct {
	*MutableImpl
	// contains filtered or unexported fields
}

func (*Packed16ThreeBlocks) Clear 

func (r *Packed16ThreeBlocks) Clear()

func (Packed16ThreeBlocks) Format 

func (m Packed16ThreeBlocks) Format() PackedFormat

func (*Packed16ThreeBlocks) Get 

func (p *Packed16ThreeBlocks) Get(index int) int64

func (*Packed16ThreeBlocks) RamBytesUsed 

func (p *Packed16ThreeBlocks) RamBytesUsed() int64

func (Packed16ThreeBlocks) Save 

func (m Packed16ThreeBlocks) Save(out util.DataOutput) error

func (*Packed16ThreeBlocks) Set 

func (r *Packed16ThreeBlocks) Set(index int, value int64)

func (*Packed16ThreeBlocks) String 

func (p *Packed16ThreeBlocks) String() string

type Packed64 

type Packed64 struct {
	*MutableImpl
	// contains filtered or unexported fields
}

func (*Packed64) Clear 

func (p *Packed64) Clear()

func (Packed64) Format 

func (m Packed64) Format() PackedFormat

func (*Packed64) Get 

func (p *Packed64) Get(index int) int64

func (*Packed64) RamBytesUsed 

func (p *Packed64) RamBytesUsed() int64

func (Packed64) Save 

func (m Packed64) Save(out util.DataOutput) error

func (*Packed64) Set 

func (p *Packed64) Set(index int, value int64)

func (*Packed64) String 

func (p *Packed64) String() string

type Packed64SingleBlock 

type Packed64SingleBlock struct {
	*MutableImpl
	// contains filtered or unexported fields
}

func (*Packed64SingleBlock) Clear 

func (p *Packed64SingleBlock) Clear()

func (*Packed64SingleBlock) Format 

func (p *Packed64SingleBlock) Format() PackedFormat

func (*Packed64SingleBlock) Get 

func (p *Packed64SingleBlock) Get(index int) int64

func (*Packed64SingleBlock) RamBytesUsed 

func (p *Packed64SingleBlock) RamBytesUsed() int64

func (Packed64SingleBlock) Save 

func (m Packed64SingleBlock) Save(out util.DataOutput) error

func (*Packed64SingleBlock) Set 

func (p *Packed64SingleBlock) Set(index int, value int64)

func (*Packed64SingleBlock) String 

func (p *Packed64SingleBlock) String() string

type Packed8ThreeBlocks 

type Packed8ThreeBlocks struct {
	*MutableImpl
	// contains filtered or unexported fields
}

func (*Packed8ThreeBlocks) Clear 

func (r *Packed8ThreeBlocks) Clear()

func (Packed8ThreeBlocks) Format 

func (m Packed8ThreeBlocks) Format() PackedFormat

func (*Packed8ThreeBlocks) Get 

func (r *Packed8ThreeBlocks) Get(index int) int64

func (*Packed8ThreeBlocks) RamBytesUsed 

func (r *Packed8ThreeBlocks) RamBytesUsed() int64

func (Packed8ThreeBlocks) Save 

func (m Packed8ThreeBlocks) Save(out util.DataOutput) error

func (*Packed8ThreeBlocks) Set 

func (r *Packed8ThreeBlocks) Set(index int, value int64)

func (*Packed8ThreeBlocks) String 

func (r *Packed8ThreeBlocks) String() string

type PackedFormat 

type PackedFormat int

*

  • A format to write packed ints. *
  • @lucene.internal

func (PackedFormat) ByteCount 

func (f PackedFormat) ByteCount(packedIntsVersion, valueCount int32, bitsPerValue uint32) int64

*

  • Computes how many byte blocks are needed to store <code>values</code>
  • values of size <code>bitsPerValue</code>.

func (PackedFormat) Id 

func (f PackedFormat) Id() int

func (PackedFormat) IsSupported 

func (f PackedFormat) IsSupported(bitsPerValue int) bool

*

  • Tests whether the provided number of bits per value is supported by the
  • format.

func (PackedFormat) OverheadPerValue 

func (f PackedFormat) OverheadPerValue(bitsPerValue int) float32

Returns the overhead per value, in bits.

type PackedIntsDecoder 

type PackedIntsDecoder interface {
	// The minum numer of byte blocks to encode in a single iteration, when using byte encoding
	ByteBlockCount() int
	// The number of values that can be stored in byteBlockCount() byte blocks
	ByteValueCount() int
	// contains filtered or unexported methods
}

func GetPackedIntsDecoder 

func GetPackedIntsDecoder(format PackedFormat, version int32, bitsPerValue uint32) PackedIntsDecoder

type PackedIntsEncoder 

type PackedIntsEncoder interface {
	ByteValueCount() int
	ByteBlockCount() int

	EncodeIntToByte(values []int, blocks []byte, iterations int)
	// contains filtered or unexported methods
}

func GetPackedIntsEncoder 

func GetPackedIntsEncoder(format PackedFormat, version int32, bitsPerValue uint32) PackedIntsEncoder

type PackedIntsReader 

type PackedIntsReader interface {
	util.Accountable
	Get(index int) int64 // NumericDocValue

	Size() int
	// contains filtered or unexported methods
}

A read-only random access array of positive integers.

func NewPackedReader 

func NewPackedReader(in DataInput) (r PackedIntsReader, err error)

func ReaderNoHeader 

func ReaderNoHeader(in DataInput, format PackedFormat, version, valueCount int32,
	bitsPerValue uint32) (r PackedIntsReader, err error)

type PackedLongValues 

type PackedLongValues interface {
	Size() int64
	Iterator() func() (interface{}, bool)
}

type PackedLongValuesBuilder 

type PackedLongValuesBuilder interface {
	util.Accountable
	Build() PackedLongValues
	Size() int64
	Add(int64) PackedLongValuesBuilder
}

func DeltaPackedBuilder 

func DeltaPackedBuilder(acceptableOverheadRatio float32) PackedLongValuesBuilder

type PackedLongValuesBuilderImpl 

type PackedLongValuesBuilderImpl struct {
	// contains filtered or unexported fields
}

func (*PackedLongValuesBuilderImpl) Add 

func (b *PackedLongValuesBuilderImpl) Add(l int64) PackedLongValuesBuilder

Add a new element to this builder.

func (*PackedLongValuesBuilderImpl) Build 

func (b *PackedLongValuesBuilderImpl) Build() PackedLongValues

Build a PackedLongValues instance that contains values that have been added to this builder. This operation is destructive.

func (*PackedLongValuesBuilderImpl) RamBytesUsed 

func (b *PackedLongValuesBuilderImpl) RamBytesUsed() int64

func (*PackedLongValuesBuilderImpl) Size 

func (b *PackedLongValuesBuilderImpl) Size() int64

Return the number of elements that have been added to this builder

type PackedLongValuesImpl 

type PackedLongValuesImpl struct {
	// contains filtered or unexported fields
}

func (*PackedLongValuesImpl) Iterator 

func (p *PackedLongValuesImpl) Iterator() func() (interface{}, bool)

func (*PackedLongValuesImpl) Size 

func (p *PackedLongValuesImpl) Size() int64

type PackedReaderIterator 

type PackedReaderIterator struct {
	*ReaderIteratorImpl
	// contains filtered or unexported fields
}

type PackedWriter 

type PackedWriter struct {
	*WriterImpl
	// contains filtered or unexported fields
}

func (*PackedWriter) Add 

func (w *PackedWriter) Add(v int64) error

func (*PackedWriter) Finish 

func (w *PackedWriter) Finish() error

type PagedGrowableWriter 

type PagedGrowableWriter struct {
	// contains filtered or unexported fields
}

A PagedGrowableWriter. This class slices data into fixed-size blocks which have independent numbers of bits per value and grow on-demand.

You should use this class instead of the AbstractAppendingLongBuffer related ones only when you need random write-access. Otherwise this class will likely be slower and less memory-efficient.

func NewPagedGrowableWriter 

func NewPagedGrowableWriter(size int64, pageSize, startBitsPerValue int,
	acceptableOverheadRatio float32) *PagedGrowableWriter

Create a new PagedGrowableWriter instance.

func (PagedGrowableWriter) Get 

func (m PagedGrowableWriter) Get(index int64) int64

func (PagedGrowableWriter) Set 

func (m PagedGrowableWriter) Set(index, value int64)

set value at index.

func (PagedGrowableWriter) Size 

func (m PagedGrowableWriter) Size() int64

type ReaderImpl 

type ReaderImpl struct {
	// contains filtered or unexported fields
}

func (*ReaderImpl) Size 

func (p *ReaderImpl) Size() int

type ReaderIterator 

type ReaderIterator interface {
	Next() (v int64, err error) // next value
	// contains filtered or unexported methods
}

Run-once iterator interface, to decode previously saved PackedInts

func ReaderIteratorNoHeader 

func ReaderIteratorNoHeader(in DataInput, format PackedFormat, version,
	valueCount, bitsPerValue, mem int) ReaderIterator

Expert: Restore a ReaderIterator from a stream without reading metadata at the beginning of the stream. This method is useful to restore data from streams which have been created using WriterNoHeader().

type ReaderIteratorImpl 

type ReaderIteratorImpl struct {
	// contains filtered or unexported fields
}

func (*ReaderIteratorImpl) Next 

func (it *ReaderIteratorImpl) Next() (v int64, err error)

Lucene(Java) manipulates underlying LongsRef to advance the pointer, which can not be implemented using Go's slice. Here I have to assume nextN() method would automatically increment the pointer without next().

type Writer 

type Writer interface {

	// Add a value to the stream.
	Add(v int64) error
	// The number of bits per value.
	BitsPerValue() int
	// Perform end-of-stream operations.
	Finish() error
	// contains filtered or unexported methods
}

A write-once Writer.

func WriterNoHeader 

func WriterNoHeader(out DataOutput, format PackedFormat,
	valueCount, bitsPerValue, mem int) Writer

Expert: Create a packed integer array writer for the given output, format, value count, and number of bits per value.

The resulting stream will be long-aligned. This means that depending on the format which is used, up to 63 bits will be wasted. An easy way to make sure tha tno space is lost is to always use a valueCount that is a multiple of 64.

This method does not write any metadata to the stream, meaning that it is your responsibility to store it somewhere else in order to be able to recover data from the stream later on:

- format (using Format.Id()), - valueCount, - bitsPerValue, - VERSION_CURRENT.

It is possible to start writing values without knowing how many of them you are actually going to write. To do this, just pass -1 as valueCount. On the other hand, for any positive value of valueCount, the returned writer will make sure that you don't write more values than expected and pad the end of stream with zeros in case you have writen less than valueCount when calling Writer.Finish().

The mem parameter lets your control how much memory can be used to buffer changes in memory before finishing to disk. High values of mem are likely to improve throughput. On the other hand, if speed is not that important to you, a value of 0 will use as little memory as possible and should already offer reasonble throughput.

type WriterImpl 

type WriterImpl struct {
	// contains filtered or unexported fields
}

func (*WriterImpl) BitsPerValue 

func (w *WriterImpl) BitsPerValue() int

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.