builder

Documentation

Index

• func NewBuilderFromDocuments(opts Options) (segment.CloseableDocumentsBuilder, error)
• func NewBuilderFromSegments(opts Options) segment.SegmentsBuilder
• func SetSortConcurrency(value int)
• type IDsMap
  • func NewIDsMap(opts IDsMapOptions) *IDsMap
  • func (m *IDsMap) Contains(k []byte) bool
  • func (m *IDsMap) Delete(k []byte)
  • func (m *IDsMap) Get(k []byte) (struct{}, bool)
  • func (m *IDsMap) Iter() map[IDsMapHash]IDsMapEntry
  • func (m *IDsMap) Len() int
  • func (m *IDsMap) Reallocate()
  • func (m *IDsMap) Reset()
  • func (m *IDsMap) Set(k []byte, v struct{})
  • func (m *IDsMap) SetUnsafe(k []byte, v struct{}, opts IDsMapSetUnsafeOptions)
• type IDsMapCopyFn
• type IDsMapEntry
  • func (e IDsMapEntry) Key() []byte
  • func (e IDsMapEntry) Value() struct{}
• type IDsMapEqualsFn
• type IDsMapFinalizeFn
• type IDsMapHash
• type IDsMapHashFn
• type IDsMapOptions
• type IDsMapSetUnsafeOptions
• type Options
  • func NewOptions() Options
• type PostingsMap
  • func NewPostingsMap(opts PostingsMapOptions) *PostingsMap
  • func (m *PostingsMap) Contains(k []byte) bool
  • func (m *PostingsMap) Delete(k []byte)
  • func (m *PostingsMap) Get(k []byte) (postings.MutableList, bool)
  • func (m *PostingsMap) Iter() map[PostingsMapHash]PostingsMapEntry
  • func (m *PostingsMap) Len() int
  • func (m *PostingsMap) Reallocate()
  • func (m *PostingsMap) Reset()
  • func (m *PostingsMap) Set(k []byte, v postings.MutableList)
  • func (m *PostingsMap) SetUnsafe(k []byte, v postings.MutableList, opts PostingsMapSetUnsafeOptions)
• type PostingsMapCopyFn
• type PostingsMapEntry
  • func (e PostingsMapEntry) Key() []byte
  • func (e PostingsMapEntry) Value() postings.MutableList
• type PostingsMapEqualsFn
• type PostingsMapFinalizeFn
• type PostingsMapHash
• type PostingsMapHashFn
• type PostingsMapOptions
• type PostingsMapSetUnsafeOptions

Functions

func NewBuilderFromDocuments

func NewBuilderFromDocuments(opts Options) (segment.CloseableDocumentsBuilder, error)

NewBuilderFromDocuments returns a builder from documents, it is not thread safe and is optimized for insertion speed and a final build step when documents are indexed.

func NewBuilderFromSegments

func NewBuilderFromSegments(opts Options) segment.SegmentsBuilder

NewBuilderFromSegments returns a new builder from segments.

func SetSortConcurrency

func SetSortConcurrency(value int)

SetSortConcurrency sets the sort concurrency for when building segments, unfortunately this must be set globally since github.com/twotwotwo/sorts does not provide an ability to set parallelism on call to sort.

Types

type IDsMap

type IDsMap struct {
	// contains filtered or unexported fields
}

IDsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. IDsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewIDsMap

func NewIDsMap(opts IDsMapOptions) *IDsMap

NewIDsMap returns a new byte keyed map.

func (*IDsMap) Contains

func (m *IDsMap) Contains(k []byte) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*IDsMap) Delete

func (m *IDsMap) Delete(k []byte)

Delete will remove a value set in the map for the specified key.

func (*IDsMap) Get

func (m *IDsMap) Get(k []byte) (struct{}, bool)

Get returns a value in the map for an identifier if found.

func (*IDsMap) Iter

func (m *IDsMap) Iter() map[IDsMapHash]IDsMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*IDsMap) Len

func (m *IDsMap) Len() int

Len returns the number of map entries in the map.

func (*IDsMap) Reallocate

func (m *IDsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*IDsMap) Reset

func (m *IDsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*IDsMap) Set

func (m *IDsMap) Set(k []byte, v struct{})

Set will set the value for an identifier.

func (*IDsMap) SetUnsafe

func (m *IDsMap) SetUnsafe(k []byte, v struct{}, opts IDsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type IDsMapCopyFn

type IDsMapCopyFn func([]byte) []byte

IDsMapCopyFn is the copy key function to execute when copying the key.

type IDsMapEntry

type IDsMapEntry struct {
	// contains filtered or unexported fields
}

IDsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (IDsMapEntry) Key

func (e IDsMapEntry) Key() []byte

Key returns the map entry key.

func (IDsMapEntry) Value

func (e IDsMapEntry) Value() struct{}

Value returns the map entry value.

type IDsMapEqualsFn

type IDsMapEqualsFn func([]byte, []byte) bool

IDsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type IDsMapFinalizeFn

type IDsMapFinalizeFn func([]byte)

IDsMapFinalizeFn is the finalize key function to execute when finished with a key.

type IDsMapHash

type IDsMapHash uint64

IDsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type IDsMapHashFn

type IDsMapHashFn func([]byte) IDsMapHash

IDsMapHashFn is the hash function to execute when hashing a key.

type IDsMapOptions

type IDsMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

IDsMapOptions provides options used when created the map.

type IDsMapSetUnsafeOptions

type IDsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

IDsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type Options

type Options interface {
	// SetNewUUIDFn sets the function used to generate new UUIDs.
	SetNewUUIDFn(value util.NewUUIDFn) Options

	// NewUUIDFn returns the function used to generate new UUIDs.
	NewUUIDFn() util.NewUUIDFn

	// SetInitialCapacity sets the initial capacity.
	SetInitialCapacity(value int) Options

	// InitialCapacity returns the initial capacity.
	InitialCapacity() int

	// SetPostingsListPool sets the postings list pool.
	SetPostingsListPool(value postings.Pool) Options

	// PostingsListPool returns the postings list pool.
	PostingsListPool() postings.Pool

	// SetConcurrency sets the indexing concurrency.
	SetConcurrency(value int) Options

	// Concurrency returns the indexing concurrency.
	Concurrency() int
}

Options is a collection of options for segment building.

func NewOptions

func NewOptions() Options

NewOptions returns new options.

type PostingsMap

type PostingsMap struct {
	// contains filtered or unexported fields
}

PostingsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. PostingsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewPostingsMap

func NewPostingsMap(opts PostingsMapOptions) *PostingsMap

NewPostingsMap returns a new byte keyed map.

func (*PostingsMap) Contains

func (m *PostingsMap) Contains(k []byte) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*PostingsMap) Delete

func (m *PostingsMap) Delete(k []byte)

Delete will remove a value set in the map for the specified key.

func (*PostingsMap) Get

func (m *PostingsMap) Get(k []byte) (postings.MutableList, bool)

Get returns a value in the map for an identifier if found.

func (*PostingsMap) Iter

func (m *PostingsMap) Iter() map[PostingsMapHash]PostingsMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*PostingsMap) Len

func (m *PostingsMap) Len() int

Len returns the number of map entries in the map.

func (*PostingsMap) Reallocate

func (m *PostingsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*PostingsMap) Reset

func (m *PostingsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*PostingsMap) Set

func (m *PostingsMap) Set(k []byte, v postings.MutableList)

Set will set the value for an identifier.

func (*PostingsMap) SetUnsafe

func (m *PostingsMap) SetUnsafe(k []byte, v postings.MutableList, opts PostingsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type PostingsMapCopyFn

type PostingsMapCopyFn func([]byte) []byte

PostingsMapCopyFn is the copy key function to execute when copying the key.

type PostingsMapEntry

type PostingsMapEntry struct {
	// contains filtered or unexported fields
}

PostingsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (PostingsMapEntry) Key

func (e PostingsMapEntry) Key() []byte

Key returns the map entry key.

func (PostingsMapEntry) Value

func (e PostingsMapEntry) Value() postings.MutableList

Value returns the map entry value.

type PostingsMapEqualsFn

type PostingsMapEqualsFn func([]byte, []byte) bool

PostingsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type PostingsMapFinalizeFn

type PostingsMapFinalizeFn func([]byte)

PostingsMapFinalizeFn is the finalize key function to execute when finished with a key.

type PostingsMapHash

type PostingsMapHash uint64

PostingsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type PostingsMapHashFn

type PostingsMapHashFn func([]byte) PostingsMapHash

PostingsMapHashFn is the hash function to execute when hashing a key.

type PostingsMapOptions

type PostingsMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

PostingsMapOptions provides options used when created the map.

type PostingsMapSetUnsafeOptions

type PostingsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

PostingsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.