build

Documentation

Overview

package build implements a more convenient interface for building zoekt indices.

Index

• Variables
• func BranchNamesEqual(a, b []zoekt.RepositoryBranch) bool
• type Branch
• type Builder
  • func NewBuilder(opts Options) (*Builder, error)
  • func (b *Builder) Add(doc zoekt.Document) error
  • func (b *Builder) AddFile(name string, content []byte) error
  • func (b *Builder) Finish() error
  • func (b *Builder) MarkFileAsChangedOrRemoved(path string)
• type HashOptions
• type IndexState
• type Options
  • func (o *Options) Args() []string
  • func (o *Options) FindAllShards() []string
  • func (o *Options) FindRepositoryMetadata() (repository *zoekt.Repository, ok bool, err error)
  • func (o *Options) Flags(fs *flag.FlagSet)
  • func (o *Options) GetHash() string
  • func (o *Options) HashOptions() HashOptions
  • func (o *Options) IgnoreSizeMax(name string) bool
  • func (o *Options) IncrementalSkipIndexing() bool
  • func (o *Options) IndexState() (IndexState, string)
  • func (o *Options) SetDefaults()

Variables

var DefaultDir = filepath.Join(os.Getenv("HOME"), ".zoekt")

Functions

func BranchNamesEqual

func BranchNamesEqual(a, b []zoekt.RepositoryBranch) bool

BranchNamesEqual compares the given zoekt.RepositoryBranch slices, and returns true iff both slices specify the same set of branch names in the same order.

Types

type Branch

type Branch struct {
	Name    string
	Version string
}

Branch describes a single branch version.

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder manages (parallel) creation of uniformly sized shards. The builder buffers up documents until it collects enough documents and then builds a shard and writes.

func NewBuilder

func NewBuilder(opts Options) (*Builder, error)

NewBuilder creates a new Builder instance.

func (*Builder) Add

func (b *Builder) Add(doc zoekt.Document) error

func (*Builder) AddFile

func (b *Builder) AddFile(name string, content []byte) error

AddFile is a convenience wrapper for the Add method

func (*Builder) Finish

func (b *Builder) Finish() error

Finish creates a last shard from the buffered documents, and clears stale shards from previous runs. This should always be called, also in failure cases, to ensure cleanup.

It is safe to call Finish() multiple times.

func (*Builder) MarkFileAsChangedOrRemoved

func (b *Builder) MarkFileAsChangedOrRemoved(path string)

MarkFileAsChangedOrRemoved indicates that the file specified by the given path has been changed or removed since the last indexing job for this repository.

If this build is a delta build, these files will be tombstoned in the older shards for this repository.

type HashOptions

type HashOptions struct {
	// contains filtered or unexported fields
}

HashOptions contains only the options in Options that upon modification leads to IndexState of IndexStateMismatch during the next index building.

type IndexState

type IndexState string

const (
	IndexStateMissing IndexState = "missing"
	IndexStateCorrupt IndexState = "corrupt"
	IndexStateVersion IndexState = "version-mismatch"
	IndexStateOption  IndexState = "option-mismatch"
	IndexStateMeta    IndexState = "meta-mismatch"
	IndexStateContent IndexState = "content-mismatch"
	IndexStateEqual   IndexState = "equal"
)

type Options

type Options struct {
	// IndexDir is a directory that holds *.zoekt index files.
	IndexDir string

	// SizeMax is the maximum file size
	SizeMax int

	// Parallelism is the maximum number of shards to index in parallel
	Parallelism int

	// ShardMax sets the maximum corpus size for a single shard
	ShardMax int

	// TrigramMax sets the maximum number of distinct trigrams per document.
	TrigramMax int

	// RepositoryDescription holds names and URLs for the repository.
	RepositoryDescription zoekt.Repository

	// SubRepositories is a path => sub repository map.
	SubRepositories map[string]*zoekt.Repository

	// DisableCTags disables the generation of ctags metadata.
	DisableCTags bool

	// CtagsPath is the path to the ctags binary to run, or empty
	// if a valid binary couldn't be found.
	CTagsPath string

	// If set, ctags must succeed.
	CTagsMustSucceed bool

	// Write memory profiles to this file.
	MemProfile string

	// LargeFiles is a slice of glob patterns, including ** for any number
	// of directories, where matching file paths should be indexed
	// regardless of their size. The full pattern syntax is here:
	// https://github.com/bmatcuk/doublestar/tree/v1#patterns.
	LargeFiles []string

	// IsDelta is true if this run contains only the changed documents since the
	// last run.
	IsDelta bool

	// DocumentRanksPath is the path to the file with document ranks. If empty,
	// ranks will be computed on-the-fly.
	DocumentRanksPath string

	// DocumentRanksVersion is a string which when changed will cause us to
	// reindex a shard. This field is used so that when the contents of
	// DocumentRanksPath changes, we can reindex.
	DocumentRanksVersion string
	// contains filtered or unexported fields
}

Options sets options for the index building.

func (*Options) Args

func (o *Options) Args() []string

Args generates command line arguments for o. It is the "inverse" of Flags.

func (*Options) FindAllShards

func (o *Options) FindAllShards() []string

func (*Options) FindRepositoryMetadata

func (o *Options) FindRepositoryMetadata() (repository *zoekt.Repository, ok bool, err error)

FindRepositoryMetadata returns the index metadata for the repository specified in the options. 'ok' is false if the repository's metadata couldn't be found or if an error occurred.

func (*Options) Flags

func (o *Options) Flags(fs *flag.FlagSet)

Flags adds flags for build options to fs. It is the "inverse" of Args.

func (*Options) GetHash

func (o *Options) GetHash() string

func (*Options) HashOptions

func (o *Options) HashOptions() HashOptions

func (*Options) IgnoreSizeMax

func (o *Options) IgnoreSizeMax(name string) bool

IgnoreSizeMax determines whether the max size should be ignored.

func (*Options) IncrementalSkipIndexing

func (o *Options) IncrementalSkipIndexing() bool

IncrementalSkipIndexing returns true if the index present on disk matches the build options.

func (*Options) IndexState

func (o *Options) IndexState() (IndexState, string)

IndexState checks how the index present on disk compares to the build options and returns the IndexState and the name of the first shard.

func (*Options) SetDefaults

func (o *Options) SetDefaults()

SetDefaults sets reasonable default options.