pipeline

Documentation

Overview

Package pipeline implements an in-process pipeline to create a combined filetree and xrefs serving table from a graphstore Service.

Index

• func Run(ctx context.Context, gs graphstore.Service, db keyvalue.DB, opts *Options) error
• type Options

Functions

func Run

func Run(ctx context.Context, gs graphstore.Service, db keyvalue.DB, opts *Options) error

Run writes the xrefs and filetree serving tables to db based on the given graphstore.Service.

Types

type Options

type Options struct {
	// MaxPageSize is maximum number of edges/cross-references that are allowed in
	// PagedEdgeSets, CrossReferences, EdgePages, and CrossReferences_Pages.  If
	// MaxPageSize <= 0, no paging is attempted.
	MaxPageSize int

	// CompressShards determines whether intermediate data written to disk should
	// be compressed.
	CompressShards bool

	// MaxShardSize is the maximum number of elements to keep in-memory before
	// flushing an intermediary data shard to disk.
	MaxShardSize int

	// IOBufferSize is the size of the reading/writing buffers for the temporary
	// file shards.
	IOBufferSize int
}

Options controls the behavior of pipeline.Run.