Documentation ¶
Overview ¶
Package pipeline implements an in-process pipeline to create a combined filetree and xrefs serving table from a stream of GraphStore-ordered entries.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type KytheBeam ¶ added in v0.0.27
type KytheBeam struct {
// contains filtered or unexported fields
}
KytheBeam controls the lifetime and generation of PCollections in the Kythe pipeline.
func FromEntries ¶ added in v0.0.27
func FromEntries(s beam.Scope, entries beam.PCollection) *KytheBeam
FromEntries creates a KytheBeam pipeline from an input collection of *spb.Entry messages.
func (*KytheBeam) Nodes ¶ added in v0.0.27
func (k *KytheBeam) Nodes() beam.PCollection
Nodes returns all *ppb.Nodes from the Kythe input graph.
type Options ¶ added in v0.0.10
type Options struct { // Verbose determines whether to emit extra, and possibly excessive, log messages. Verbose bool // MaxPageSize is maximum number of edges/cross-references that are allowed in // PagedEdgeSets, CrossReferences, EdgePages, and CrossReferences_Pages. If // MaxPageSize <= 0, no paging is attempted. MaxPageSize int // CompressShards determines whether intermediate data written to disk should // be compressed. CompressShards bool // MaxShardSize is the maximum number of elements to keep in-memory before // flushing an intermediary data shard to disk. MaxShardSize int // IOBufferSize is the size of the reading/writing buffers for the temporary // file shards. IOBufferSize int }
Options controls the behavior of pipeline.Run.
Directories ¶
Path | Synopsis |
---|---|
Package nodes provides Beam transformations over *ppb.Nodes.
|
Package nodes provides Beam transformations over *ppb.Nodes. |
Package paths specializes the util/reduce interfaces for Kythe *ipb.Paths.
|
Package paths specializes the util/reduce interfaces for Kythe *ipb.Paths. |
Click to show internal directories.
Click to hide internal directories.