annotee

Documentation

Index

• Constants
• func ConvertBuildSteps(c context.Context, annSteps []*annotpb.Step_Substep, constructLogURL bool, ...) ([]*pb.Step, error)
• func ConvertRootStep(c context.Context, rootStep *annotpb.Step) (*pb.Build, error)
• type CoordinatorLinkGenerator
  • func (g *CoordinatorLinkGenerator) CanGenerateLinks() bool
  • func (g *CoordinatorLinkGenerator) GetLink(names ...types.StreamName) string
• type LinkGenerator
• type Options
• type Processor
  • func New(c context.Context, o Options) *Processor
  • func (p *Processor) Finish() *annotation.State
  • func (p *Processor) IngestLine(s *Stream, line string) error
  • func (p *Processor) RunStreams(streams []*Stream) error
• type Stream

Constants

const (
	// STDOUT is the system STDOUT stream name.
	STDOUT = types.StreamName("stdout")
	// STDERR is the system STDERR stream.
	STDERR = types.StreamName("stderr")

	// DefaultAnnotationSubpath is the default annotation subpath. It will be used
	// if an explicit subpath is not provided.
	DefaultAnnotationSubpath = types.StreamName("annotations")
)

const (
	// DefaultBufferSize is the Stream BufferSize value that will be used if no
	// buffer size is provided.
	DefaultBufferSize = 8192
)

const StepSep = "|"

StepSep separates parent and child steps.

Functions

func ConvertBuildSteps

func ConvertBuildSteps(c context.Context, annSteps []*annotpb.Step_Substep, constructLogURL bool, defaultLogdogHost, defaultLogdogPrefix string) ([]*pb.Step, error)

ConvertBuildSteps converts a build given the root step's substeps, which must be the actual steps of the build, and the Logdog URL for links conversion. The provided context is used only for logging.

If constructLogURL is True, all returned steps will have fully-qualified logdog urls (i.e. logdog://host/prefix/+/log_name) and viewer urls will also be populated. Otherwise, the logdog url will simply be the same as log stream name and it is caller's responsibility to calculate the full URL. Values of defaultLogdogHost and defaultLogdogPrefix are only meaningful when constructLogURL is True. TODO(yiwzhang): Remove the option to construct full logdog url after kitchen is deprecated. Currently, this functionality is used in Kitchen.

Does not verify that the returned build satisfies all the constraints described in the proto files.

Unsupported fields:

• Substep.annotation_stream,
• Step.link,
• Link.isolate_object,
• Link.dm_link.

func ConvertRootStep

func ConvertRootStep(c context.Context, rootStep *annotpb.Step) (*pb.Build, error)

ConvertRootStep converts an annotation root step to a Build proto msg.

This function will populate following fields in the return build.

• EndTime
• StartTime
• Steps
• Status
• SummaryMarkdown
• Output.Logs
• Output.Properties

Types

type CoordinatorLinkGenerator

type CoordinatorLinkGenerator struct {
	Host    string
	Project string
	Prefix  types.StreamName
}

CoordinatorLinkGenerator is a LinkGenerator implementation

func (*CoordinatorLinkGenerator) CanGenerateLinks

func (g *CoordinatorLinkGenerator) CanGenerateLinks() bool

CanGenerateLinks returns true if g is sufficiently configured to generate LogDog Coordinator links.

func (*CoordinatorLinkGenerator) GetLink

func (g *CoordinatorLinkGenerator) GetLink(names ...types.StreamName) string

GetLink implements LinkGenerator.

type LinkGenerator

type LinkGenerator interface {
	// GetLink returns a link for the specified aggregate streams.
	//
	// If no link could be generated, GetLink may return an empty string.
	GetLink(name ...types.StreamName) string
}

LinkGenerator generates links for a given log stream.

type Options

type Options struct {
	// Base is the base log stream name. This is prepended to every log name, as
	// well as any generate log names.
	Base types.StreamName
	// AnnotationSubpath is the path underneath of Base where the annotation
	// stream will be written.
	//
	// If empty, DefaultAnnotationSubpath will be used.
	AnnotationSubpath types.StreamName

	// LinkGenerator generates links to alias for a given log stream.
	//
	// If nil, no link annotations will be injected.
	LinkGenerator LinkGenerator

	// Client is the LogDog Butler Client to use for stream creation.
	Client *streamclient.Client

	// Execution describes the current applicaton's execution parameters. This
	// will be used to construct annotation state.
	Execution *annotation.Execution

	// TeeAnnotations, if true, causes all encountered annotations to be
	// tee'd, if teeing is configured.
	TeeAnnotations bool
	// TeeText, if true, causes all encountered non-annotation lines to be
	// tee'd, if teeing is configured.
	TeeText bool

	// MetadataUpdateInterval is the amount of time to wait after stream metadata
	// updates to push the updated metadata protobuf to the metadata stream.
	//
	//	- If this is < 0, metadata will only be pushed at the beginning and end of
	//	  a step.
	//	- If this equals 0, metadata will be pushed every time it's updated.
	//	- If this is 0, DefaultMetadataUpdateInterval will be used.
	MetadataUpdateInterval time.Duration

	// Offline specifies whether parsing happens not at the same time as
	// emitting. If true and CURRENT_TIMESTAMP annotations are not provided
	// then step start/end times are left empty.
	Offline bool

	// CloseSteps specified whether outstanding open steps must be closed.
	CloseSteps bool

	// AnnotationUpdated is synchronously called when the annotation message
	// changes.
	// stepBinary is binary-serialized annopb.Step.
	// stepBinary must not be mutated.
	// The call blocks writing datagrams to the output stream.
	AnnotationUpdated func(stepBinary []byte)
}

Options are the configuration options for a Processor.

type Processor

type Processor struct {
	// contains filtered or unexported fields
}

Processor consumes data from a list of Stream entries and interacts with the supplied Client instance.

A Processor must be instantiated with New.

func New

func New(c context.Context, o Options) *Processor

New instantiates a new Processor.

func (*Processor) Finish

func (p *Processor) Finish() *annotation.State

Finish instructs the Processor to close any outstanding state. This should be called when all automatic state updates have completed in case any steps didn't properly close their state.

Finish will return the closed annotation state that was accumulated during processing.

func (*Processor) IngestLine

func (p *Processor) IngestLine(s *Stream, line string) error

IngestLine ingests a single line of text from an input stream, responding to any annotations encountered.

This method is not goroutine-safe.

func (*Processor) RunStreams

func (p *Processor) RunStreams(streams []*Stream) error

RunStreams executes the Processor, consuming data from its configured streams and forwarding it to LogDog. Run will block until all streams have terminated.

If a stream terminates with an error, or if there is an error processing the stream data, Run will return an error. If multiple Streams fail with errors, an errors.MultiError will be returned. io.EOF does not count as an error.

type Stream

type Stream struct {
	// Reader is the stream data reader. It will be processed until it returns
	// an error or io.EOF.
	Reader io.Reader
	// Name is the logdog stream name.
	Name types.StreamName
	// Tee, if not nil, is a writer where all consumed stream data should be
	// forwarded.
	Tee io.Writer
	// Alias is the base stream name that this stream should alias to.
	Alias string

	// Annotate, if true, causes annotations in this Stream to be captured and
	// an annotation LogDog stream to be emitted.
	Annotate bool

	// StripAnnotations, if true, causes all encountered annotations to be
	// stripped from incoming stream data. Otherwise, those annotations will
	// still advnace the annotation state (if Annotate is true), but will not be
	// included in any output streams.
	StripAnnotations bool

	// EmitAllLink, if true, instructs an "all STDOUT/STDERR" link to be injected
	// into this Stream.
	EmitAllLink bool

	// BufferSize is the size of the read buffer that will be used when processing
	// this stream's data.
	BufferSize int
}

Stream describes a single process stream.