coordinator

Documentation

Index

• Variables
• func WithState(stateP *LogStream) interface{ ... }
• type Client
  • func NewClient(c *prpc.Client) *Client
  • func (c *Client) Query(ctx context.Context, project string, path string, o QueryOptions, ...) error
  • func (c *Client) Stream(project string, path types.StreamPath) *Stream
• type GetParam
  • func Index(i types.MessageIndex) GetParam
  • func LimitBytes(limit int) GetParam
  • func LimitCount(limit int) GetParam
  • func NonContiguous() GetParam
• type LogStream
• type QueryCallback
• type QueryOptions
• type QueryStreamType
• type QueryTrinary
• type Stream
  • func (s *Stream) Fetcher(c context.Context, o *fetcher.Options) *fetcher.Fetcher
  • func (s *Stream) Get(ctx context.Context, params ...GetParam) ([]*logpb.LogEntry, error)
  • func (s *Stream) State(ctx context.Context) (*LogStream, error)
  • func (s *Stream) Tail(ctx context.Context, params ...TailParam) (*logpb.LogEntry, error)
• type StreamState
• type TailParam
  • func Complete() TailParam

Variables

var (
	// ErrNoAccess is returned from methods to indicate that the requested
	// operation could not be performed because the user does not have access
	// to that data or function.
	ErrNoAccess = errors.New("coordinator: no access")

	// ErrNoSuchStream is returned when the requested strem path does not exist.
	ErrNoSuchStream = errors.New("coordinator: no such stream")
)

var (
	// Scopes is the set of scopes needed for the Coordinator user endpoints.
	Scopes = []string{
		auth.OAuthScopeEmail,
	}
)

Functions

func WithState

func WithState(stateP *LogStream) interface {
	GetParam
	TailParam
}

WithState returns a Get/Tail parameter that loads the log stream's state into the supplied LogState pointer.

Types

type Client

type Client struct {
	// C is the underlying LogsClient interface.
	C logdog.LogsClient
	// Host is the LogDog host. This is loaded from the pRPC client in NewClient.
	Host string
}

Client wraps a Logs client with user-friendly methods.

Each method should operate independently, so calling methods from different goroutines must not cause any problems.

func NewClient

func NewClient(c *prpc.Client) *Client

NewClient returns a new Client instance bound to a pRPC Client.

func (*Client) Query

func (c *Client) Query(ctx context.Context, project string, path string, o QueryOptions, cb QueryCallback) error

Query executes a query, invoking the supplied callback once for each query result.

The path is the query parameter.

The path expression may substitute a glob ("*") for a specific path component. That is, any stream that matches the remaining structure qualifies regardless of its value in that specific positional field.

An unbounded wildcard may appear as a component at the end of both the prefix and name query components. "**" matches all remaining components.

If the supplied path query does not contain a path separator ("+"), it will be treated as if the prefix is "**".

Examples:

• Empty ("") will return all streams.
• **/+/** will return all streams.
• foo/bar/** will return all streams with the "foo/bar" prefix.
• foo/bar/**/+/baz will return all streams beginning with the "foo/bar" prefix and named "baz" (e.g., "foo/bar/qux/lol/+/baz")
• foo/bar/+/** will return all streams with a "foo/bar" prefix.
• foo/*/+/baz will return all streams with a two-component prefix whose first value is "foo" and whose name is "baz".
• foo/bar will return all streams whose name is "foo/bar".
• */* will return all streams with two-component names.

func (*Client) Stream

func (c *Client) Stream(project string, path types.StreamPath) *Stream

Stream returns a Stream instance for the named stream.

type GetParam

type GetParam interface {
	// contains filtered or unexported methods
}

GetParam is a condition or parameter to apply to a Get request.

func Index

func Index(i types.MessageIndex) GetParam

Index returns a stream Get parameter that causes the Get request to retrieve logs starting at the requested stream index instead of the default, zero.

func LimitBytes

func LimitBytes(limit int) GetParam

LimitBytes applies a byte constraint to the returned logs. If the supplied limit is <= 0, then no byte constraint will be applied and the server will choose how many logs to return.

func LimitCount

func LimitCount(limit int) GetParam

LimitCount applies a count constraint to the returned logs. If the supplied limit is <= 0, then no count constraint will be applied and the server will choose how many logs to return.

func NonContiguous

func NonContiguous() GetParam

NonContiguous returns a stream Get parameter that causes the Get request to allow non-contiguous records to be returned. By default, only contiguous records starting from the specific Index will be returned.

By default, a log stream will return only contiguous records starting at the requested index. For example, if a stream had: {0, 1, 2, 4, 5} and a request was made for index 0, Get will return {0, 1, 2}, for index 3 {}, and for index 4 {4, 5}.

If NonContiguous is true, a request for 0 will return {0, 1, 2, 4, 5} and so on.

Log entries generally should not be missing, but may be if either the logs are still streaming (since they can be ingested out of order) or if a data loss or corruption occurs.

type LogStream

type LogStream struct {
	// Project is the log stream's project.
	Project string
	// Path is the path of the log stream.
	Path types.StreamPath

	// Desc is the log stream's descriptor.
	//
	// TODO(iannucci): Do not embed proto messages! This should be a pointer, and
	// all existing shallow clones of it should be converted to proto.Clone or
	// similar.
	Desc logpb.LogStreamDescriptor

	// State is the stream's current state.
	State StreamState
}

LogStream is returned metadata about a log stream.

type QueryCallback

type QueryCallback func(r *LogStream) bool

QueryCallback is a callback method type that is used in query requests.

If it returns false, additional callbacks and queries will be aborted.

type QueryOptions

type QueryOptions struct {
	// Tags is the list of tags to require. The value may be empty if key presence
	// is all that is being asserted.
	Tags map[string]string
	// ContentType, if not empty, restricts results to streams with the supplied
	// content type.
	ContentType string

	// StreamType, if not STAny, is the stream type to query for.
	StreamType QueryStreamType

	// Purged, if not QBoth, selects logs streams that are/aren't purged.
	Purged QueryTrinary

	// State, if true, requests that the query results include the log streams'
	// state.
	State bool
}

QueryOptions is the set of query options that can accompany a query.

type QueryStreamType

type QueryStreamType int

QueryStreamType is a 3-value query option type.

const (
	// Any means that the value should not have an effect.
	Any QueryStreamType = iota
	// Text selects only text streams.
	Text
	// Binary selects only binary streams.
	Binary
	// Datagram selects only datagram streams.
	Datagram
)

type QueryTrinary

type QueryTrinary int

QueryTrinary is a 3-value query option type.

const (
	// Both means that the value should not have an effect.
	Both QueryTrinary = iota
	// Yes is a positive effect.
	Yes
	// No is a negative effect.
	No
)

type Stream

type Stream struct {
	// contains filtered or unexported fields
}

Stream is an interface to Coordinator stream-level commands. It is bound to and operates on a single log stream path.

func (*Stream) Fetcher

func (s *Stream) Fetcher(c context.Context, o *fetcher.Options) *fetcher.Fetcher

Fetcher returns a Fetcher implementation for this Stream.

If you pass a nil fetcher.Options, a default option set will be used. The o.Source field will always be overwritten to be based off this stream.

func (*Stream) Get

func (s *Stream) Get(ctx context.Context, params ...GetParam) ([]*logpb.LogEntry, error)

Get retrieves log stream entries from the Coordinator. The supplied parameters shape which entries are requested and what information is returned.

func (*Stream) State

func (s *Stream) State(ctx context.Context) (*LogStream, error)

State fetches the LogStreamDescriptor for a given log stream.

func (*Stream) Tail

func (s *Stream) Tail(ctx context.Context, params ...TailParam) (*logpb.LogEntry, error)

Tail performs a tail call, returning the last log entry in the stream. If stateP is not nil, the stream's state will be requested and loaded into the variable.

type StreamState

type StreamState struct {
	// Created is the time, represented as a UTC RFC3339 string, when the log
	// stream was created.
	Created time.Time
	// Updated is the time, represented as a UTC RFC3339 string, when the log
	// stream was last updated.
	Updated time.Time

	// TerminalIndex is the stream index of the log stream's terminal message. If
	// its value is <0, then the log stream has not terminated yet.
	// In this case, FinishedIndex is the index of that terminal message.
	TerminalIndex types.MessageIndex

	// Archived is true if the stream is marked as archived.
	Archived bool
	// ArchiveIndexURL is the Google Storage URL where the log stream's index is
	// archived.
	ArchiveIndexURL string
	// ArchiveStreamURL is the Google Storage URL where the log stream's raw
	// stream data is archived. If this is not empty, the log stream is considered
	// archived.
	ArchiveStreamURL string
	// ArchiveDataURL is the Google Storage URL where the log stream's assembled
	// data is archived. If this is not empty, the log stream is considered
	// archived.
	ArchiveDataURL string

	// Purged indicates the purged state of a log. A log that has been purged is
	// only acknowledged to administrative clients.
	Purged bool
}

StreamState represents the client-side state of the log stream.

It is a type-promoted version of logdog.LogStreamState.

type TailParam

type TailParam interface {
	// contains filtered or unexported methods
}

TailParam is a condition or parameter to apply to a Tail request.

func Complete

func Complete() TailParam

Complete instructs the Tail call to retrieve a complete record.

If frgmented, the resulting record will be manufactured from its composite parts, and will not actually represent any single record in the log stream. The time offset, prefix and stream indices, sequence number, and content will be derived from the initial log entry in the composite set.

If the log stream is a TEXT or BINARY stream, no behavior change will occur, and the last log record will be returned.

If the log stream is a DATAGRAM stream and the Tail record is parked partial, additional log entries will be fetched via Get and the full log stream will be assembled. If the partial datagram entry is the "last" in its sequence, the full datagram ending with it will be returned. If it's partial in the middle of a sequence, the previous complete datagram will be returned.