logs

Documentation

Index

• func New(ctx context.Context) *taggedLogger
• type TaggedLogger
  • func NewMasterLogger(ctx context.Context) TaggedLogger
• type TestTaggedLogger
  • func NewTest(ctx context.Context) *TestTaggedLogger
  • func (l TestTaggedLogger) Context(ctx context.Context) context.Context
  • func (l *TestTaggedLogger) Errf(format string, args ...any)
  • func (logger TestTaggedLogger) JobID() string
  • func (logger TestTaggedLogger) LogStep(name string, cb func() error) (retErr error)
  • func (l *TestTaggedLogger) Logf(format string, args ...any)
  • func (logger TestTaggedLogger) WithData(data []*common.Input) TaggedLogger
  • func (logger TestTaggedLogger) WithJob(jobID string) TaggedLogger
  • func (logger TestTaggedLogger) WithUserCode() TaggedLogger
  • func (logger TestTaggedLogger) Writer(stream string) io.WriteCloser

Functions

func New

func New(ctx context.Context) *taggedLogger

New adapts context-based logging to the worker.

NOTE(jonathan): This is kind of a workaround to switch over to new logging without any risk of breaking the worker. The logging flow and context flow in the worker are not compatible with each other; I tried to fix this, but each little tweak made a different test break in a different way. To properly fix the context situation in the worker to enable clean logging, we'll have to get a lot more test coverage.

Types

type TaggedLogger

type TaggedLogger interface {
	// Logf logs to stdout and object storage (if stats are enabled), including
	// metadata about the current pipeline and job
	Logf(formatString string, args ...interface{})
	// Errf logs only to stderr
	Errf(formatString string, args ...interface{})

	// LogStep will log before and after the given callback function runs, using
	// the name provided
	LogStep(name string, cb func() error) error

	// These helpers will clone the current logger and construct a new logger that
	// includes the given metadata in log messages.
	WithJob(jobID string) TaggedLogger
	WithData(data []*common.Input) TaggedLogger
	WithUserCode() TaggedLogger

	JobID() string
	Writer(string) io.WriteCloser

	Context(context.Context) context.Context
}

TaggedLogger is an interface providing logging functionality for use by the worker, worker master, and user code processes

func NewMasterLogger

func NewMasterLogger(ctx context.Context) TaggedLogger

NewMasterLogger constructs a TaggedLogger for the given pipeline that includes the 'Master' flag. The root context should come from worker.NewWorker, with projectId, pipelineName, and workerId already set.

type TestTaggedLogger

type TestTaggedLogger struct {
	Logs, Errors []string
	// contains filtered or unexported fields
}

TestTaggedLogger is a taggedLogger that captures (some) logs for testing. It is not safe for concurrent use.

func NewTest

func NewTest(ctx context.Context) *TestTaggedLogger

NewTest creates a new TestTaggedLogger. The context probably wants to be pctx.TestContext, so you can both see (with go test -v) and capture logs.

func (TestTaggedLogger) Context

func (l TestTaggedLogger) Context(ctx context.Context) context.Context

Context allows integration with log.* and m.* for new-style logging in the worker. The logger inside the taggedLogger is added to the provided context, and returned.

func (*TestTaggedLogger) Errf

func (l *TestTaggedLogger) Errf(format string, args ...any)

func (TestTaggedLogger) JobID

func (logger TestTaggedLogger) JobID() string

JobID returns the current job that the logger is configured with.

func (TestTaggedLogger) LogStep

func (logger TestTaggedLogger) LogStep(name string, cb func() error) (retErr error)

LogStep will log before and after the given callback function runs, using the name provided.

func (*TestTaggedLogger) Logf

func (l *TestTaggedLogger) Logf(format string, args ...any)

func (TestTaggedLogger) WithData

func (logger TestTaggedLogger) WithData(data []*common.Input) TaggedLogger

WithData clones the current logger and returns a new one that will include the given data inputs in log statement metadata.

func (TestTaggedLogger) WithJob

func (logger TestTaggedLogger) WithJob(jobID string) TaggedLogger

WithJob clones the current logger and returns a new one that will include the given job ID in log statement metadata.

This is also where we turn off log rate limiting, though WithData actually seems like a slightly more aggressive but still safe choice. If there is a lot of log spam with jobId but not datumId, move it down there. We have tests that test this!

func (TestTaggedLogger) WithUserCode

func (logger TestTaggedLogger) WithUserCode() TaggedLogger

WithUserCode clones the current logger and returns a new one that will include the 'User' flag in log statement metadata.

func (TestTaggedLogger) Writer

func (logger TestTaggedLogger) Writer(stream string) io.WriteCloser

Writer returns an io.WriteCloser that logs each line to the logger. The logs are NOT rate-limited, even if the parent logger is.