archivist

Documentation

Index

• type Archivist
  • func (a *Archivist) ArchiveTask(ctx context.Context, task *logdog.ArchiveTask) error
• type CLClient
• type Settings
• type SettingsLoader

Types

type Archivist

type Archivist struct {
	// Service is the client to use to communicate with Coordinator's Services
	// endpoint.
	Service logdog.ServicesClient

	// SettingsLoader loads archival settings for a specific project.
	SettingsLoader SettingsLoader

	// Storage is the archival source Storage instance.
	Storage storage.Storage

	// GSClientFactory obtains a Google Storage client for archive generation.
	GSClientFactory func(ctx context.Context, luciProject string) (gs.Client, error)

	// CLClientFactory obtains a Cloud Logging client for log exports.
	// `luciProject` is the ID of the LUCI project to export logs from, and
	// `clProject` is the ID of the Google Cloud project to export logs to.
	CLClientFactory func(ctx context.Context, luciProject, clProject string, onError func(err error)) (CLClient, error)
}

Archivist is a stateless configuration capable of archiving individual log streams.

func (*Archivist) ArchiveTask

func (a *Archivist) ArchiveTask(ctx context.Context, task *logdog.ArchiveTask) error

ArchiveTask processes and executes a single log stream archive task.

If the supplied Context is Done, operation may terminate before completion, returning the Context's error.

type CLClient

type CLClient interface {
	Close() error
	Logger(logID string, opts ...cl.LoggerOption) *cl.Logger
	Ping(context.Context) error
}

CLClient is a general interface for CloudLogging client and intended to enable unit tests to stub out CloudLogging.

type Settings

type Settings struct {
	// GSBase is the base Google Storage path. This includes the bucket name
	// and any associated path.
	GSBase gs.Path
	// GSStagingBase is the base Google Storage path for archive staging. This
	// includes the bucket name and any associated path.
	GSStagingBase gs.Path

	// IndexStreamRange is the maximum number of stream indexes in between index
	// entries. See archive.Manifest for more information.
	IndexStreamRange int
	// IndexPrefixRange is the maximum number of prefix indexes in between index
	// entries. See archive.Manifest for more information.
	IndexPrefixRange int
	// IndexByteRange is the maximum number of stream data bytes in between index
	// entries. See archive.Manifest for more information.
	IndexByteRange int

	// CloudLoggingProjectID is the ID of the Google Cloud Platform project to export
	// logs to.
	//
	// May be empty, if no export is configured.
	CloudLoggingProjectID string
	// CloudLoggingBufferLimit is the maximum number of megabytes that the
	// Cloud Logger will keep in memory per concurrent-task before flushing them
	// out.
	CloudLoggingBufferLimit int
}

Settings defines the archival parameters for a specific archival operation.

In practice, this will be formed from service and project settings.

type SettingsLoader

type SettingsLoader func(ctx context.Context, project string) (*Settings, error)

SettingsLoader returns archival Settings for a given project.