bucket

package
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 23, 2021 License: Apache-2.0 Imports: 20 Imported by: 4

Documentation

Index

Constants

View Source
const (
	// S3 is the value for the S3 storage backend.
	S3 = "s3"

	// GCS is the value for the GCS storage backend.
	GCS = "gcs"

	// Azure is the value for the Azure storage backend.
	Azure = "azure"

	// Swift is the value for the Openstack Swift storage backend.
	Swift = "swift"

	// Filesystem is the value for the filesystem storage backend.
	Filesystem = "filesystem"
)

Variables

View Source
var (
	ErrUnsupportedStorageBackend = errors.New("unsupported storage backend")
)

Functions

func DeletePrefix added in v1.7.0

func DeletePrefix(ctx context.Context, bkt objstore.Bucket, prefix string, logger log.Logger) (int, error)

DeletePrefix removes all objects with given prefix, recursively. It returns number of deleted objects. If deletion of any object fails, it returns error and stops.

func NewClient

func NewClient(ctx context.Context, cfg Config, name string, logger log.Logger, reg prometheus.Registerer) (client objstore.Bucket, err error)

NewClient creates a new bucket client based on the configured backend

Types

type ClientMock

type ClientMock struct {
	mock.Mock
}

ClientMock mocks objstore.Bucket

func (*ClientMock) Attributes

func (m *ClientMock) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)

ObjectSize mocks objstore.Bucket.Attributes()

func (*ClientMock) Close

func (m *ClientMock) Close() error

Close mocks objstore.Bucket.Close()

func (*ClientMock) Delete

func (m *ClientMock) Delete(ctx context.Context, name string) error

Delete mocks objstore.Bucket.Delete()

func (*ClientMock) Exists

func (m *ClientMock) Exists(ctx context.Context, name string) (bool, error)

Exists mocks objstore.Bucket.Exists()

func (*ClientMock) Get

func (m *ClientMock) Get(ctx context.Context, name string) (io.ReadCloser, error)

Get mocks objstore.Bucket.Get()

func (*ClientMock) GetRange

func (m *ClientMock) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)

GetRange mocks objstore.Bucket.GetRange()

func (*ClientMock) IsObjNotFoundErr

func (m *ClientMock) IsObjNotFoundErr(err error) bool

IsObjNotFoundErr mocks objstore.Bucket.IsObjNotFoundErr()

func (*ClientMock) Iter

func (m *ClientMock) Iter(ctx context.Context, dir string, f func(string) error) error

Iter mocks objstore.Bucket.Iter()

func (*ClientMock) MockDelete

func (m *ClientMock) MockDelete(name string, err error)

func (*ClientMock) MockExists

func (m *ClientMock) MockExists(name string, exists bool, err error)

func (*ClientMock) MockGet

func (m *ClientMock) MockGet(name, content string, err error)

MockGet is a convenient method to mock Get() and Exists()

func (*ClientMock) MockIter

func (m *ClientMock) MockIter(prefix string, objects []string, err error)

MockIter is a convenient method to mock Iter()

func (*ClientMock) MockIterWithCallback

func (m *ClientMock) MockIterWithCallback(prefix string, objects []string, err error, cb func())

MockIterWithCallback is a convenient method to mock Iter() and get a callback called when the Iter API is called.

func (*ClientMock) MockUpload added in v1.7.0

func (m *ClientMock) MockUpload(name string, err error)

func (*ClientMock) Name

func (m *ClientMock) Name() string

Name mocks objstore.Bucket.Name()

func (*ClientMock) Upload

func (m *ClientMock) Upload(ctx context.Context, name string, r io.Reader) error

Upload mocks objstore.Bucket.Upload()

type Config

type Config struct {
	Backend string `yaml:"backend"`
	// Backends
	S3         s3.Config         `yaml:"s3"`
	GCS        gcs.Config        `yaml:"gcs"`
	Azure      azure.Config      `yaml:"azure"`
	Swift      swift.Config      `yaml:"swift"`
	Filesystem filesystem.Config `yaml:"filesystem"`

	// Not used internally, meant to allow callers to wrap Buckets
	// created using this config
	Middlewares []func(objstore.Bucket) (objstore.Bucket, error) `yaml:"-"`
}

Config holds configuration for accessing long-term storage.

func (*Config) RegisterFlags

func (cfg *Config) RegisterFlags(f *flag.FlagSet)

RegisterFlags registers the backend storage config.

func (*Config) RegisterFlagsWithPrefix

func (cfg *Config) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet)

func (*Config) Validate

func (cfg *Config) Validate() error

type UserBucketClient

type UserBucketClient struct {
	UserBucketReaderClient
	// contains filtered or unexported fields
}

UserBucketClient is a wrapper around a objstore.Bucket that prepends writes with a userID

func NewUserBucketClient

func NewUserBucketClient(userID string, bucket objstore.Bucket) *UserBucketClient

func (*UserBucketClient) Close

func (b *UserBucketClient) Close() error

Close implements io.Closer

func (*UserBucketClient) Delete

func (b *UserBucketClient) Delete(ctx context.Context, name string) error

Delete removes the object with the given name.

func (*UserBucketClient) Name

func (b *UserBucketClient) Name() string

Name returns the bucket name for the provider.

func (*UserBucketClient) Upload

func (b *UserBucketClient) Upload(ctx context.Context, name string, r io.Reader) error

Upload the contents of the reader as an object into the bucket.

func (*UserBucketClient) WithExpectedErrs

WithExpectedErrs allows to specify a filter that marks certain errors as expected, so it will not increment thanos_objstore_bucket_operation_failures_total metric.

type UserBucketReaderClient

type UserBucketReaderClient struct {
	// contains filtered or unexported fields
}

UserBucketReaderClient is a wrapper around a objstore.BucketReader that reads from user-specific subfolder.

func (*UserBucketReaderClient) Attributes

Attributes returns attributes of the specified object.

func (*UserBucketReaderClient) Exists

func (b *UserBucketReaderClient) Exists(ctx context.Context, name string) (bool, error)

Exists checks if the given object exists in the bucket.

func (*UserBucketReaderClient) Get

Get returns a reader for the given object name.

func (*UserBucketReaderClient) GetRange

func (b *UserBucketReaderClient) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)

GetRange returns a new range reader for the given object name and range.

func (*UserBucketReaderClient) IsObjNotFoundErr

func (b *UserBucketReaderClient) IsObjNotFoundErr(err error) bool

IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

func (*UserBucketReaderClient) Iter

func (b *UserBucketReaderClient) Iter(ctx context.Context, dir string, f func(string) error) error

Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full object name including the prefix of the inspected directory.

func (*UserBucketReaderClient) ReaderWithExpectedErrs

ReaderWithExpectedErrs allows to specify a filter that marks certain errors as expected, so it will not increment thanos_objstore_bucket_operation_failures_total metric.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL