bucket

package
v3.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2024 License: AGPL-3.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// S3 is the value for the S3 storage backend.
	S3 = "s3"

	// GCS is the value for the GCS storage backend.
	GCS = "gcs"

	// Azure is the value for the Azure storage backend.
	Azure = "azure"

	// Swift is the value for the Openstack Swift storage backend.
	Swift = "swift"

	// Filesystem is the value for the filesystem storage backend.
	Filesystem = "filesystem"
)

Variables

View Source 
var (
	SupportedBackends = []string{S3, GCS, Azure, Swift, Filesystem}

	ErrUnsupportedStorageBackend = errors.New("unsupported storage backend")
)

Functions

func NewClient 

func NewClient(ctx context.Context, cfg Config, name string, logger log.Logger, reg prometheus.Registerer) (client objstore.Bucket, err error)

NewClient creates a new bucket client based on the configured backend

func NewUserBucketClient 

func NewUserBucketClient(userID string, bucket objstore.Bucket, cfgProvider TenantConfigProvider) objstore.InstrumentedBucket

NewUserBucketClient returns a bucket client to use to access the storage on behalf of the provided user. The cfgProvider can be nil.

Types

type Config 

type Config struct {
	Backend string `yaml:"backend"`
	// Backends
	S3         s3.Config         `yaml:"s3"`
	GCS        gcs.Config        `yaml:"gcs"`
	Azure      azure.Config      `yaml:"azure"`
	Swift      swift.Config      `yaml:"swift"`
	Filesystem filesystem.Config `yaml:"filesystem"`

	// Not used internally, meant to allow callers to wrap Buckets
	// created using this config
	Middlewares []func(objstore.Bucket) (objstore.Bucket, error) `yaml:"-"`

	// Used to inject additional backends into the config. Allows for this config to
	// be embedded in multiple contexts and support non-object storage based backends.
	ExtraBackends []string `yaml:"-"`
}

Config holds configuration for accessing long-term storage.

func (*Config) RegisterFlags 

func (cfg *Config) RegisterFlags(f *flag.FlagSet)

RegisterFlags registers the backend storage config.

func (*Config) RegisterFlagsWithPrefix 

func (cfg *Config) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet)

func (*Config) Validate 

func (cfg *Config) Validate() error

type PrefixedBucketClient 

type PrefixedBucketClient struct {
	// contains filtered or unexported fields
}

func NewPrefixedBucketClient 

func NewPrefixedBucketClient(bucket objstore.Bucket, prefix string) *PrefixedBucketClient

NewPrefixedBucketClient returns a new PrefixedBucketClient.

func (*PrefixedBucketClient) Attributes 

func (b *PrefixedBucketClient) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)

Attributes returns attributes of the specified object.

func (*PrefixedBucketClient) Close 

func (b *PrefixedBucketClient) Close() error

Close implements io.Closer

func (*PrefixedBucketClient) Delete 

func (b *PrefixedBucketClient) Delete(ctx context.Context, name string) error

Delete removes the object with the given name.

func (*PrefixedBucketClient) Exists 

func (b *PrefixedBucketClient) Exists(ctx context.Context, name string) (bool, error)

Exists checks if the given object exists in the bucket.

func (*PrefixedBucketClient) Get 

func (b *PrefixedBucketClient) Get(ctx context.Context, name string) (io.ReadCloser, error)

Get returns a reader for the given object name.

func (*PrefixedBucketClient) GetRange 

func (b *PrefixedBucketClient) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)

GetRange returns a new range reader for the given object name and range.

func (*PrefixedBucketClient) IsAccessDeniedErr 

func (b *PrefixedBucketClient) IsAccessDeniedErr(err error) bool

IsAccessDeniedErr returns true if access to object is denied.

func (*PrefixedBucketClient) IsObjNotFoundErr 

func (b *PrefixedBucketClient) IsObjNotFoundErr(err error) bool

IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

func (*PrefixedBucketClient) Iter 

func (b *PrefixedBucketClient) Iter(ctx context.Context, dir string, f func(string) error, options ...objstore.IterOption) error

Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full object name including the prefix of the inspected directory. The configured prefix will be stripped before supplied function is applied.

func (*PrefixedBucketClient) Name 

func (b *PrefixedBucketClient) Name() string

Name returns the bucket name for the provider.

func (*PrefixedBucketClient) ReaderWithExpectedErrs 

func (b *PrefixedBucketClient) ReaderWithExpectedErrs(fn objstore.IsOpFailureExpectedFunc) objstore.BucketReader

WithExpectedErrs allows to specify a filter that marks certain errors as expected, so it will not increment thanos_objstore_bucket_operation_failures_total metric.

func (*PrefixedBucketClient) Upload 

func (b *PrefixedBucketClient) Upload(ctx context.Context, name string, r io.Reader) (err error)

Upload the contents of the reader as an object into the bucket.

func (*PrefixedBucketClient) WithExpectedErrs 

func (b *PrefixedBucketClient) WithExpectedErrs(fn objstore.IsOpFailureExpectedFunc) objstore.Bucket

ReaderWithExpectedErrs allows to specify a filter that marks certain errors as expected, so it will not increment thanos_objstore_bucket_operation_failures_total metric.

type SSEBucketClient 

type SSEBucketClient struct {
	// contains filtered or unexported fields
}

SSEBucketClient is a wrapper around a objstore.BucketReader that configures the object storage server-side encryption (SSE) for a given user.

func NewSSEBucketClient 

func NewSSEBucketClient(userID string, bucket objstore.Bucket, cfgProvider TenantConfigProvider) *SSEBucketClient

NewSSEBucketClient makes a new SSEBucketClient. The cfgProvider can be nil.

func (*SSEBucketClient) Attributes 

func (b *SSEBucketClient) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)

Attributes implements objstore.Bucket.

func (*SSEBucketClient) Close 

func (b *SSEBucketClient) Close() error

Close implements objstore.Bucket.

func (*SSEBucketClient) Delete 

func (b *SSEBucketClient) Delete(ctx context.Context, name string) error

Delete implements objstore.Bucket.

func (*SSEBucketClient) Exists 

func (b *SSEBucketClient) Exists(ctx context.Context, name string) (bool, error)

Exists implements objstore.Bucket.

func (*SSEBucketClient) Get 

func (b *SSEBucketClient) Get(ctx context.Context, name string) (io.ReadCloser, error)

Get implements objstore.Bucket.

func (*SSEBucketClient) GetRange 

func (b *SSEBucketClient) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)

GetRange implements objstore.Bucket.

func (*SSEBucketClient) IsAccessDeniedErr 

func (b *SSEBucketClient) IsAccessDeniedErr(err error) bool

IsAccessDeniedErr returns true if access to object is denied.

func (*SSEBucketClient) IsObjNotFoundErr 

func (b *SSEBucketClient) IsObjNotFoundErr(err error) bool

IsObjNotFoundErr implements objstore.Bucket.

func (*SSEBucketClient) Iter 

func (b *SSEBucketClient) Iter(ctx context.Context, dir string, f func(string) error, options ...objstore.IterOption) error

Iter implements objstore.Bucket.

func (*SSEBucketClient) Name 

func (b *SSEBucketClient) Name() string

Name implements objstore.Bucket.

func (*SSEBucketClient) ReaderWithExpectedErrs 

func (b *SSEBucketClient) ReaderWithExpectedErrs(fn objstore.IsOpFailureExpectedFunc) objstore.BucketReader

ReaderWithExpectedErrs implements objstore.Bucket.

func (*SSEBucketClient) Upload 

func (b *SSEBucketClient) Upload(ctx context.Context, name string, r io.Reader) error

Upload the contents of the reader as an object into the bucket.

func (*SSEBucketClient) WithExpectedErrs 

func (b *SSEBucketClient) WithExpectedErrs(fn objstore.IsOpFailureExpectedFunc) objstore.Bucket

WithExpectedErrs implements objstore.Bucket.

type TenantConfigProvider 

type TenantConfigProvider interface {
	// S3SSEType returns the per-tenant S3 SSE type.
	S3SSEType(userID string) string

	// S3SSEKMSKeyID returns the per-tenant S3 KMS-SSE key id or an empty string if not set.
	S3SSEKMSKeyID(userID string) string

	// S3SSEKMSEncryptionContext returns the per-tenant S3 KMS-SSE key id or an empty string if not set.
	S3SSEKMSEncryptionContext(userID string) string
}

TenantConfigProvider defines a per-tenant config provider.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.