storage

Documentation

Index

• Constants
• func WithRetryPolicy(policy RetryPolicy) s3Opt
• type Backend
• type BackendMethod
• type RetryPolicy
• type S3
  • func NewS3(cfg S3Config, opts ...s3Opt) *S3
  • func (s *S3) Delete(ctx context.Context, path string) error
  • func (s *S3) Get(ctx context.Context, path string) (io.ReadCloser, error)
  • func (s *S3) Size(ctx context.Context, path string) (int64, error)
  • func (s *S3) Upload(ctx context.Context, file io.Reader, path string) error
• type S3Client
• type S3Config
• type Swift
  • func NewSwift(cfg SwiftConfig) (*Swift, error)
  • func (s *Swift) Delete(ctx context.Context, path string) error
  • func (s *Swift) Get(ctx context.Context, path string) (io.ReadCloser, error)
  • func (s *Swift) Size(ctx context.Context, path string) (int64, error)
  • func (s *Swift) Upload(ctx context.Context, reader io.Reader, path string) error
• type SwiftConfig

Constants

const (
	NotFoundErrCode = "NotFound"
)

Functions

func WithRetryPolicy

func WithRetryPolicy(policy RetryPolicy) s3Opt

WithRetryPolicy is an option to constructor NewS3 to add a Retry Policy impacting GET operations

Types

type Backend

type Backend interface {
	Get(ctx context.Context, path string) (io.ReadCloser, error)
	Upload(ctx context.Context, file io.Reader, path string) error
	Size(ctx context.Context, path string) (int64, error)
	Delete(ctx context.Context, path string) error
}

Backend represents something which is able to store files on an object storage service

type BackendMethod

type BackendMethod string

BackendMethod represents the name of a Method included in the Backend interface

const (
	GetMethod    BackendMethod = "Get"
	UploadMethod BackendMethod = "Upload"
	SizeMethod   BackendMethod = "Size"
	DeleteMethod BackendMethod = "Delete"
)

type RetryPolicy

type RetryPolicy struct {
	WaitDuration   time.Duration
	Attempts       int
	MethodHandlers map[BackendMethod][]string
}

type S3

type S3 struct {
	// contains filtered or unexported fields
}

func NewS3

func NewS3(cfg S3Config, opts ...s3Opt) *S3

func (*S3) Delete

func (s *S3) Delete(ctx context.Context, path string) error

func (*S3) Get

func (s *S3) Get(ctx context.Context, path string) (io.ReadCloser, error)

func (*S3) Size

func (s *S3) Size(ctx context.Context, path string) (int64, error)

Size returns the size of the content of the object. A retry mecanism is implemented because of the eventual consistency of S3 backends NotFound error are sometimes returned when the object was just uploaded.

func (*S3) Upload

func (s *S3) Upload(ctx context.Context, file io.Reader, path string) error

type S3Client

type S3Client interface {
	GetObjectRequest(input *s3.GetObjectInput) s3.GetObjectRequest
	HeadObjectRequest(input *s3.HeadObjectInput) s3.HeadObjectRequest
	DeleteObjectRequest(input *s3.DeleteObjectInput) s3.DeleteObjectRequest
}

type S3Config

type S3Config struct {
	AK       string
	SK       string
	Region   string
	Endpoint string
	Bucket   string
}

type Swift

type Swift struct {
	// contains filtered or unexported fields
}

func NewSwift

func NewSwift(cfg SwiftConfig) (*Swift, error)

NewSwift instantiate a new connection to a Swift object storage. The configuration is taken from the environment. Refer to the github.com/ncw/swift documentation for more information.

func (*Swift) Delete

func (s *Swift) Delete(ctx context.Context, path string) error

func (*Swift) Get

func (s *Swift) Get(ctx context.Context, path string) (io.ReadCloser, error)

func (*Swift) Size

func (s *Swift) Size(ctx context.Context, path string) (int64, error)

Size returns the size of the content of the object. A retry mecanism is implemented because of the eventual consistency of Swift backends NotFound error are sometimes returned when the object was just uploaded.

func (*Swift) Upload

func (s *Swift) Upload(ctx context.Context, reader io.Reader, path string) error

type SwiftConfig

type SwiftConfig struct {
	Prefix    string
	Container string
	ChunkSize int64
}