s3

Documentation

Index

• func ContentDispositionedReader(r io.Reader, contentDisposition string) io.Reader
• func ContentTypedReader(r io.Reader, contentType string) io.Reader
• func SizedReader(r io.Reader, size int64) io.Reader
• type ContentDispositioned
• type ContentTyped
• type Filestore
  • func NewFilestore(ctx context.Context, endpoint, bucketName string, opts ...Option) (*Filestore, error)
  • func (f *Filestore) Exists(ctx context.Context, hash string) (bool, error)
  • func (f *Filestore) Fetch(ctx context.Context, hash string) (io.ReadCloser, error)
  • func (f *Filestore) ImgproxyURLSource(hash string) (string, error)
  • func (f *Filestore) Iterate(ctx context.Context, maxBatch int, callback func(hashes []string) error) error
  • func (f *Filestore) Remove(ctx context.Context, hash string) error
  • func (f *Filestore) Size(ctx context.Context, hash string) (int64, error)
  • func (f *Filestore) Store(ctx context.Context, r io.Reader) (string, error)
  • func (f *Filestore) StoreHashed(ctx context.Context, r io.Reader, hash string) error
• type Option
  • func WithBucketAutoCreate() Option
  • func WithBucketLookupDNS() Option
  • func WithBucketLookupPath() Option
  • func WithCredentialsV2(accessKey, secretKey, token string) Option
  • func WithCredentialsV4(accessKey, secretKey, token string) Option
  • func WithRegion(region string) Option
  • func WithSecure() Option
  • func WithTrailingHeaders() Option
  • func WithTransport(transport http.RoundTripper) Option
• type Sized

Functions

func ContentDispositionedReader

func ContentDispositionedReader(r io.Reader, contentDisposition string) io.Reader

ContentDispositionedReader wraps a reader and its content disposition to implement ContentDispositioned.

func ContentTypedReader

func ContentTypedReader(r io.Reader, contentType string) io.Reader

ContentTypedReader wraps a reader and its content type to implement ContentTyped.

func SizedReader

func SizedReader(r io.Reader, size int64) io.Reader

SizedReader wraps a reader and its size of the data to implement Sized.

Types

type ContentDispositioned

type ContentDispositioned interface {
	// ContentDisposition of the data (e.g. "inline; filename=\"test.png\"").
	ContentDisposition() string
}

ContentDispositioned is a reader that also returns the content disposition of the data.

type ContentTyped

type ContentTyped interface {
	// ContentType (media type) of the data.
	ContentType() string
}

ContentTyped is a reader that also returns the content type of the data.

type Filestore

type Filestore struct {
	Client     *minio.Client
	URL        string
	BucketName string
}

Filestore is a file store that stores files in a S3 compatible object storage (e.g. AWS S3 or MinIO).

func NewFilestore

func NewFilestore(ctx context.Context, endpoint, bucketName string, opts ...Option) (*Filestore, error)

NewFilestore creates a new S3 file store.

func (*Filestore) Exists

func (f *Filestore) Exists(ctx context.Context, hash string) (bool, error)

func (*Filestore) Fetch

func (f *Filestore) Fetch(ctx context.Context, hash string) (io.ReadCloser, error)

Fetch gets an object from the S3 bucket by hash and returns a reader for the object. It will stat the object to check for existence. If the object does not exist, it will return ErrNotExist.

func (*Filestore) ImgproxyURLSource

func (f *Filestore) ImgproxyURLSource(hash string) (string, error)

ImgproxyURLSource implements the ImgproxyURLSourcer interface. It returns a URL to the object that will be understood by imgproxy in the form of "s3://bucket-name/object-key".

func (*Filestore) Iterate

func (f *Filestore) Iterate(ctx context.Context, maxBatch int, callback func(hashes []string) error) error

Iterate iterates over all objects in the S3 bucket and calls the callback with a maxBatch amount of hashes. Iteration will stop if the callback returns an error.

func (*Filestore) Remove

func (f *Filestore) Remove(ctx context.Context, hash string) error

Remove removes an object from the S3 bucket by hash. It is not guaranteed to error if the hash does not exist.

func (*Filestore) Size

func (f *Filestore) Size(ctx context.Context, hash string) (int64, error)

Size returns the size of an object in the S3 bucket by hash.

func (*Filestore) Store

func (f *Filestore) Store(ctx context.Context, r io.Reader) (string, error)

Store stores an object in the S3 bucket by hash. The reader should implement Sized for better performance (the client can optimize the operation given the size and reduce memory usage). The reader can implement ContentTyped or ContentDispositioned to set the content type or content disposition of the object.

func (*Filestore) StoreHashed

func (f *Filestore) StoreHashed(ctx context.Context, r io.Reader, hash string) error

type Option

type Option func(*options)

Option is a functional option for creating a S3 file store.

func WithBucketAutoCreate

func WithBucketAutoCreate() Option

WithBucketAutoCreate sets the automatic creation of the bucket if it doesn't exist yet.

func WithBucketLookupDNS

func WithBucketLookupDNS() Option

WithBucketLookupDNS sets the bucket lookup to DNS style.

func WithBucketLookupPath

func WithBucketLookupPath() Option

WithBucketLookupPath sets the bucket lookup to path style. If not set, the bucket lookup is set to auto.

func WithCredentialsV2

func WithCredentialsV2(accessKey, secretKey, token string) Option

WithCredentialsV2 sets the credentials for the S3 client using V2 signatures. The token can be left empty.

func WithCredentialsV4

func WithCredentialsV4(accessKey, secretKey, token string) Option

WithCredentialsV4 sets the credentials for the S3 client using V4 signatures. The V4 signature should be used with MinIO. The token can be left empty.

func WithRegion

func WithRegion(region string) Option

WithRegion sets the region for the S3 client. The region can be left empty for MinIO.

func WithSecure

func WithSecure() Option

WithSecure sets the secure flag for the S3 client (i.e. use HTTPS for the endpoint).

func WithTrailingHeaders

func WithTrailingHeaders() Option

WithTrailingHeaders sets the trailing headers flag for the S3 client.

func WithTransport

func WithTransport(transport http.RoundTripper) Option

WithTransport sets a custom HTTP transport for testing or special needs.

type Sized

type Sized interface {
	// Size of the data that can be read.
	Size() int64
}

Sized is a reader that can return the size of the data.