s3stream

README

s3stream V3

What is it?

Streaming client for Amazon AWS S3

Golang library that allows Get and Put operations from/to Amazon S3 in streaming fashion.

V3

V3 of this package is compatible with https://github.com/aws/aws-sdk-go-v2

How to install

go get github.com/indiependente/s3stream/v3

Usage

Get:

rc, err := store.Get(ctx, prefix, bucketname, filename)

rc is an io.ReadCloser which you can stream from.

Put:

n, err := store.Put(ctx, prefix, bucketname, filename, r)

r is an io.Reader and Put will stream its content to the specified file in the desired bucket.

Look at the tests for more info on its usage.

You can run the tests locally by using Minio (https://github.com/minio/minio). Example:

docker compose up  
go test -race -cover ./...
docker compose down

Documentation

Index

• func WithReadPartSize(size int64) func(s *Store) error
• type Store
  • func NewStore(conf aws.Config, opts ...func(*Store) error) (Store, error)
  • func NewStoreWithClient(client *s3.Client, opts ...func(*Store) error) (Store, error)
  • func (s Store) Get(ctx context.Context, prefix, bucketname, filename string) (io.ReadCloser, error)
  • func (s Store) Put(ctx context.Context, prefix, bucketname, filename string, r io.Reader) (int, error)

Functions

func WithReadPartSize

func WithReadPartSize(size int64) func(s *Store) error

WithReadPartSize allows to set the part size of the multipart get operation. The part size cannot be greater than the AWS MaxPartSize constant of 5GB. https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html

Types

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the S3 implementation of the Store interface.

func NewStore

func NewStore(conf aws.Config, opts ...func(*Store) error) (Store, error)

NewStore returns a Store given the input options.

func NewStoreWithClient

func NewStoreWithClient(client *s3.Client, opts ...func(*Store) error) (Store, error)

NewStoreWithClient returns a Store given the input client.

func (Store) Get

func (s Store) Get(ctx context.Context, prefix, bucketname, filename string) (io.ReadCloser, error)

Get returns the content of the file in input reading it from the underlying S3 bucket.

func (Store) Put

func (s Store) Put(ctx context.Context, prefix, bucketname, filename string, r io.Reader) (int, error)

Put stores the content of the reader in input with the specified name. Returns number of bytes written and an error if any.