s3

Documentation

Overview

Package s3 enables in-order streaming of data to and from files stored in Amazon S3.

Index

• Variables
• type Options
• type Storage
  • func New(name string, opts *Options) (s *Storage, e error)
  • func (s *Storage) Close() error
  • func (s *Storage) Read(p []byte) (int, error)
  • func (s *Storage) Seek(offset int64, whence int) (int64, error)
  • func (s *Storage) Sync() error
  • func (s *Storage) Write(p []byte) (int, error)

Variables

var (
	// ErrSeekPositionNotSupported occurs when a syncable type receives a
	// seek destination which it can not fulfil to. These syncable types
	// usually only support seeking to the beginning or to the end, as they
	// write to and read from multiple data files.
	ErrSeekPositionNotSupported = errors.New("Seek position is not supported.")
)

Types

type Options

type Options struct {
	// MinSize is used to determine the number of megabytes to write to
	// a file before the file should be rotated, and a new file used. This
	// is only available on certain types which support rotation of files,
	// but do not support appendable writes. The file is overwritten on
	// each write using the buffered data. MinSize should be specified in
	// megabytes.
	MinSize int
}

Options represents syncing configuration options.

type Storage

type Storage struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Storage represents a Amazon S3 reader and writer.

func New

func New(name string, opts *Options) (s *Storage, e error)

New creates a new Syncable storage instance for reading and writing.

func (*Storage) Close

func (s *Storage) Close() error

Close closes the underlying Syncable storage data stream, and prevents any further reads of writes to the Storage instance.

func (*Storage) Read

func (s *Storage) Read(p []byte) (int, error)

Read reads up to len(p) bytes into p, reading from the Storage data stream, and automatically rotating files when the end of a file is reached. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. If the Storage has reached the final log file, then an EOF error will be returned.

func (*Storage) Seek

func (s *Storage) Seek(offset int64, whence int) (int64, error)

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekE8nd means relative to the end. Seek returns the new offset relative to the start of the file and an error, if any. In some storage types, Seek only supports seeking to the beginning and end of the data stream.

func (*Storage) Sync

func (s *Storage) Sync() error

Sync ensures that any buffered data in the stream is committed to stable storage. In some Syncable types, Seek does nothing, as the data is written and persisted immediately when a Write is made. On Syncable types which support BufferWrites, then Sync will ensure the data stored is flushed.

func (*Storage) Write

func (s *Storage) Write(p []byte) (int, error)

Write writes len(p) bytes from p to the underlying Storage data stream. It returns the number of bytes written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. Write will always append data to the end of the Storage data stream, ensuring data is append-only and never overwritten.