azio

README

azio

azio is a Go package providing high-level APIs for interacting with Azure blob storage.

Documentation

Index

• Constants
• func BlobOpener(ctx context.Context, p *BlobPath) (func(ctx context.Context, bufSize int) io.ReadSeeker, error)
• func CopyContextConfig(dst, src context.Context) context.Context
• func OpenBlob(ctx context.Context, p *BlobPath, bufSize int) (io.ReadSeeker, error)
• func ReadBlob(ctx context.Context, p *BlobPath) ([]byte, error)
• func ReadBlobRange(ctx context.Context, p *BlobPath, start int64, out []byte) (int, error)
• func Stat(ctx context.Context, p *BlobPath) (fs.FileInfo, error)
• func WriteBlob(ctx context.Context, p *BlobPath, r io.Reader) error
• type BlobConcurrency
  • func DefaultBlobConcurrency() BlobConcurrency
  • func GetBlobConcurrency(ctx context.Context) BlobConcurrency
• type BlobPath
  • func ParseBlobPath(s string) (*BlobPath, error)
  • func (b *BlobPath) Base() string
  • func (b *BlobPath) Client(ctx context.Context) (*azblob.Client, error)
  • func (b *BlobPath) Join(subPath string) *BlobPath
  • func (b *BlobPath) String() string
• type ClientStore
  • func GetClientStore(ctx context.Context) *ClientStore
  • func GlobalClientStore() *ClientStore
  • func NewClientStore(options azblob.ClientOptions) *ClientStore
  • func NewDefaultClientStore() *ClientStore
  • func (c *ClientStore) Client(account string) (*azblob.Client, error)
• type ContextKey

Constants

const (
	DefaultMinBytes       = 1 << 24
	DefaultMaxConcurrency = 64
)

const (
	DefaultBufferSize = 1 << 23
)

const (
	DefaultMaxRetries = 50
)

Functions

func BlobOpener

func BlobOpener(
	ctx context.Context,
	p *BlobPath,
) (func(ctx context.Context, bufSize int) io.ReadSeeker, error)

BlobOpener is like OpenBlob(), but returns a function that can be used to create an arbitrary number of readers without performing any additional I/O each time a reader is created.

This can be useful when multiple Goroutines need to read from the same file, where using a single io.ReadSeeker would not be appropriate.

func CopyContextConfig

func CopyContextConfig(dst, src context.Context) context.Context

CopyContextConfig copies the configuration from src into a new context based on dst.

func OpenBlob

func OpenBlob(
	ctx context.Context,
	p *BlobPath,
	bufSize int,
) (io.ReadSeeker, error)

OpenBlob creates an io.ReadSeeker to access the blob. Sequential reads are optimized by using a buffer.

When the file is seeked to a different offset, the buffer may be cleared, so it should not be excessively large if a lot of seeking is intended.

func ReadBlob

func ReadBlob(ctx context.Context, p *BlobPath) ([]byte, error)

ReadBlob reads an entire blob into memory.

func ReadBlobRange

func ReadBlobRange(ctx context.Context, p *BlobPath, start int64, out []byte) (int, error)

ReadBlobRange reads a range of bytes from the blob into memory. May return fewer bytes if the range is out of bounds.

func Stat

func Stat(ctx context.Context, p *BlobPath) (fs.FileInfo, error)

Stat returns information about the specified file path.

func WriteBlob

func WriteBlob(ctx context.Context, p *BlobPath, r io.Reader) error

WriteBlob writes a file to blob store.

Types

type BlobConcurrency

type BlobConcurrency struct {
	// MinBytes is the minimum number of bytes to read per concurrent request.
	MinBytes int64

	// MaxConcurrency is the maximum number of concurrent requests.
	MaxConcurrency uint16
}

BlobConcurrency configures how many concurrent reads to perform at once while reading a blob.

func DefaultBlobConcurrency

func DefaultBlobConcurrency() BlobConcurrency

DefaultBlobConcurrency creates a BlobConcurrency configuration with sane defaults.

func GetBlobConcurrency

func GetBlobConcurrency(ctx context.Context) BlobConcurrency

GetBlobConcurrency reads the ContextKeyBlobConcurrency from the context, or returns a default value.

type BlobPath

type BlobPath struct {
	Account   string
	Container string
	Path      string
}

A BlobPath points to a blob on an Azure blob storage account.

func ParseBlobPath

func ParseBlobPath(s string) (*BlobPath, error)

ParseBlobPath processes "az://" and "https://" URLs into Azure blob paths.

func (*BlobPath) Base

func (b *BlobPath) Base() string

func (*BlobPath) Client

func (b *BlobPath) Client(ctx context.Context) (*azblob.Client, error)

Client constructs a blob client that can work for this path. The context may be used to override the ClientStore.

func (*BlobPath) Join

func (b *BlobPath) Join(subPath string) *BlobPath

func (*BlobPath) String

func (b *BlobPath) String() string

String gets a human-readable form of the path, compatible with ParseBlobPath.

type ClientStore

type ClientStore struct {
	// contains filtered or unexported fields
}

A ClientStore manages separate blob clients for different storage accounts.

func GetClientStore

func GetClientStore(ctx context.Context) *ClientStore

GetClientStore reads the ContextKeyClientStore from the context, or returns the default, global context store.

func GlobalClientStore

func GlobalClientStore() *ClientStore

func NewClientStore

func NewClientStore(options azblob.ClientOptions) *ClientStore

NewClientStore creates an empty account-to-client mapping which will use the given

func NewDefaultClientStore

func NewDefaultClientStore() *ClientStore

NewDefaultClientStore creates a ClientStore with the default options.

func (*ClientStore) Client

func (c *ClientStore) Client(account string) (*azblob.Client, error)

Client creates or reuses a client for the Azure storage account.

type ContextKey

type ContextKey int

const (
	ContextKeyClientStore ContextKey = iota
	ContextKeyBlobConcurrency
)