s3fs

README

s3fs

Package s3fs provides a S3 implementation for Go1.16 filesystem interface.

Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

The implementation wraps aws sdk go s3 client.

const bucket = "my-bucket"

s, err := session.NewSession()
if err != nil {
    log.Fatal(err)
}

s3fs := s3fs.New(s3.New(s), bucket)

// print out all files in s3 bucket.
_ = fs.WalkDir(s3fs, ".", func(path string, d fs.DirEntry, err error) error {
    if err != nil {
        return err
    }

    if d.IsDir() {
        fmt.Println("dir:", path)
        return nil
    }
    fmt.Println("file:", path)
    return nil
})

Installation

go get github.com/jszwec/s3fs

Requirements

• Go1.16+

Documentation

Overview

Package s3fs provides a S3 implementation for Go1.16 filesystem interface.

Index

• func WithReadSeeker(fsys *S3FS)
• type Client
• type Option
• type S3FS
  • func New(cl Client, bucket string, opts ...Option) *S3FS
  • func (f *S3FS) Open(name string) (fs.File, error)
  • func (f *S3FS) ReadDir(name string) ([]fs.DirEntry, error)
  • func (f *S3FS) Stat(name string) (fs.FileInfo, error)
• Bugs

Functions

func WithReadSeeker

func WithReadSeeker(fsys *S3FS)

WithReadSeeker enables Seek functionality on files opened with this fs.

BUG(WilliamFrei): Seeking on S3 requires reopening the file at the specified position. This can cause problems if the file changed between opening and calling Seek. In that case, fs.ErrNotExist error is returned, which has to be handled by the caller.

Types

type Client

type Client interface {
	HeadObject(ctx context.Context, params *s3.HeadObjectInput, optFns ...func(*s3.Options)) (*s3.HeadObjectOutput, error)
	ListObjects(ctx context.Context, params *s3.ListObjectsInput, optFns ...func(*s3.Options)) (*s3.ListObjectsOutput, error)
	GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
}

Client wraps the s3 client methods that this package is using. This interface may change in the future and should not be relied on by packages using it.

type Option

type Option func(*S3FS)

Option is a function that provides optional features to S3FS.

type S3FS

type S3FS struct {
	// contains filtered or unexported fields
}

S3FS is a S3 filesystem implementation.

S3 has a flat structure instead of a hierarchy. S3FS simulates directories by using prefixes and delims ("/"). Because directories are simulated, ModTime is always a default Time value (IsZero returns true).

func New

func New(cl Client, bucket string, opts ...Option) *S3FS

New returns a new filesystem that works on the specified bucket.

func (*S3FS) Open

func (f *S3FS) Open(name string) (fs.File, error)

Open implements fs.FS.

func (*S3FS) ReadDir

func (f *S3FS) ReadDir(name string) ([]fs.DirEntry, error)

ReadDir implements fs.ReadDirFS.

func (*S3FS) Stat

func (f *S3FS) Stat(name string) (fs.FileInfo, error)

Stat implements fs.StatFS.

Notes

Bugs

• Seeking on S3 requires reopening the file at the specified position. This can cause problems if the file changed between opening and calling Seek. In that case, fs.ErrNotExist error is returned, which has to be handled by the caller.