filesystem

package
v0.22.18 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 23, 2024 License: MIT Imports: 37 Imported by: 116

Documentation

Index

Constants

View Source 
const Scheme = "s3"

Scheme is the URL scheme s3blob registers its URLOpener under on blob.DefaultMux.

Variables

View Source 
var ThumbSizeRegex = regexp.MustCompile(`^(\d+)x(\d+)(t|b|f)?$`)

Functions

func HexEscape added in v0.22.7 

func HexEscape(s string, shouldEscape func(s []rune, i int) bool) string

HexEscape returns s, with all runes for which shouldEscape returns true escaped to "__0xXXX__", where XXX is the hex representation of the rune value. For example, " " would escape to "__0x20__".

Non-UTF-8 strings will have their non-UTF-8 characters escaped to unicode.ReplacementChar; the original value is lost. Please file an issue if you need non-UTF8 support.

Note: shouldEscape takes the whole string as a slice of runes and an index. Passing it a single byte or a single rune doesn't provide enough context for some escape decisions; for example, the caller might want to escape the second "/" in "//" but not the first one. We pass a slice of runes instead of the string or a slice of bytes because some decisions will be made on a rune basis (e.g., encode all non-ASCII runes).

func HexUnescape added in v0.22.7 

func HexUnescape(s string) string

HexUnescape reverses HexEscape.

func NewDefaultV2Config added in v0.22.7 

func NewDefaultV2Config(ctx context.Context) (awsv2.Config, error)

NewDefaultV2Config returns a aws.Config for AWS SDK v2, using the default options.

func OpenBucketV2 added in v0.22.7 

func OpenBucketV2(ctx context.Context, client *s3v2.Client, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucketV2 returns a *blob.Bucket backed by S3, using AWS SDK v2.

func URLEscape added in v0.22.7 

func URLEscape(s string) string

URLEscape uses url.PathEscape to escape s.

func URLUnescape added in v0.22.7 

func URLUnescape(s string) string

URLUnescape reverses URLEscape using url.PathUnescape. If the unescape returns an error, it returns s.

func UseV2 added in v0.22.7 

func UseV2(q url.Values) bool

UseV2 returns true iff the URL parameters indicate that the provider should use the AWS SDK v2.

"awssdk=v1" will force V1. "awssdk=v2" will force V2. No "awssdk" parameter (or any other value) will return the default, currently V1. Note that the default may change in the future.

func V2ConfigFromURLParams added in v0.22.7 

func V2ConfigFromURLParams(ctx context.Context, q url.Values) (awsv2.Config, error)

V2ConfigFromURLParams returns an aws.Config for AWS SDK v2 initialized based on the URL parameters in q. It is intended to be used by URLOpeners for AWS services if UseV2 returns true.

https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/aws#Config

It returns an error if q contains any unknown query parameters; callers should remove any query parameters they know about from q before calling V2ConfigFromURLParams.

The following query options are supported:

  • region: The AWS region for requests; sets WithRegion.
  • profile: The shared config profile to use; sets SharedConfigProfile.
  • endpoint: The AWS service endpoint to send HTTP request.

Types

type BytesReader added in v0.11.0 

type BytesReader struct {
	Bytes []byte
}

BytesReader defines a FileReader from bytes content.

func (*BytesReader) Open added in v0.11.0 

func (r *BytesReader) Open() (io.ReadSeekCloser, error)

Open implements the filesystem.FileReader interface.

type File added in v0.10.0 

type File struct {
	Reader       FileReader
	Name         string
	OriginalName string
	Size         int64
}

File defines a single file io.ReadSeekCloser resource.

The file could be from a local path, multipart/form-data header, etc.

func NewFileFromBytes added in v0.11.0 

func NewFileFromBytes(b []byte, name string) (*File, error)

NewFileFromBytes creates a new File instance from the provided byte slice.

func NewFileFromMultipart added in v0.10.0 

func NewFileFromMultipart(mh *multipart.FileHeader) (*File, error)

NewFileFromMultipart creates a new File from the provided multipart header.

func NewFileFromPath added in v0.10.0 

func NewFileFromPath(path string) (*File, error)

NewFileFromPath creates a new File instance from the provided local file path.

func NewFileFromUrl added in v0.20.0 

func NewFileFromUrl(ctx context.Context, url string) (*File, error)

NewFileFromUrl creates a new File from the provided url by downloading the resource and load it as BytesReader.

Example 

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

file, err := filesystem.NewFileFromUrl(ctx, "https://example.com/image.png")

type FileReader added in v0.10.0 

type FileReader interface {
	Open() (io.ReadSeekCloser, error)
}

FileReader defines an interface for a file resource reader.

type MultipartReader added in v0.10.0 

type MultipartReader struct {
	Header *multipart.FileHeader
}

MultipartReader defines a FileReader from multipart.FileHeader.

func (*MultipartReader) Open added in v0.10.0 

func (r *MultipartReader) Open() (io.ReadSeekCloser, error)

Open implements the filesystem.FileReader interface.

type Options added in v0.22.7 

type Options struct {
	// UseLegacyList forces the use of ListObjects instead of ListObjectsV2.
	// Some S3-compatible services (like CEPH) do not currently support
	// ListObjectsV2.
	UseLegacyList bool
}

Options sets options for constructing a *blob.Bucket backed by fileblob.

type PathReader added in v0.10.0 

type PathReader struct {
	Path string
}

PathReader defines a FileReader from a local file path.

func (*PathReader) Open added in v0.10.0 

func (r *PathReader) Open() (io.ReadSeekCloser, error)

Open implements the filesystem.FileReader interface.

type System 

type System struct {
	// contains filtered or unexported fields
}

func NewLocal 

func NewLocal(dirPath string) (*System, error)

NewLocal initializes a new local filesystem instance.

NB! Make sure to call `Close()` after you are done working with it.

func NewS3 

func NewS3(
	bucketName string,
	region string,
	endpoint string,
	accessKey string,
	secretKey string,
	s3ForcePathStyle bool,
) (*System, error)

NewS3 initializes an S3 filesystem instance.

NB! Make sure to call `Close()` after you are done working with it.

func (*System) Attributes 

func (s *System) Attributes(fileKey string) (*blob.Attributes, error)

Attributes returns the attributes for the file with fileKey path.

func (*System) Close 

func (s *System) Close() error

Close releases any resources used for the related filesystem.

func (*System) Copy added in v0.20.0 

func (s *System) Copy(srcKey, dstKey string) error

Copy copies the file stored at srcKey to dstKey.

If dstKey file already exists, it is overwritten.

func (*System) CreateThumb 

func (s *System) CreateThumb(originalKey string, thumbKey, thumbSize string) error

CreateThumb creates a new thumb image for the file at originalKey location. The new thumb file is stored at thumbKey location.

thumbSize is in the format: - 0xH (eg. 0x100) - resize to H height preserving the aspect ratio - Wx0 (eg. 300x0) - resize to W width preserving the aspect ratio - WxH (eg. 300x100) - resize and crop to WxH viewbox (from center) - WxHt (eg. 300x100t) - resize and crop to WxH viewbox (from top) - WxHb (eg. 300x100b) - resize and crop to WxH viewbox (from bottom) - WxHf (eg. 300x100f) - fit inside a WxH viewbox (without cropping)

func (*System) Delete 

func (s *System) Delete(fileKey string) error

Delete deletes stored file at fileKey location.

func (*System) DeletePrefix 

func (s *System) DeletePrefix(prefix string) []error

DeletePrefix deletes everything starting with the specified prefix.

The prefix could be subpath (ex. "/a/b/") or filename prefix (ex. "/a/b/file_").

func (*System) Exists 

func (s *System) Exists(fileKey string) (bool, error)

Exists checks if file with fileKey path exists or not.

func (*System) GetFile added in v0.12.0 

func (s *System) GetFile(fileKey string) (*blob.Reader, error)

GetFile returns a file content reader for the given fileKey.

NB! Make sure to call `Close()` after you are done working with it.

func (*System) List added in v0.16.0 

func (s *System) List(prefix string) ([]*blob.ListObject, error)

List returns a flat list with info for all files under the specified prefix.

func (*System) Serve 

func (s *System) Serve(res http.ResponseWriter, req *http.Request, fileKey string, name string) error

Serve serves the file at fileKey location to an HTTP response.

If the `download` query parameter is used the file will be always served for download no matter of its type (aka. with "Content-Disposition: attachment").

func (*System) SetContext added in v0.16.0 

func (s *System) SetContext(ctx context.Context)

SetContext assigns the specified context to the current filesystem.

func (*System) Upload 

func (s *System) Upload(content []byte, fileKey string) error

Upload writes content into the fileKey location.

func (*System) UploadFile added in v0.10.0 

func (s *System) UploadFile(file *File, fileKey string) error

UploadFile uploads the provided multipart file to the fileKey location.

func (*System) UploadMultipart added in v0.8.0 

func (s *System) UploadMultipart(fh *multipart.FileHeader, fileKey string) error

UploadMultipart uploads the provided multipart file to the fileKey location.

type URLOpener added in v0.22.7 

type URLOpener struct {
	// UseV2 indicates whether the AWS SDK V2 should be used.
	UseV2 bool

	// Options specifies the options to pass to OpenBucket.
	Options Options
}

URLOpener opens S3 URLs like "s3://mybucket".

The URL host is used as the bucket name.

Use "awssdk=v1" to force using AWS SDK v1, "awssdk=v2" to force using AWS SDK v2, or anything else to accept the default.

For V1, see gocloud.dev/aws/ConfigFromURLParams for supported query parameters for overriding the aws.Session from the URL. For V2, see gocloud.dev/aws/V2ConfigFromURLParams.

func (*URLOpener) OpenBucketURL added in v0.22.7 

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens a blob.Bucket based on u.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.