s3

package
v4.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 12, 2023 License: AGPL-3.0 Imports: 29 Imported by: 1

Documentation

Overview

Package s3 provides an endpoint for connecting to Amazon S3 or an S3-compatible storage

Index

Constants

View Source 
const (
	UserAgentAppName = "pydio.sync.client.s3"
	UserAgentVersion = "1.0"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ChecksumMapper 

type ChecksumMapper interface {
	// Get finds a checksum for a given eTag, returns false if not found
	Get(eTag string) (string, bool)
	// Set stores an eTag checksum couple if it does not already exists
	Set(eTag, checksum string)
	// Purge removes unknown values based on the full list of know values
	Purge(knownETags []string) int
}

ChecksumMapper maintains a mapping of eTags => checksum

type Client 

type Client struct {
	Oc                          nodes.StorageClient
	Bucket                      string
	RootPath                    string
	Host                        string
	ServerRequiresNormalization bool
	// contains filtered or unexported fields
}

Client wraps a Minio Client to speak with an S3-compatible backend

func NewObjectClient added in v4.0.1 

func NewObjectClient(ctx context.Context, oc nodes.StorageClient, host, bucket, rootPath string, options model.EndpointOptions) *Client

func NewS3Mock 

func NewS3Mock(bucketName ...string) *Client

func (*Client) ComputeChecksum 

func (c *Client) ComputeChecksum(ctx context.Context, node *tree.Node) error

func (*Client) CreateNode 

func (c *Client) CreateNode(ctx context.Context, node *tree.Node, updateIfExists bool) (err error)

func (*Client) DeleteNode 

func (c *Client) DeleteNode(ctx context.Context, path string) (err error)

func (*Client) GetEndpointInfo 

func (c *Client) GetEndpointInfo() model.EndpointInfo

func (*Client) GetReaderOn 

func (c *Client) GetReaderOn(ctx context.Context, path string) (out io.ReadCloser, err error)

func (*Client) GetWriterOn 

func (c *Client) GetWriterOn(cancel context.Context, path string, targetSize int64) (out io.WriteCloser, writeDone chan bool, writeErr chan error, err error)

func (*Client) LoadNode 

func (c *Client) LoadNode(ctx context.Context, path string, extendedStats ...bool) (node *tree.Node, err error)

func (*Client) MoveNode 

func (c *Client) MoveNode(ctx context.Context, oldPath string, newPath string) (err error)

func (*Client) SetChecksumMapper 

func (c *Client) SetChecksumMapper(mapper ChecksumMapper, purgeAfterWalk bool)

SetChecksumMapper passes a ChecksumMapper storage that will prevent in-place copy of objects or metadata modification and store md5 for a given eTag locally instead.

func (*Client) SetPlainSizeComputer 

func (c *Client) SetPlainSizeComputer(computer func(nodeUUID string) (int64, error))

SetPlainSizeComputer passes a computer function to extract plain size from an encrypted node

func (*Client) SetServerRequiresNormalization 

func (c *Client) SetServerRequiresNormalization()

SetServerRequiresNormalization is used on MacOS to normalize UTF-8 chars to/from NFC/NFD

func (*Client) SkipRecomputeEtagByCopy 

func (c *Client) SkipRecomputeEtagByCopy()

SkipRecomputeEtagByCopy sets a special behavior to avoir recomputing etags by in-place copying objects on storages that do not support this feature

func (*Client) Stat 

func (c *Client) Stat(ctx context.Context, pa string) (i os.FileInfo, err error)

func (*Client) UpdateNodeUuid 

func (c *Client) UpdateNodeUuid(ctx context.Context, node *tree.Node) (*tree.Node, error)

UpdateNodeUuid makes this endpoint an UuidReceiver

func (*Client) Walk 

func (c *Client) Walk(ctx context.Context, walkFunc model.WalkNodesFunc, root string, recursive bool) (err error)

func (*Client) Watch 

func (c *Client) Watch(recursivePath string) (*model.WatchObject, error)

type MemChecksumMapper 

type MemChecksumMapper struct {
	// contains filtered or unexported fields
}

MemChecksumMapper is an in-memory implementation for ChecksumMapper interface

func NewMemChecksumMapper 

func NewMemChecksumMapper() *MemChecksumMapper

NewMemChecksumMapper instantiates a new ChecksumMapper

func (*MemChecksumMapper) Get 

func (m *MemChecksumMapper) Get(eTag string) (string, bool)

Get finds a checksum by eTag

func (*MemChecksumMapper) Purge 

func (m *MemChecksumMapper) Purge(knownETags []string) int

Purge compares existing eTags to stored eTags and removes unnecessary ones

func (*MemChecksumMapper) Set 

func (m *MemChecksumMapper) Set(eTag, checksum string)

Set stores a checksum for a given eTag

type MultiBucketClient 

type MultiBucketClient struct {
	// contains filtered or unexported fields
}

func NewMultiBucketClient 

func NewMultiBucketClient(ctx context.Context, oc nodes.StorageClient, host string, bucketsFilter string, options model.EndpointOptions) (*MultiBucketClient, error)

NewMultiBucketClient creates an s3 wrapped client that lists buckets as top level folders

func (*MultiBucketClient) ComputeChecksum 

func (m *MultiBucketClient) ComputeChecksum(ctx context.Context, node *tree.Node) (err error)

func (*MultiBucketClient) CreateNode 

func (m *MultiBucketClient) CreateNode(ctx context.Context, node *tree.Node, updateIfExists bool) (err error)

func (*MultiBucketClient) DeleteNode 

func (m *MultiBucketClient) DeleteNode(ctx context.Context, path string) (err error)

func (*MultiBucketClient) GetEndpointInfo 

func (m *MultiBucketClient) GetEndpointInfo() model.EndpointInfo

func (*MultiBucketClient) GetReaderOn 

func (m *MultiBucketClient) GetReaderOn(ctx context.Context, path string) (out io.ReadCloser, err error)

func (*MultiBucketClient) GetWriterOn 

func (m *MultiBucketClient) GetWriterOn(cancel context.Context, path string, targetSize int64) (out io.WriteCloser, writeDone chan bool, writeErr chan error, err error)

func (*MultiBucketClient) LoadNode 

func (m *MultiBucketClient) LoadNode(ctx context.Context, p string, extendedStats ...bool) (node *tree.Node, err error)

func (*MultiBucketClient) MoveNode 

func (m *MultiBucketClient) MoveNode(ctx context.Context, oldPath string, newPath string) (err error)

func (*MultiBucketClient) ProvidesMetadataNamespaces 

func (m *MultiBucketClient) ProvidesMetadataNamespaces() (out []glob.Glob, o bool)

ProvidesMetadataNamespaces implements MetadataProvider interface

func (*MultiBucketClient) SetChecksumMapper 

func (m *MultiBucketClient) SetChecksumMapper(cs ChecksumMapper)

func (*MultiBucketClient) SetPlainSizeComputer 

func (m *MultiBucketClient) SetPlainSizeComputer(computer func(nodeUUID string) (int64, error))

func (*MultiBucketClient) SetServerRequiresNormalization 

func (m *MultiBucketClient) SetServerRequiresNormalization()

func (*MultiBucketClient) SkipRecomputeEtagByCopy 

func (m *MultiBucketClient) SkipRecomputeEtagByCopy()

SkipRecomputeEtagByCopy sets a special behavior to avoir recomputing etags by in-place copying objects on storages that do not support this feature

func (*MultiBucketClient) UpdateNodeUuid 

func (m *MultiBucketClient) UpdateNodeUuid(ctx context.Context, node *tree.Node) (n *tree.Node, err error)

func (*MultiBucketClient) Walk 

func (m *MultiBucketClient) Walk(ctx context.Context, walknFc model.WalkNodesFunc, root string, recursive bool) (err error)

func (*MultiBucketClient) Watch 

func (m *MultiBucketClient) Watch(recursivePath string) (*model.WatchObject, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.