s3

package
v0.28.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 20, 2023 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	Path                  string `mapstructure:"path"`
	AWSRegion             string `mapstructure:"region"`
	GlobMaxTotalSize      int64  `mapstructure:"glob.max_total_size"`
	GlobMaxObjectsMatched int    `mapstructure:"glob.max_objects_matched"`
	GlobMaxObjectsListed  int64  `mapstructure:"glob.max_objects_listed"`
	GlobPageSize          int    `mapstructure:"glob.page_size"`
	S3Endpoint            string `mapstructure:"endpoint"`
	// contains filtered or unexported fields
}

func ParseConfig

func ParseConfig(props map[string]any) (*Config, error)

type Connector added in v0.27.0

type Connector struct{}

func (Connector) ConsumeAsIterator added in v0.27.0

func (c Connector) ConsumeAsIterator(ctx context.Context, env *connectors.Env, source *connectors.Source, logger *zap.Logger) (connectors.FileIterator, error)

ConsumeAsIterator returns a file iterator over objects stored in gcs.

The credentials are read from following env variables

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_SESSION_TOKEN

Additionally in case env.AllowHostCredentials is true it looks for credentials stored on host machine as well

func (Connector) GetBucketMetadata added in v0.27.0

func (c Connector) GetBucketMetadata(ctx context.Context, req *runtimev1.S3GetBucketMetadataRequest, env *connectors.Env) (string, error)

func (Connector) GetCredentialsInfo added in v0.27.0

func (c Connector) GetCredentialsInfo(ctx context.Context, env *connectors.Env) (provider string, exist bool, err error)

func (Connector) HasAnonymousAccess added in v0.27.0

func (c Connector) HasAnonymousAccess(ctx context.Context, env *connectors.Env, source *connectors.Source) (bool, error)

func (Connector) ListBuckets added in v0.27.0

func (c Connector) ListBuckets(ctx context.Context, env *connectors.Env) ([]string, error)

func (Connector) ListObjects added in v0.27.0

func (Connector) Spec added in v0.27.0

func (c Connector) Spec() connectors.Spec

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL