s3

package
v1.58.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2022 License: Apache-2.0 Imports: 59 Imported by: 0

Documentation

Overview

Package s3 provides an interface to Amazon S3 oject storage

Index

Constants

View Source 
const (
	DefaultAccessKey = "minioadmin"
	DefaultSecretKey = "minioadmin"
)

Default access and secret keys.

View Source 
const (
	// AccountOn indicates that credentials are enabled
	AccountOn = "on"
	// AccountOff indicates that credentials are disabled
	AccountOff = "off"
)

Variables

View Source 
var (
	ErrInvalidAccessKeyLength = fmt.Errorf("access key length should be between %d and %d", accessKeyMinLen, accessKeyMaxLen)
	ErrInvalidSecretKeyLength = fmt.Errorf("secret key length should be between %d and %d", secretKeyMinLen, secretKeyMaxLen)
)

Common errors generated for access and secret key validation.

View Source 
var (
	DefaultCredentials = Credentials{
		AccessKey: DefaultAccessKey,
		SecretKey: DefaultSecretKey,
	}
)

Default access credentials

View Source 
var ErrInvalidDuration = errors.New("invalid token expiry")

ErrInvalidDuration invalid token expiry

Functions

func ExpToInt64 added in v1.56.0 

func ExpToInt64(expI interface{}) (expAt int64, err error)

ExpToInt64 - convert input interface value to int64.

func IsAccessKeyValid added in v1.56.0 

func IsAccessKeyValid(accessKey string) bool

IsAccessKeyValid - validate access key for right length.

func IsSecretKeyValid added in v1.56.0 

func IsSecretKeyValid(secretKey string) bool

IsSecretKeyValid - validate secret key for right length.

func JWTSignWithAccessKey added in v1.56.0 

func JWTSignWithAccessKey(accessKey string, m map[string]interface{}, tokenSecret string) (string, error)

JWTSignWithAccessKey - generates a session token.

func NewFs 

func NewFs(ctx context.Context, name, root string, m configmap.Mapper) (fs.Fs, error)

NewFs constructs an Fs from the path, bucket:path

func ParseWithClaims added in v1.56.0 

func ParseWithClaims(tokenStr string, claims *MapClaims, fn func(*MapClaims) ([]byte, error)) error

ParseWithClaims - parse the token string, valid methods.

func ParseWithStandardClaims added in v1.56.0 

func ParseWithStandardClaims(tokenStr string, claims *StandardClaims, key []byte) error

ParseWithStandardClaims - parse the token string, valid methods.

Types

type AssumeRoleWithWebIdentityResponse 

type AssumeRoleWithWebIdentityResponse struct {
	XMLName          xml.Name          `xml:"https://sts.amazonaws.com/doc/2011-06-15/ AssumeRoleWithWebIdentityResponse" json:"-"`
	Result           WebIdentityResult `xml:"AssumeRoleWithWebIdentityResult"`
	ResponseMetadata struct {
		RequestID string `xml:"RequestId,omitempty"`
	} `xml:"ResponseMetadata,omitempty"`
}

AssumeRoleWithWebIdentityResponse the struct of the STS WebIdentity call response

type AssumedRoleUser 

type AssumedRoleUser struct {
	Arn           string
	AssumedRoleID string `xml:"AssumeRoleId"`
}

AssumedRoleUser - The identifiers for the temporary security credentials that the operation returns. Please also see https://docs.aws.amazon.com/goto/WebAPI/sts-2011-06-15/AssumedRoleUser

type Credentials added in v1.56.0 

type Credentials struct {
	AccessKey    string    `xml:"AccessKeyId" json:"accessKey,omitempty"`
	SecretKey    string    `xml:"SecretAccessKey" json:"secretKey,omitempty"`
	Expiration   time.Time `xml:"Expiration" json:"expiration,omitempty"`
	SessionToken string    `xml:"SessionToken" json:"sessionToken,omitempty"`
	Status       string    `xml:"-" json:"status,omitempty"`
	ParentUser   string    `xml:"-" json:"parentUser,omitempty"`
	Groups       []string  `xml:"-" json:"groups,omitempty"`
}

Credentials holds access and secret keys.

func CreateCredentials added in v1.56.0 

func CreateCredentials(accessKey, secretKey string) (cred Credentials, err error)

CreateCredentials returns new credential with the given access key and secret key. Error is returned if given access key or secret key are invalid length.

func CreateNewCredentialsWithMetadata added in v1.56.0 

func CreateNewCredentialsWithMetadata(accessKey, secretKey string, m map[string]interface{}, tokenSecret string) (cred Credentials, err error)

CreateNewCredentialsWithMetadata - creates new credentials using the specified access & secret keys and generate a session token if a secret token is provided.

func GetNewCredentials added in v1.56.0 

func GetNewCredentials() (cred Credentials, err error)

GetNewCredentials generates and returns new credential.

func GetNewCredentialsWithMetadata added in v1.56.0 

func GetNewCredentialsWithMetadata(m map[string]interface{}, tokenSecret string) (cred Credentials, err error)

GetNewCredentialsWithMetadata generates and returns new credential with expiry.

func (Credentials) Equal added in v1.56.0 

func (cred Credentials) Equal(ccred Credentials) bool

Equal - returns whether two credentials are equal or not.

func (Credentials) IsExpired added in v1.56.0 

func (cred Credentials) IsExpired() bool

IsExpired - returns whether Credential is expired or not.

func (Credentials) IsServiceAccount added in v1.56.0 

func (cred Credentials) IsServiceAccount() bool

IsServiceAccount - returns whether credential is a service account or not

func (Credentials) IsTemp added in v1.56.0 

func (cred Credentials) IsTemp() bool

IsTemp - returns whether credential is temporary or not.

func (Credentials) IsValid added in v1.56.0 

func (cred Credentials) IsValid() bool

IsValid - returns whether credential is valid or not.

func (Credentials) String added in v1.56.0 

func (cred Credentials) String() string

type Fs 

type Fs struct {
	// contains filtered or unexported fields
}

Fs represents a remote s3 server

func (*Fs) CleanUp 

func (f *Fs) CleanUp(ctx context.Context) (err error)

CleanUp removes all pending multipart uploads older than 24 hours

func (*Fs) Command 

func (f *Fs) Command(ctx context.Context, name string, arg []string, opt map[string]string) (out interface{}, err error)

Command the backend to run a named command

The command run is name args may be used to read arguments from opts may be used to read optional arguments from

The result should be capable of being JSON encoded If it is a string or a []string it will be shown to the user otherwise it will be JSON encoded and shown to the user like that

func (*Fs) Copy 

func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Copy src to this remote using server-side copy operations.

This is stored with the remote path given

It returns the destination Object and a possible error

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantCopy

func (*Fs) Features 

func (f *Fs) Features() *fs.Features

Features returns the optional features of this Fs

func (*Fs) Hashes 

func (f *Fs) Hashes() hash.Set

Hashes returns the supported hash sets.

func (*Fs) List 

func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error)

List the objects and directories in dir into entries. The entries can be returned in any order but should be for a complete directory.

dir should be "" to list the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

func (*Fs) ListR 

func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (err error)

ListR lists the objects and directories of the Fs starting from dir recursively into out.

dir should be "" to start from the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

It should call callback for each tranche of entries read. These need not be returned in any particular order. If callback returns an error then the listing will stop immediately.

Don't implement this unless you have a more efficient way of listing recursively than doing a directory traversal.

func (*Fs) Mkdir 

func (f *Fs) Mkdir(ctx context.Context, dir string) error

Mkdir creates the bucket if it doesn't exist

func (*Fs) Name 

func (f *Fs) Name() string

Name of the remote (as passed into NewFs)

func (*Fs) NewObject 

func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error)

NewObject finds the Object at remote. If it can't be found it returns the error fs.ErrorObjectNotFound.

func (*Fs) Precision 

func (f *Fs) Precision() time.Duration

Precision of the remote 

func (f *Fs) PublicLink(ctx context.Context, remote string, expire fs.Duration, unlink bool) (link string, err error)

PublicLink generates a public link to the remote path (usually readable by anyone)

func (*Fs) Put 

func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

Put the Object into the bucket

func (*Fs) PutStream 

func (f *Fs) PutStream(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

PutStream uploads to the remote path with the modTime given of indeterminate size

func (*Fs) Rmdir 

func (f *Fs) Rmdir(ctx context.Context, dir string) error

Rmdir deletes the bucket if the fs is at the root

Returns an error if it isn't empty

func (*Fs) Root 

func (f *Fs) Root() string

Root of the remote (as passed into NewFs)

func (*Fs) String 

func (f *Fs) String() string

String converts this Fs to a string

type IAMProvider 

type IAMProvider struct {
	// contains filtered or unexported fields
}

IAMProvider credential provider for oidc

func (*IAMProvider) IsExpired 

func (t *IAMProvider) IsExpired() bool

IsExpired test

func (*IAMProvider) Retrieve 

func (t *IAMProvider) Retrieve() (credentials.Value, error)

Retrieve credentials

type MapClaims added in v1.56.0 

type MapClaims struct {
	AccessKey string `json:"accessKey,omitempty"`
	jwtgo.MapClaims
}

MapClaims - implements custom unmarshaller

func ExtractClaims added in v1.56.0 

func ExtractClaims(token, secretKey string) (*MapClaims, error)

ExtractClaims extracts JWT claims from a security token using a secret key

func NewMapClaims added in v1.56.0 

func NewMapClaims() *MapClaims

NewMapClaims - Initializes a new map claims

func (*MapClaims) GetAccessKey added in v1.56.0 

func (c *MapClaims) GetAccessKey() string

GetAccessKey will return the access key. If nil an empty string will be returned.

func (*MapClaims) Lookup added in v1.56.0 

func (c *MapClaims) Lookup(key string) (value string, ok bool)

Lookup returns the value and if the key is found.

func (*MapClaims) Map added in v1.56.0 

func (c *MapClaims) Map() map[string]interface{}

Map returns underlying low-level map claims.

func (*MapClaims) MarshalJSON added in v1.56.0 

func (c *MapClaims) MarshalJSON() ([]byte, error)

MarshalJSON marshals the MapClaims struct

func (*MapClaims) SetAccessKey added in v1.56.0 

func (c *MapClaims) SetAccessKey(accessKey string)

SetAccessKey sets access key as jwt subject and custom "accessKey" field.

func (*MapClaims) SetExpiry added in v1.56.0 

func (c *MapClaims) SetExpiry(t time.Time)

SetExpiry sets expiry in unix epoch secs

func (*MapClaims) Valid added in v1.56.0 

func (c *MapClaims) Valid() error

Valid - implements https://godoc.org/github.com/golang-jwt/jwt#Claims compatible claims interface, additionally validates "accessKey" fields.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object describes a s3 object

func (*Object) Fs 

func (o *Object) Fs() fs.Info

Fs returns the parent Fs

func (*Object) GetTier 

func (o *Object) GetTier() string

GetTier returns storage class as string

func (*Object) Hash 

func (o *Object) Hash(ctx context.Context, t hash.Type) (string, error)

Hash returns the Md5sum of an object returning a lowercase hex string

func (*Object) MimeType 

func (o *Object) MimeType(ctx context.Context) string

MimeType of an Object if known, "" otherwise

func (*Object) ModTime 

func (o *Object) ModTime(ctx context.Context) time.Time

ModTime returns the modification time of the object

It attempts to read the objects mtime and if that isn't present the LastModified returned in the http headers

func (*Object) Open 

func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.ReadCloser, err error)

Open an object for read

func (*Object) Remote 

func (o *Object) Remote() string

Remote returns the remote path

func (*Object) Remove 

func (o *Object) Remove(ctx context.Context) error

Remove an object

func (*Object) SetModTime 

func (o *Object) SetModTime(ctx context.Context, modTime time.Time) error

SetModTime sets the modification time of the local fs object

func (*Object) SetTier 

func (o *Object) SetTier(tier string) (err error)

SetTier performs changing storage class

func (*Object) Size 

func (o *Object) Size() int64

Size returns the size of an object in bytes

func (*Object) Storable 

func (o *Object) Storable() bool

Storable raturns a boolean indicating if this object is storable

func (*Object) String 

func (o *Object) String() string

Return a string version

func (*Object) Update 

func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) error

Update the Object from in with modTime and size

type Options 

type Options struct {
	Provider              string               `config:"provider"`
	EnvAuth               bool                 `config:"env_auth"`
	Account               string               `config:"account"`    // Add account option
	UseOidcAgent          bool                 `config:"oidc_agent"` // Add oidc-agent option
	AccessKeyID           string               `config:"access_key_id"`
	SecretAccessKey       string               `config:"secret_access_key"`
	Region                string               `config:"region"`
	Endpoint              string               `config:"endpoint"`
	LocationConstraint    string               `config:"location_constraint"`
	ACL                   string               `config:"acl"`
	BucketACL             string               `config:"bucket_acl"`
	RequesterPays         bool                 `config:"requester_pays"`
	ServerSideEncryption  string               `config:"server_side_encryption"`
	SSEKMSKeyID           string               `config:"sse_kms_key_id"`
	SSECustomerAlgorithm  string               `config:"sse_customer_algorithm"`
	SSECustomerKey        string               `config:"sse_customer_key"`
	SSECustomerKeyMD5     string               `config:"sse_customer_key_md5"`
	StorageClass          string               `config:"storage_class"`
	UploadCutoff          fs.SizeSuffix        `config:"upload_cutoff"`
	CopyCutoff            fs.SizeSuffix        `config:"copy_cutoff"`
	ChunkSize             fs.SizeSuffix        `config:"chunk_size"`
	MaxUploadParts        int64                `config:"max_upload_parts"`
	DisableChecksum       bool                 `config:"disable_checksum"`
	SharedCredentialsFile string               `config:"shared_credentials_file"`
	Profile               string               `config:"profile"`
	SessionToken          string               `config:"session_token"`
	UploadConcurrency     int                  `config:"upload_concurrency"`
	ForcePathStyle        bool                 `config:"force_path_style"`
	V2Auth                bool                 `config:"v2_auth"`
	UseAccelerateEndpoint bool                 `config:"use_accelerate_endpoint"`
	LeavePartsOnError     bool                 `config:"leave_parts_on_error"`
	ListChunk             int64                `config:"list_chunk"`
	NoCheckBucket         bool                 `config:"no_check_bucket"`
	NoHead                bool                 `config:"no_head"`
	NoHeadObject          bool                 `config:"no_head_object"`
	Enc                   encoder.MultiEncoder `config:"encoding"`
	MemoryPoolFlushTime   fs.Duration          `config:"memory_pool_flush_time"`
	MemoryPoolUseMmap     bool                 `config:"memory_pool_use_mmap"`
	DisableHTTP2          bool                 `config:"disable_http2"`
}

Options defines the configuration for this backend

type SigningMethodHMAC added in v1.56.0 

type SigningMethodHMAC struct {
	Name string
	Hash crypto.Hash
}

SigningMethodHMAC - Implements the HMAC-SHA family of signing methods signing methods Expects key type of []byte for both signing and validation 

var (
	SigningMethodHS256 *SigningMethodHMAC
	SigningMethodHS384 *SigningMethodHMAC
	SigningMethodHS512 *SigningMethodHMAC
)

Specific instances for HS256, HS384, HS512

func ParseUnverifiedMapClaims added in v1.56.0 

func ParseUnverifiedMapClaims(tokenString string, claims *MapClaims, buf []byte) (*SigningMethodHMAC, error)

ParseUnverifiedMapClaims - WARNING: Don't use this method unless you know what you're doing

This method parses the token but doesn't validate the signature. It's only ever useful in cases where you know the signature is valid (because it has been checked previously in the stack) and you want to extract values from it.

func ParseUnverifiedStandardClaims added in v1.56.0 

func ParseUnverifiedStandardClaims(tokenString string, claims *StandardClaims, buf []byte) (*SigningMethodHMAC, error)

ParseUnverifiedStandardClaims - WARNING: Don't use this method unless you know what you're doing

This method parses the token but doesn't validate the signature. It's only ever useful in cases where you know the signature is valid (because it has been checked previously in the stack) and you want to extract values from it.

type StandardClaims added in v1.56.0 

type StandardClaims struct {
	AccessKey string `json:"accessKey,omitempty"`
	jwtgo.StandardClaims
}

StandardClaims are basically standard claims with "accessKey"

func NewStandardClaims added in v1.56.0 

func NewStandardClaims() *StandardClaims

NewStandardClaims - initializes standard claims

func (*StandardClaims) SetAccessKey added in v1.56.0 

func (c *StandardClaims) SetAccessKey(accessKey string)

SetAccessKey sets access key as jwt subject and custom "accessKey" field.

func (*StandardClaims) SetAudience added in v1.56.0 

func (c *StandardClaims) SetAudience(aud string)

SetAudience sets audience for these claims

func (*StandardClaims) SetExpiry added in v1.56.0 

func (c *StandardClaims) SetExpiry(t time.Time)

SetExpiry sets expiry in unix epoch secs

func (*StandardClaims) SetIssuer added in v1.56.0 

func (c *StandardClaims) SetIssuer(issuer string)

SetIssuer sets issuer for these claims

func (*StandardClaims) Valid added in v1.56.0 

func (c *StandardClaims) Valid() error

Valid - implements https://godoc.org/github.com/golang-jwt/jwt#Claims compatible claims interface, additionally validates "accessKey" fields.

type WebIdentityResult 

type WebIdentityResult struct {
	AssumedRoleUser AssumedRoleUser `xml:",omitempty"`
	Audience        string          `xml:",omitempty"`
	// Ref: https://github.com/minio/minio/blob/master/internal/auth/credentials.go#L96
	Credentials                 Credentials `xml:",omitempty"`
	PackedPolicySize            int         `xml:",omitempty"`
	Provider                    string      `xml:",omitempty"`
	SubjectFromWebIdentityToken string      `xml:",omitempty"`
}

WebIdentityResult - Contains the response to a successful AssumeRoleWithWebIdentity request, including temporary credentials that can be used to make MinIO API requests.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.