s3util

Documentation

Overview

Package s3util contains methods for interacting with S3.

Package s3util contains utilities for working with the file system.

Package s3util contains methods for interacting with S3.

Package s3util contains methods for interacting with S3.

Index

• Constants
• func GetBucketRegion(log log.T, bucketName string, httpProvider HttpProvider) (region string)
• func GetS3Endpoint(region string) (s3Endpoint string)
• func GetS3GenericEndPoint(region string) (s3Endpoint string)
• func GetS3Header(log log.T, bucketName string, instanceRegion string, httpProvider HttpProvider) string
• type AmazonS3URL
  • func ParseAmazonS3URL(log log.T, s3URL *url.URL) (output AmazonS3URL)
  • func (output AmazonS3URL) IsBucketAndKeyPresent() bool
  • func (output AmazonS3URL) String() string
• type AmazonS3Util
  • func NewAmazonS3Util(log log.T, bucketName string) *AmazonS3Util
  • func (u *AmazonS3Util) S3Upload(log log.T, bucketName string, objectKey string, filePath string) (err error)
• type HttpProvider
• type HttpProviderImpl
  • func (HttpProviderImpl) Head(url string) (*http.Response, error)
• type IAmazonS3Util
• type MockS3Uploader
  • func (uploader *MockS3Uploader) GetS3BucketRegionFromErrorMsg(log log.T, errMsg string) string
  • func (uploader *MockS3Uploader) GetS3ClientRegion() string
  • func (uploader *MockS3Uploader) IsS3ErrorRelatedToAccessDenied(errMsg string) bool
  • func (uploader *MockS3Uploader) IsS3ErrorRelatedToWrongBucketRegion(errMsg string) bool
  • func (uploader *MockS3Uploader) S3Upload(log log.T, bucketName string, bucketKey string, contentPath string) error
  • func (uploader *MockS3Uploader) SetS3ClientRegion(region string)
  • func (uploader *MockS3Uploader) UploadS3TestFile(log log.T, bucketName, key string) error

Constants

const (
	// EndpointPattern is a valid regular expression for s3 url pattern
	EndpointPattern = "^(.+\\.)?s3[.-]([a-z0-9-]+)\\."

	// cn- is a prefix for China region
	ChinaRegionPrefix = "cn-"
)

Functions

func GetBucketRegion

func GetBucketRegion(log log.T, bucketName string, httpProvider HttpProvider) (region string)

This function returns the Amazon S3 Bucket region based on its name and the EC2 instance region. It will return the same instance region if it failed to guess the bucket region.

func GetS3Endpoint

func GetS3Endpoint(region string) (s3Endpoint string)

This function returns the s3 endpoint specified by the user in appconfig.

If the user didn't specify one, it will return the Amazon S3 endpoint in a certain region

func GetS3GenericEndPoint

func GetS3GenericEndPoint(region string) (s3Endpoint string)

This function will get the generic S3 endpoint for a certain region. Most regions will use us-east-1 endpoint except special ones

func GetS3Header

func GetS3Header(log log.T, bucketName string, instanceRegion string, httpProvider HttpProvider) string

This function return the S3 bucket region that is returned as a result of a CURL operation on an S3 path. The starting endpoint does not need to be the same as the returned region. For example, we might query an endpoint in us-east-1 and get a return of us-west-2, if the bucket is actually in us-west-2.

Types

type AmazonS3URL

type AmazonS3URL struct {
	IsValidS3URI bool
	IsPathStyle  bool
	Bucket       string
	Key          string
	Region       string
}

AmazonS3URL holds interesting pieces after parsing a s3 URL

func ParseAmazonS3URL

func ParseAmazonS3URL(log log.T, s3URL *url.URL) (output AmazonS3URL)

ParseAmazonS3URL parses a URL and returns AmazonS3URL object

func (AmazonS3URL) IsBucketAndKeyPresent

func (output AmazonS3URL) IsBucketAndKeyPresent() bool

IsBucketAndKeyPresent checks the AmazonS3URL if it contains both bucket and key

func (AmazonS3URL) String

func (output AmazonS3URL) String() string

String returns the string representation of the AmazonS3URL

type AmazonS3Util

type AmazonS3Util struct {
	// contains filtered or unexported fields
}

func NewAmazonS3Util

func NewAmazonS3Util(log log.T, bucketName string) *AmazonS3Util

func (*AmazonS3Util) S3Upload

func (u *AmazonS3Util) S3Upload(log log.T, bucketName string, objectKey string, filePath string) (err error)

S3Upload uploads a file to s3.

type HttpProvider

type HttpProvider interface {
	Head(url string) (*http.Response, error)
}

type HttpProviderImpl

type HttpProviderImpl struct{}

HttpProviderImpl provides http capabilities

func (HttpProviderImpl) Head

func (HttpProviderImpl) Head(url string) (*http.Response, error)

type IAmazonS3Util

type IAmazonS3Util interface {
	S3Upload(log log.T, bucketName string, objectKey string, filePath string) error
	GetBucketRegion(log log.T, bucketName string) string
	GetS3Header(log log.T, bucketName string, instanceRegion string) string
}

type MockS3Uploader

type MockS3Uploader struct {
	mock.Mock
}

MockS3Uploader mocks an s3 uploader.

func (*MockS3Uploader) GetS3BucketRegionFromErrorMsg

func (uploader *MockS3Uploader) GetS3BucketRegionFromErrorMsg(log log.T, errMsg string) string

GetS3BucketRegionFromErrorMsg mocks the method with the same name.

func (*MockS3Uploader) GetS3ClientRegion

func (uploader *MockS3Uploader) GetS3ClientRegion() string

GetS3ClientRegion mocks the method with the same name.

func (*MockS3Uploader) IsS3ErrorRelatedToAccessDenied

func (uploader *MockS3Uploader) IsS3ErrorRelatedToAccessDenied(errMsg string) bool

IsS3ErrorRelatedToAccessDenied mocks the method with the same name.

func (*MockS3Uploader) IsS3ErrorRelatedToWrongBucketRegion

func (uploader *MockS3Uploader) IsS3ErrorRelatedToWrongBucketRegion(errMsg string) bool

IsS3ErrorRelatedToWrongBucketRegion mocks the method with the same name.

func (*MockS3Uploader) S3Upload

func (uploader *MockS3Uploader) S3Upload(log log.T, bucketName string, bucketKey string, contentPath string) error

S3Upload mocks the method with the same name.

func (*MockS3Uploader) SetS3ClientRegion

func (uploader *MockS3Uploader) SetS3ClientRegion(region string)

SetS3ClientRegion mocks the method with the same name.

func (*MockS3Uploader) UploadS3TestFile

func (uploader *MockS3Uploader) UploadS3TestFile(log log.T, bucketName, key string) error

UploadS3TestFile mocks the method with the same name.