gos3

README

gos3 : Simple no frills AWS S3 Library using REST with V4 Signing

Overview

GoS3 is a Go library for manipulating objects in S3 buckets using REST API calls or Presigned URLs signed using AWS Signature Version 4.

Install

go get github.com/animber-coder/gos3

Example

testTxt, _ := os.Open("testdata/test.txt")
defer testTxt.Close()

// Create an instance of the package
// You can either create by manually supplying credentials
// (preferably using Environment vars)
s3 := gos3.New(Region, AWSAccessKey, AWSSecretKey)
// or you can use this on an EC2 instance to 
// obtain credentials from IAM attached to the instance.
s3, _ := gos3.NewUsingIAM(Region)

// You can also set a custom endpoint to a compatible s3 instance. 
s3.SetEndpoint(CustomEndpoint)

// Note: Consider adding a testTxt.Seek(0, 0)
// in case you have read 
// the body, as the pointer is shared by the library.

// File Upload is as simple as providing the following
// details.
resp, err := s3.FileUpload(gos3.UploadInput{
    Bucket:      AWSBucket,
    ObjectKey:   "test.txt",
    ContentType: "text/plain",
    FileName:    "test.txt",
    Body:        testTxt,
})

// Similarly, Files can be deleted.
err := s3.FileDelete(gos3.DeleteInput{
    Bucket:    os.Getenv("AWS_S3_BUCKET"),
    ObjectKey: "test.txt",
})

// You can also download the file.
file, _ := s3.FileDownload(gos3.DownloadInput{
    Bucket:    AWSBucket,
    ObjectKey: "test.txt",
})

data, _ := ioutil.ReadAll(file)
file.Close()

// You can also use this library to generate
// Presigned URLs that can for eg. be used to
// GET/PUT files on S3 through the browser.
var time, _ = time.Parse(time.RFC1123, "Fri, 24 May 2013 00:00:00 GMT")

url := s.GeneratePresignedURL(PresignedInput{
    Bucket:        AWSBucket,
    ObjectKey:     "test.txt",
    Method:        "GET",
    Timestamp:     time,
    ExpirySeconds: 86400,
})

Contributing

You are more than welcome to contribute to this project. Fork and make a Pull Request, or create an Issue if you see any problem or want to propose a feature.

Author

Rohan Verma hello@rohanverma.net

License

BSD-2-Clause-FreeBSD

Documentation

Index

• Constants
• type DeleteInput
• type DetailsInput
• type DetailsResponse
• type DownloadInput
• type IAMResponse
• type PolicyJSON
• type PresignedInput
• type PutResponse
• type S3
  • func New(region, accessKey, secretKey string) *S3
  • func NewUsingIAM(region string) (*S3, error)
  • func (s3 *S3) CreateUploadPolicies(uploadConfig UploadConfig) (UploadPolicies, error)
  • func (s3 *S3) FileDelete(u DeleteInput) error
  • func (s3 *S3) FileDetails(u DetailsInput) (DetailsResponse, error)
  • func (s3 *S3) FileDownload(u DownloadInput) (io.ReadCloser, error)
  • func (s3 *S3) FilePut(u UploadInput) (PutResponse, error)
  • func (s3 *S3) FileUpload(u UploadInput) (UploadResponse, error)
  • func (s3 *S3) GeneratePresignedURL(in PresignedInput) string
  • func (s3 *S3) SetClient(client *http.Client) *S3
  • func (s3 *S3) SetEndpoint(uri string) *S3
  • func (s3 *S3) SetIAMData(iamResp IAMResponse)
  • func (s3 *S3) SetToken(token string) *S3
• type UploadConfig
• type UploadInput
• type UploadPolicies
• type UploadResponse

Constants

const (

	// AMZMetaPrefix to prefix metadata key.
	AMZMetaPrefix = "x-amz-meta-"
)

Types

type DeleteInput

type DeleteInput struct {
	Bucket    string
	ObjectKey string
}

DeleteInput is passed to FileDelete as a parameter.

type DetailsInput

type DetailsInput struct {
	Bucket    string
	ObjectKey string
}

DetailsInput is passed to FileDetails as a parameter.

type DetailsResponse

type DetailsResponse struct {
	ContentType   string
	ContentLength string
	AcceptRanges  string
	Date          string
	Etag          string
	LastModified  string
	Server        string
	AmzID2        string
	AmzRequestID  string
	AmzMeta       map[string]string
	ExtraHeaders  map[string]string
}

DetailsResponse is returned by FileDetails.

type DownloadInput

type DownloadInput struct {
	Bucket    string
	ObjectKey string
}

DownloadInput is passed to FileDownload as a parameter.

type IAMResponse

type IAMResponse struct {
	Code            string `json:"Code"`
	LastUpdated     string `json:"LastUpdated"`
	Type            string `json:"Type"`
	AccessKeyID     string `json:"AccessKeyId"`
	SecretAccessKey string `json:"SecretAccessKey"`
	Token           string `json:"Token"`
	Expiration      string `json:"Expiration"`
}

IAMResponse is used by NewUsingIAM to auto detect the credentials.

type PolicyJSON

type PolicyJSON struct {
	Expiration string        `json:"expiration"`
	Conditions []interface{} `json:"conditions"`
}

PolicyJSON is policy rule.

type PresignedInput

type PresignedInput struct {
	Bucket        string
	ObjectKey     string
	Method        string
	Timestamp     time.Time
	ExtraHeaders  map[string]string
	ExpirySeconds int
	Protocol      string
	Endpoint      string
}

PresignedInput is passed to GeneratePresignedURL as a parameter.

type PutResponse

type PutResponse struct {
	ETag    string
	Headers http.Header
}

PutResponse is returned when the action is successful, and the service sends back an HTTP 200 response. The response returns ETag along with HTTP headers.

type S3

type S3 struct {
	AccessKey string
	SecretKey string
	Region    string
	Client    *http.Client

	Token     string
	Endpoint  string
	URIFormat string
}

S3 provides a wrapper around your S3 credentials.

func New

func New(region, accessKey, secretKey string) *S3

New returns an instance of S3.

func NewUsingIAM

func NewUsingIAM(region string) (*S3, error)

NewUsingIAM automatically generates an Instance of S3 using instance metatdata.

func (*S3) CreateUploadPolicies

func (s3 *S3) CreateUploadPolicies(uploadConfig UploadConfig) (UploadPolicies, error)

CreateUploadPolicies creates amazon s3 sigv4 compatible policy and signing keys with the signature returns the upload policy. https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/API/sigv4-authentication-HTTPPOST.html

func (*S3) FileDelete

func (s3 *S3) FileDelete(u DeleteInput) error

FileDelete makes a DELETE call with the file written as multipart and on successful upload, checks for 204 No Content.

func (*S3) FileDetails

func (s3 *S3) FileDetails(u DetailsInput) (DetailsResponse, error)

func (*S3) FileDownload

func (s3 *S3) FileDownload(u DownloadInput) (io.ReadCloser, error)

FileDownload makes a GET call and returns a io.ReadCloser. After reading the response body, ensure closing the response.

func (*S3) FilePut

func (s3 *S3) FilePut(u UploadInput) (PutResponse, error)

FilePut makes a PUT call to S3.

func (*S3) FileUpload

func (s3 *S3) FileUpload(u UploadInput) (UploadResponse, error)

FileUpload makes a POST call with the file written as multipart and on successful upload, checks for 200 OK.

func (*S3) GeneratePresignedURL

func (s3 *S3) GeneratePresignedURL(in PresignedInput) string

GeneratePresignedURL creates a Presigned URL that can be used for Authentication using Query Parameters. (https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html)

func (*S3) SetClient

func (s3 *S3) SetClient(client *http.Client) *S3

SetClient can be used to set the http client to be used by the package. If client passed is nil, http.DefaultClient is used.

func (*S3) SetEndpoint

func (s3 *S3) SetEndpoint(uri string) *S3

SetEndpoint can be used to the set a custom endpoint for using an alternate instance compatible with the s3 API. If no protocol is included in the URI, defaults to HTTPS.

func (*S3) SetIAMData

func (s3 *S3) SetIAMData(iamResp IAMResponse)

setIAMData sets the IAM data on the S3 instance.

func (*S3) SetToken

func (s3 *S3) SetToken(token string) *S3

SetToken can be used to set a Temporary Security Credential token obtained from using an IAM role or AWS STS.

type UploadConfig

type UploadConfig struct {
	// Required
	BucketName         string
	ObjectKey          string
	ContentType        string
	ContentDisposition string
	ACL                string
	FileSize           int64
	// Optional
	UploadURL  string
	Expiration time.Duration
	MetaData   map[string]string
}

UploadConfig generate policies from config for POST requests to S3 using Signing V4.

type UploadInput

type UploadInput struct {
	// essential fields
	Bucket      string
	ObjectKey   string
	FileName    string
	ContentType string

	// optional fields
	ContentDisposition string
	ACL                string
	// Setting key/value pairs adds user-defined metadata
	// keys to the object, prefixed with AMZMetaPrefix.
	CustomMetadata map[string]string

	Body io.ReadSeeker
}

UploadInput is passed to FileUpload as a parameter.

type UploadPolicies

type UploadPolicies struct {
	URL  string
	Form map[string]string
}

UploadPolicies Amazon s3 upload policies.

type UploadResponse

type UploadResponse struct {
	Location string `xml:"Location"`
	Bucket   string `xml:"Bucket"`
	Key      string `xml:"Key"`
	ETag     string `xml:"ETag"`
}

UploadResponse receives the following XML in case of success, since we set a 201 response from S3. Sample response:

<PostResponse>
  <Location>https://s3.amazonaws.com/link-to-the-file</Location>
  <Bucket>s3-bucket</Bucket>
  <Key>development/8614bd40-691b-4668-9241-3b342c6cf429/image.jpg</Key>
  <ETag>"32-bit-tag"</ETag>
</PostResponse>