Documentation
¶
Overview ¶
Package ots3 provides a s3 uploader with opentracing capabilities.
Introduction ¶
S3 is the de facto standard for cloud file systems.The transport of S3 is HTTP(s) which is pleasantly simple to trace. This package also features a go kit server and client for the upload service.
Simple Usage ¶
Creating a s3 manager:
var manager = NewManager(accessKey, accessSecret, endpoint, region, bucket) url, err := manager.Upload(context.Background(), "myfile", file)
Integration ¶
Package ots3 exports the following configuration:
s3:
default:
accessKey:
accessSecret:
bucket:
endpoint:
region:
cdnUrl:
To use package ots3 with package core:
var c *core.C = core.New()
c.Provide(ots3.Providers())
c.AddModuleFunc(ots3.New)
c.Invoke(function(manager *ots3.Manager) {
// do something with manager
})
Adding the module created by mods3.New is optional. This module provides an "/upload" path for the http router. If this is not relevant, just leave it out.
Sometimes there are valid reasons to connect to more than one s3 server. Inject mods3.Maker to factory a *ots3.Manager with a specific configuration entry.
c.Invoke(function(maker ots3.Maker) {
manager, err := maker.Make("default")
})
Future scope ¶
Currently this package only focus on the file upload aspect of s3. Other s3 features can be incrementally implemented.
Example ¶
file, err := os.Open("./testdata/basn0g01-30.png")
if err != nil {
panic(err)
}
defer file.Close()
uploader := NewManager(envDefaultS3AccessKey, envDefaultS3AccessSecret, envDefaultS3Endpoint, envDefaultS3Region, envDefaultS3Bucket)
_ = uploader.CreateBucket(context.Background(), envDefaultS3Bucket)
url, _ := uploader.Upload(context.Background(), "foo", file)
url = strings.Replace(url, envDefaultS3Endpoint, "http://example.org", 1)
fmt.Println(url)
Output: http://example.org/mybucket/foo.png
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Providers ¶ added in v0.2.0
func Providers() []interface{}
Providers returns a set of dependencies providers related to S3. It includes the s3 Manager, the Maker and exported configurations.
Depends On: log.Logger contract.ConfigAccessor opentracing.Tracer `optional:"true"` Provide: Factory Maker *Manager Uploader
Types ¶
type Config ¶
type Config struct {
// contains filtered or unexported fields
}
Config contains a various of configurations for Manager. It is mean to be modified by Option.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager manages S3 uploads.
func NewManager ¶
NewManager creates a new S3 manager
func (*Manager) CreateBucket ¶
CreateBucket create a buckets in s3 server. TODO: handle acl
func (*Manager) Upload ¶
func (m *Manager) Upload(ctx context.Context, name string, reader io.Reader) (newUrl string, err error)
Upload uploads an io.reader to the S3 server, and returns the url on S3. The extension of the uploaded file is auto detected.
func (*Manager) UploadFromUrl ¶
UploadFromUrl fetches a file from an external url, copy them to the S3 server, and generate a new, local url. It uses streams to relay files (instead of buffering the entire file in memory). it gives the file a random name using the global seed.
type Option ¶
type Option func(*Config)
Option is the type of functional options to alter Config.
func WithHttpClient ¶
WithHttpClient is an option that replaces the default http client. Useful for interceptors like tracing and metrics.
func WithLocationFunc ¶
WithLocationFunc is an option that decides the how url is mapped to S3 bucket and path. Useful when not serving file directly from S3, but from a CDN.
func WithPathPrefix ¶
WithPathPrefix is an option that changes the path prefix of uploaded file.
func WithTracer ¶
func WithTracer(tracer opentracing.Tracer) Option
WithTracer is an option that add opentracing.Tracer via the hook of S3 client.
type S3Config ¶
type S3Config struct {
AccessKey string `json:"accessKey" yaml:"accessKey"`
AccessSecret string `json:"accessSecret" yaml:"accessSecret"`
Endpoint string `json:"endpoint" yaml:"endpoint"`
Region string `json:"region" yaml:"region"`
Bucket string `json:"bucket" yaml:"bucket"`
CdnUrl string `json:"cdnUrl" yaml:"cdnUrl"`
}
S3Config contains credentials of S3 server
Source Files