Documentation
¶
Overview ¶
Package ots3 provides a s3 uploader with opentracing capabilities.
Introduction ¶
S3 is the de facto standard for cloud file systems.The transport of S3 is HTTP(s) which is pleasantly simple to trace. This package also features a go kit server and client for the upload service.
Simple Usage ¶
Creating a s3 manager:
var manager = NewManager(accessKey, accessSecret, endpoint, region, bucket) url, err := manager.Upload(context.Background(), "myfile", file)
Integration ¶
Package ots3 exports the following configuration:
s3:
default:
accessKey:
accessSecret:
bucket:
endpoint:
region:
cdnUrl:
To use package ots3 with package core:
var c *core.C = core.New()
c.Provide(mods3.Provide)
c.AddModuleFunc(mods3.New)
c.Invoke(function(manager *ots3.Manager) {
// do something with manager
})
Adding the module created by mods3.New is optional. This module provides an "/upload" path for the http router. If this is not relevant, just leave it out.
Sometimes there are valid reasons to connect to more than one s3 server. Inject mods3.S3Maker to factory a *ots3.Manager with a specific configuration entry.
c.Invoke(function(maker mods3.S3Maker) {
manager, err := maker.Make("default")
})
Future scope ¶
Currently this package only focus on the file upload aspect of s3. Other s3 features can be incrementally implemented.
Example ¶
package main
import (
"context"
"fmt"
"os"
)
func createBucket() {
}
func main() {
file, err := os.Open("./testdata/basn0g01-30.png")
if err != nil {
panic(err)
}
defer file.Close()
uploader := NewManager("Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG", "https://play.minio.io:9000", "asia", "mybucket")
_ = uploader.CreateBucket(context.Background(), "mybucket")
url, _ := uploader.Upload(context.Background(), "foo", file)
fmt.Println(url)
}
Output: https://play.minio.io:9000/mybucket/foo.png
Index ¶
- func MakeHttpHandler(endpoint endpoint.Endpoint, middleware endpoint.Middleware) http.Handler
- func MakeUploadEndpoint(uploader Uploader) endpoint.Endpoint
- func Middleware(logger log.Logger, env contract.Env) endpoint.Middleware
- func NewClient(uri *url.URL) *httptransport.Client
- type ClientUploader
- type Config
- type Manager
- type Module
- type Option
- type Request
- type Response
- type S3Config
- type S3Factory
- type S3In
- type S3Maker
- type S3Out
- type UploadService
- type Uploader
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MakeHttpHandler ¶
MakeHttpHandler creates a go kit transport in http for *UploadService.
func MakeUploadEndpoint ¶
MakeUploadEndpoint creates a Upload endpoint
func Middleware ¶
Middleware adds logging and error handling to the endpoint.
Types ¶
type ClientUploader ¶
type ClientUploader struct {
// contains filtered or unexported fields
}
ClientUploader implements the Uploader interface. It uploads files to the remote server.
func NewClientUploader ¶
func NewClientUploader(client *httptransport.Client) *ClientUploader
NewClientUploader creates a *ClientUploader
func NewClientUploaderFromUrl ¶
func NewClientUploaderFromUrl(uri *url.URL) *ClientUploader
NewClientUploaderFromUrl creates a *ClientUploader from the url of the remote *UploadService
type Config ¶
type Config struct {
// contains filtered or unexported fields
}
Config contains a various of configurations for Manager. It is mean to be modified by Option.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager manages S3 uploads.
func NewManager ¶
NewManager creates a new S3 manager
func (*Manager) CreateBucket ¶
CreateBucket create a buckets in s3 server. TODO: handle acl
func (*Manager) Upload ¶
func (m *Manager) Upload(ctx context.Context, name string, reader io.Reader) (newUrl string, err error)
Upload uploads an io.reader to the S3 server, and returns the url on S3. The extension of the uploaded file is auto detected.
func (*Manager) UploadFromUrl ¶
UploadFromUrl fetches a file from an external url, copy them to the S3 server, and generate a new, local url. It uses streams to relay files (instead of buffering the entire file in memory). it gives the file a random name using the global seed.
type Module ¶
type Module struct {
// contains filtered or unexported fields
}
Module is a s3 module that adds a file upload path. It uses the default configuration.
func (Module) ProvideHttp ¶
ProvideHttp adds a "/upload" path to router
type Option ¶
type Option func(*Config)
Option is the type of functional options to alter Config.
func WithHttpClient ¶
WithHttpClient is an option that replaces the default http client. Useful for interceptors like tracing and metrics.
func WithLocationFunc ¶
WithLocationFunc is an option that decides the how url is mapped to S3 bucket and path. Useful when not serving file directly from S3, but from a CDN.
func WithPathPrefix ¶
WithPathPrefix is an option that changes the path prefix of uploaded file.
func WithTracer ¶
func WithTracer(tracer opentracing.Tracer) Option
WithTracer is an option that add opentracing.Tracer via the hook of S3 client.
type Request ¶
type Request struct {
// contains filtered or unexported fields
}
Request models a go kit request in UploadEndpoint
type Response ¶
type Response struct {
Data struct {
Url string `json:"url"`
} `json:"data"`
Code int `json:"code"`
Message string `json:"message,omitempty"`
}
Response models a go kit Response in UploadEndpoint
type S3Config ¶
type S3Config struct {
AccessKey string `json:"accessKey" yaml:"accessKey"`
AccessSecret string `json:"accessSecret" yaml:"accessSecret"`
Endpoint string `json:"endpoint" yaml:"endpoint"`
Region string `json:"region" yaml:"region"`
Bucket string `json:"bucket" yaml:"bucket"`
CdnUrl string `json:"cdnUrl" yaml:"cdnUrl"`
}
S3Config contains credentials of S3 server
type S3In ¶
type S3In struct {
di.In
Logger log.Logger
Conf contract.ConfigAccessor
Tracer opentracing.Tracer `optional:"true"`
}
S3In is the injection parameter for Provide.
type S3Out ¶
type S3Out struct {
di.Out
Manager *Manager
Factory *S3Factory
Maker S3Maker
Uploader Uploader
ExportedConfig []config.ExportedConfig `group:"config,flatten"`
}
S3Out is the di output of Provide.
type UploadService ¶
An UploadService is a go kit service to handle file upload
Source Files