tus

README

File upload using the tus protocol.

This example implements a tus compliant file upload mechanism. tus is a protocol built on top of HTTP that allows for uploads to be resumed. There are a number of client libraries that support tus written in different languages such as go-tus or tus-js-client. This example illustrates how to write a Goa service that can be used in conjunction with such clients to implement resumable uploads.

Methods

The tus core protocol requires the server to implement a number of HTTP methods:

• HEAD requests should return status information on on-going uploads.
• PATCH requests are used to do the actual upload.
• OPTIONS requests allow for discovery (e.g. tus protocol version, supported extensions)

The protocol also suggests implementing additional HTTP methods:

• POST to create the upload target (creation and creation-with-upload extensions)
• DELETE to cancel uploads (termination extension).

Extensions

This example implements the tus core protocol as well as the following extensions:

• creation
• creation-with-upload
• creation-defer-length
• expiration
• checksum
• termination

This example supports version 1.0.0 of the tus protocol.

Usage

This example implementation can be adapted for "real production" scenarios by doing two modifications:

1. Modifying the function that creates the writer used to write incoming bytes. The default implementation writes to local disk.
2. Providing an implementation of the metadata store interface that persists the data instead of simply using an in-memory map. This data is what makes is possible to resume uploads and thus must be shared by all application servers that participate to the production cluster.

Limitations

This implementation does not support the Concatenation extension. There is no fundamental reason why it couldn't (other than time!) and would make a great addition!

Example

Build and start the upload server:

cd $GOPATH/src/goa.design/examples/tus/cmd/upload
go build; and ./upload
[tus] 17:19:14 HTTP "Head" mounted on HEAD /upload/{id}
[tus] 17:19:14 HTTP "Patch" mounted on PATCH /upload/{id}
[tus] 17:19:14 HTTP "Options" mounted on OPTIONS /upload
[tus] 17:19:14 HTTP "Post" mounted on POST /upload
[tus] 17:19:14 HTTP "Delete" mounted on DELETE /upload/{id}
[tus] 17:19:14 HTTP server listening on "localhost:8080"

Build the client and show usage:

cd $GOPATH/src/goa.design/examples/tus/cmd/upload-cli
go build
./upload-cli --help
./upload-cli is a command line client for the tus upload API.

Usage:
    ./upload-cli [-host HOST][-url URL][-timeout SECONDS][-verbose|-v] SERVICE ENDPOINT [flags]

    -host HOST:  server host (development). valid values: development
    -url URL:    specify service URL overriding host URL (http://localhost:8080)
    -timeout:    maximum number of seconds to wait for response (30)
    -verbose|-v: print request and response details (false)

Commands:
    tus (head|patch|options|post|delete)

Additional help:
    ./upload-cli SERVICE [ENDPOINT] --help

Example:
    ./upload-cli tus head --id "6m6dfuts16k6ac7a5gna" --tus-resumable "1.0.0"

Perform an upload of the file images/goa.png:

./upload-cli --url http://localhost:8080 tus post --tus-resumable "1.0.0" --stream ../../images/goa.png --upload-defer-length 1

note: here we are uploading the file in a single chunk. If the image was big we would cut it in chunks and upload each one individually using a series of POST requests and specifying the offset each time.

Retrieve the status of the upload (replace bqjnf2cl42v556c7du4g with the value you got above):

./upload-cli --url http://localhost:8080 tus head --tus-resumable "1.0.0" --id bqjnf2cl42v556c7du4g

Complete the upload (again replacing the id with the proper value):

./upload-cli --url http://localhost:8080 tus delete --tus-resumable "1.0.0" --id bqjnf2cl42v556c7du4g

Documentation

Index

• Constants
• func New(store persist.Store, newWriter func(string, *int64) (io.WriteCloser, error), ...) tus.Service
• func Write(r io.ReadCloser, w io.Writer, u *persist.Upload, offset int64, ...) (n int64, err error)

Constants

const TusResumable = "1.0.0"

TusResumable is the version of tus implemented by this service.

Functions

func New

func New(store persist.Store, newWriter func(string, *int64) (io.WriteCloser, error), maxSize int64, uploadTimeout time.Duration, logger *log.Logger) tus.Service

New creates a TUS service that accepts uploads up to maxSize bytes and that expire after uploadTimeout. If maxSize is 0 then there is no limit on the size of uploads. If uploadTimeout is 0 then uploads never expire. newWriter is called for each upload. The unique upload identifier is given as argument to the function as well as the length of the upload if known (nil otherwise). The function returns the writer used to write the uploaded bytes. It must be possible to call Write concurrently on two writers returned by different invocation of the function. The given persist store is used to maintain state of ongoing uploads, it should persist the state so that uploads can be resumed across restarts.

func Write

func Write(r io.ReadCloser, w io.Writer, u *persist.Upload, offset int64, checksum *string) (n int64, err error)

Write validates the checksum if not nil then writes the data to the given writer, updates the given upload state and returns the updated upload offset value. checksum must start with "sha1 ", "md5 " or "crc32 " and be followed by the Base64 encoded checksum. It is an error to write to an upload whose state is not Started.