stream

Documentation

Overview

Package stream contains chainable functions acting on sizeable data streams such as cores. It can write a local copy, compress, and/or upload to s3.

Index

• func Compress(opts *opts.Opts, uncompressed string, nextFn NextFn) (err error)
• func LocalCopy(cfg *opts.Opts, localpath string, buf io.Reader) error
• func Write(cfg *opts.Opts, suffix string, stream io.Reader) (err error)
• type NextFn

Functions

func Compress

func Compress(opts *opts.Opts, uncompressed string, nextFn NextFn) (err error)

compress, pass output to 'nextFn' (typically Upload)

func LocalCopy

func LocalCopy(cfg *opts.Opts, localpath string, buf io.Reader) error

try to write buf to a local file, then upload in the unlikely event file creation fails, go straight from the buffer

func Write

func Write(cfg *opts.Opts, suffix string, stream io.Reader) (err error)

Types

type NextFn

type NextFn func(cfg *opts.Opts, name string, stream io.Reader) error