buffered

README

Buffered

Package buffered provides buffered I/O using a buffer file to copy from a single source to multiple consumers/sinks. The buffer file enables decoupling the source and the sinks so that slow consumers don't affect reading from the source nor writing to faster consumers.

                                                                         +---------------------------------+
 +---------------------------------+                              +----->|             Sink 1              |
 |             Source              |-----+                        |      +---------------------------------+
 +---------------------------------+     |                        |
                                         |                        |      +-------------+
                                         |                        +----->|   Sink 2    |
                                         |      +-----------+     |      +-------------+
                                         |      |           |     |
                                         +----->|Buffer file|-----+
                                                |           |
                                                +-----------+


                                      buffered.Copy(src, sink1, sink2)

Documentation

Overview

Package buffered provides buffered I/O using a buffer file to copy from a single source to multiple consumers. The buffer file enables decoupling the source and the consumers (sinks) so that slow consumers don't affect reading from the source nor writing to faster consumers.

Index

• type Writer
  • func (w *Writer) Copy(src io.ReadCloser, consumer ...io.WriteCloser) (<-chan struct{}, <-chan error)

Types

type Writer

type Writer struct {

	// The directory to use for the temporary buffer files.
	// If Tmpdir is empty, Writer uses the default directory for temporary files (see os.TempDir)
	Tmpdir string

	// The size of the in memory buffer to use when reading from the source. Default: 32 KiB
	ReadBufferSize int
}

Writer can be used to read from a single source and write to multiple consumers. Writer uses a temporary buffer file to decouple the throughput of the source from the throughput of the sink(s).

func (*Writer) Copy

func (w *Writer) Copy(src io.ReadCloser, consumer ...io.WriteCloser) (<-chan struct{}, <-chan error)

Copy reads from the given src and writes to all the consumers using a temporary buffer file. Copy closes the src after reading from it successfully. Copy also closes all consumers as soon as the content has been successfully written. Copy returns a pair of channels. The first channel is the "done" channel and will receive a value after the last consumer finished. The second channel receives all errors that occured during the copy operation.