parallelwriter

Documentation

Index

• Variables
• type BufferedWriter
  • func NewBufferedWriter(w WriteFlusher, bufferSize int) *BufferedWriter
  • func (b *BufferedWriter) Close() error
  • func (b *BufferedWriter) Flush() error
  • func (b *BufferedWriter) Write(buf []byte) (int, error)
• type PipelinedFlushWriter
  • func NewPipelinedFlushWriter(wf WriteFlusher) *PipelinedFlushWriter
  • func (w *PipelinedFlushWriter) Write(buf []byte) (int, error)
• type WriteFlusher
  • func NopFlusher(w io.Writer) WriteFlusher
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w *Writer) Write(b []byte) (int, error)

Variables

var (
	ErrClosed = errors.New("buffered_writer: closed")
)

Types

type BufferedWriter

type BufferedWriter struct {
	// contains filtered or unexported fields
}

func NewBufferedWriter

func NewBufferedWriter(w WriteFlusher, bufferSize int) *BufferedWriter

func (*BufferedWriter) Close

func (b *BufferedWriter) Close() error

Close closes the writer and prevents any new writes from succeeding. This does not close the underlying writer which must be closed to ensure any pending writes are canceled. It must be safe to close the underlying writer from a different goroutine than an active Write.

Close does not block.

func (*BufferedWriter) Flush

func (b *BufferedWriter) Flush() error

func (*BufferedWriter) Write

func (b *BufferedWriter) Write(buf []byte) (int, error)

type PipelinedFlushWriter

type PipelinedFlushWriter struct {
	// contains filtered or unexported fields
}

PipelinedFlushWriter is a writer that serialises and batches parallel writes, similar to Writer, but with two additional properties:

1. Flush is performed after every Write.
2. Flush and Write are pipelined, so that a Flush may occur in parallel with a Write, but Writes are serialised, and Flushes are serialised.

This behaviour is particularly useful for write-ahead logs, where the caller wants to ensure writes are flushed to disk, but also wants to do many writes in parallel for high total throughput.

func NewPipelinedFlushWriter

func NewPipelinedFlushWriter(wf WriteFlusher) *PipelinedFlushWriter

func (*PipelinedFlushWriter) Write

func (w *PipelinedFlushWriter) Write(buf []byte) (int, error)

Write serialises, batches, and flushes writes to the underlying Writer. Parallel writes are atomic, and buffers are never interleaved or broken to the underlying Writer.

type WriteFlusher

type WriteFlusher interface {
	io.Writer
	Flush() error
}

func NopFlusher

func NopFlusher(w io.Writer) WriteFlusher

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer serialises and batches concurrent writes, to minimise Write calls to the underlying io.Writer. This is useful when the underlying io.Writer has properties that make each Write expensive, such as sending across a network and waiting for an ack, or flushing after every write. If your io.Writer does not have this property, you should instead synchronize using a sync.Mutex.

func NewWriter

func NewWriter(w io.Writer) *Writer

func (*Writer) Write

func (w *Writer) Write(b []byte) (int, error)

Write serialises and batches writes to the underlying Writer. Parallel writes are atomic, and buffers are never interleaved or broken to the underlying Writer.