pargzip

Documentation

Overview

Package pargzip contains a parallel gzip writer implementation. By compressing each chunk of data in parallel, all the CPUs on the machine can be used, at a slight loss of compression efficiency. In addition, this implementation can use the system gzip binary as a child process, which is faster than Go's native implementation.

Index

• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w *Writer) Close() error
  • func (w *Writer) Write(p []byte) (n int, err error)

Types

type Writer

type Writer struct {
	// UseSystemGzip controls whether the system gzip binary is
	// used. The default from NewWriter is true.
	UseSystemGzip bool

	// ChunkSize is the number of bytes to gzip at once.
	// The default from NewWriter is 1MB.
	ChunkSize int

	// Parallel is the number of chunks to compress in parallel.
	// The default from NewWriter is runtime.NumCPU().
	Parallel int
	// contains filtered or unexported fields
}

A Writer is an io.WriteCloser. Writes to a Writer are compressed and written to w.

Any exported fields may only be mutated before the first call to Write.

func NewWriter

func NewWriter(w io.Writer) *Writer

NewWriter returns a new Writer. Writes to the returned writer are compressed and written to w.

It is the caller's responsibility to call Close on the WriteCloser when done. Writes may be buffered and not flushed until Close.

Any fields on Writer may only be modified before the first call to Write.

func (*Writer) Close

func (w *Writer) Close() error

func (*Writer) Write

func (w *Writer) Write(p []byte) (n int, err error)