bufferutil

Documentation

Overview

Package bufferutil provides helpers for dealing with buffers.

The main exported types are `Pool` or a re-usable pool of memory chunks, and BufferHandlers which are used to syndicate binary writes to multiple listeners.

Index

• type Buffer
  • func NewBuffer(contents []byte) *Buffer
  • func (b *Buffer) Bytes() []byte
  • func (b *Buffer) String() string
  • func (b *Buffer) Write(contents []byte) (written int, err error)
• type BufferChunk
  • func (bc BufferChunk) Copy() BufferChunk
  • func (bc BufferChunk) MarshalJSON() ([]byte, error)
  • func (bc *BufferChunk) UnmarshalJSON(contents []byte) error
• type BufferChunkHandler
• type BufferHandlers
  • func (bl *BufferHandlers) Add(uid string, handler BufferChunkHandler)
  • func (bl *BufferHandlers) Close()
  • func (bl *BufferHandlers) Handle(chunk BufferChunk)
  • func (bl *BufferHandlers) Remove(uid string)
• type Pool
  • func NewPool(bufferSize int) *Pool
  • func (p *Pool) Get() *bytes.Buffer
  • func (p *Pool) Put(b *bytes.Buffer)
• type PutOnCloser
  • func PutOnClose(buffer *bytes.Buffer, pool *Pool) *PutOnCloser
  • func (poc PutOnCloser) Close() error

Types

type Buffer

type Buffer struct {
	sync.RWMutex
	// Size is the size of all bytes written to the buffer.
	Size int64
	// Lines are the string lines broken up by newlines with associated timestamps
	Chunks []BufferChunk `json:"chunks"`
	// Handler is an optional listener for new line events.
	Handler BufferChunkHandler `json:"-"`
}

Buffer is a writer that accepts binary but splits out onto new lines.

func NewBuffer

func NewBuffer(contents []byte) *Buffer

NewBuffer creates a new line writer from a given set of bytes.

func (*Buffer) Bytes

func (b *Buffer) Bytes() []byte

Bytes returns the bytes written to the writer.

func (*Buffer) String

func (b *Buffer) String() string

String returns the current combined output as a string.

func (*Buffer) Write

func (b *Buffer) Write(contents []byte) (written int, err error)

Write writes the contents to the output buffer. An important gotcha here is the `contents` parameter is by reference, as a result you can get into some bad loop capture states where this buffer will be assigned to multiple chunks but be effectively the same value. As a result, when you write to the output buffer, we fully copy this contents parameter for storage in the buffer.

type BufferChunk

type BufferChunk struct {
	Timestamp time.Time
	Data      []byte
}

BufferChunk is a single write to a buffer with a timestamp.

func (BufferChunk) Copy

func (bc BufferChunk) Copy() BufferChunk

Copy returns a copy of the chunk.

func (BufferChunk) MarshalJSON

func (bc BufferChunk) MarshalJSON() ([]byte, error)

MarshalJSON implemnts json.Marshaler.

func (*BufferChunk) UnmarshalJSON

func (bc *BufferChunk) UnmarshalJSON(contents []byte) error

UnmarshalJSON implements json.Unmarshaler.

type BufferChunkHandler

type BufferChunkHandler func(BufferChunk)

BufferChunkHandler is a handler for output chunks.

type BufferHandlers

type BufferHandlers struct {
	sync.RWMutex
	Handlers map[string]*async.Worker
}

BufferHandlers is a synchronized map of listeners for new lines to a line buffer.

func (*BufferHandlers) Add

func (bl *BufferHandlers) Add(uid string, handler BufferChunkHandler)

Add adds a handler.

func (*BufferHandlers) Close

func (bl *BufferHandlers) Close()

Close shuts down the handlers.

func (*BufferHandlers) Handle

func (bl *BufferHandlers) Handle(chunk BufferChunk)

Handle calls the handlers.

func (*BufferHandlers) Remove

func (bl *BufferHandlers) Remove(uid string)

Remove removes a listener.

type Pool

type Pool struct {
	sync.Pool
}

Pool is a sync.Pool of bytes.Buffer.

func NewPool

func NewPool(bufferSize int) *Pool

NewPool returns a new Pool, which returns bytes buffers pre-sized to a given minimum size.

The purpose of a buffer pool is to reduce the number of gc collections incurred when using bytes buffers repeatedly; instead of marking the buffer as to be collected, it is returned to the pool to be re-used.

Example:

pool := bufferutil.NewPool(1024) // pre-allocate 1024 bytes per buffer.

func() {
	buf := pool.Get()
	defer pool.Put(buf)

	// do things with the buffer ...
}()

func (*Pool) Get

func (p *Pool) Get() *bytes.Buffer

Get returns a pooled bytes.Buffer instance.

func (*Pool) Put

func (p *Pool) Put(b *bytes.Buffer)

Put returns the pooled instance.

type PutOnCloser

type PutOnCloser struct {
	*bytes.Buffer
	Pool *Pool
}

PutOnCloser is a helper wrapper that will return a buffer to a given pool.

func PutOnClose

func PutOnClose(buffer *bytes.Buffer, pool *Pool) *PutOnCloser

PutOnClose wraps a buffer with a close function that will return the buffer to the pool.

func (PutOnCloser) Close

func (poc PutOnCloser) Close() error

Close returns the buffer to the pool.