batcher

README

batcher is a generic library to process objects in batches. A producer adds objects into the batcher, and the batcher asynchronously triggers the processor function with a batch of objects under 1 of 2 conditions:

1. The max buffered items has been reached
1. The flush period has occurred

Documentation

Index

• type Buffer
• type InMemory
  • func NewInMemory[Item any](buf Buffer[Item], flushDuration time.Duration) *InMemory[Item]
  • func (m *InMemory[Item]) Add(items ...Item) error
  • func (m *InMemory[_]) Close()

Types

type Buffer

type Buffer[Item any] interface {
	// Adds the given item to the buffer, even if it would cause the buffer to
	// exceed its size limit. Returns true if the buffer is at or exceeds its size
	// limit after this operation, and false otherwise.
	Add(item Item) (bool, error)

	// Flushes the buffer to the items' consumer. On return, the buffer will be
	// empty, even when an error occurs.
	Flush() error
}

Encapsulates a buffer of items and its consumer. The buffer has a soft size limit. Implementations will accept items that cause the buffer to exceed this limit, but clients are expected to call Flush on the buffer when the Add operation reports that the soft limit has been reached.

Implementations do not need to be thread-safe.

type InMemory

type InMemory[Item any] struct {
	// contains filtered or unexported fields
}

A wrapper around a Buffer[Item] that manages thread-safety and flushing.

func NewInMemory

func NewInMemory[Item any](
	buf Buffer[Item],
	flushDuration time.Duration,
) *InMemory[Item]

func (*InMemory[Item]) Add

func (m *InMemory[Item]) Add(items ...Item) error

func (*InMemory[_]) Close

func (m *InMemory[_]) Close()