chunk

Documentation

Overview

Package chunk provides a utility for sending sets of protobuf messages in groups of smaller chunks. This is useful for gRPC, which has limitations around the maximum size of a message that you can send.

This code is adapted from the gitaly project, which is licensed under the MIT license. A copy of that license text can be found at https://mit-license.org/.

The code this file was based off can be found here: https://gitlab.com/gitlab-org/gitaly/-/blob/v16.2.0/internal/helper/chunk/chunker.go

Index

• func SendAll[T proto.Message](sendFunc func([]T) error, items ...T) error
• type Chunker
  • func New[T proto.Message](sendFunc func([]T) error) *Chunker[T]
  • func (c *Chunker[T]) Flush() error
  • func (c *Chunker[T]) Send(items ...T) error

Functions

func SendAll

func SendAll[T proto.Message](sendFunc func([]T) error, items ...T) error

SendAll is a convenience function that immediately sends all provided items in smaller chunks using the provided sendFunc.

See the documentation for Chunker.Send() for more information.

Types

type Chunker

type Chunker[T proto.Message] struct {
	// contains filtered or unexported fields
}

Chunker lets you spread items you want to send over multiple chunks. This type is not thread-safe.

func New

func New[T proto.Message](sendFunc func([]T) error) *Chunker[T]

New returns a new Chunker that will use the given sendFunc to send chunks of messages.

func (*Chunker[T]) Flush

func (c *Chunker[T]) Flush() error

Flush sends remaining items in the current chunk, if any.

func (*Chunker[T]) Send

func (c *Chunker[T]) Send(items ...T) error

Send will append the provided items to the current chunk, and send the chunk if it is full.

Callers should ensure that they call Flush() after the last call to Send().