persistentqueue

Documentation

Index

• Constants
• type FastQueue
  • func MustOpenFastQueue(path, name string, maxInmemoryBlocks int, maxPendingBytes int64, ...) *FastQueue
  • func (fq *FastQueue) Dirname() string
  • func (fq *FastQueue) GetInmemoryQueueLen() int
  • func (fq *FastQueue) GetPendingBytes() uint64
  • func (fq *FastQueue) IsWriteBlocked() bool
  • func (fq *FastQueue) MustClose()
  • func (fq *FastQueue) MustReadBlock(dst []byte) ([]byte, bool)
  • func (fq *FastQueue) MustWriteBlockIgnoreDisabledPQ(block []byte)
  • func (fq *FastQueue) TryWriteBlock(block []byte) bool
  • func (fq *FastQueue) UnblockAllReaders()

Constants

const DefaultChunkFileSize = (MaxBlockSize + 8) * 16

DefaultChunkFileSize represents default chunk file size

const MaxBlockSize = 32 * 1024 * 1024

MaxBlockSize is the maximum size of the block persistent queue can work with.

Types

type FastQueue

type FastQueue struct {
	// contains filtered or unexported fields
}

FastQueue is fast persistent queue, which prefers sending data via memory.

It falls back to sending data via file when readers don't catch up with writers.

func MustOpenFastQueue

func MustOpenFastQueue(path, name string, maxInmemoryBlocks int, maxPendingBytes int64, isPQDisabled bool) *FastQueue

MustOpenFastQueue opens persistent queue at the given path.

It holds up to maxInmemoryBlocks in memory before falling back to file-based persistence.

if maxPendingBytes is 0, then the queue size is unlimited. Otherwise its size is limited by maxPendingBytes. The oldest data is dropped when the queue reaches maxPendingSize. if isPQDisabled is set to true, then write requests that exceed in-memory buffer capacity are rejected. in-memory queue part can be stored on disk during gracefull shutdown.

func (*FastQueue) Dirname

func (fq *FastQueue) Dirname() string

Dirname returns the directory name for persistent queue.

func (*FastQueue) GetInmemoryQueueLen

func (fq *FastQueue) GetInmemoryQueueLen() int

GetInmemoryQueueLen returns the length of inmemory queue.

func (*FastQueue) GetPendingBytes

func (fq *FastQueue) GetPendingBytes() uint64

GetPendingBytes returns the number of pending bytes in the fq.

func (*FastQueue) IsWriteBlocked

func (fq *FastQueue) IsWriteBlocked() bool

IsWriteBlocked checks if data can be pushed into fq

func (*FastQueue) MustClose

func (fq *FastQueue) MustClose()

MustClose unblocks all the readers.

It is expected no new writers during and after the call.

func (*FastQueue) MustReadBlock

func (fq *FastQueue) MustReadBlock(dst []byte) ([]byte, bool)

MustReadBlock reads the next block from fq to dst and returns it.

func (*FastQueue) MustWriteBlockIgnoreDisabledPQ

func (fq *FastQueue) MustWriteBlockIgnoreDisabledPQ(block []byte)

MustWriteBlockIgnoreDisabledPQ unconditionally writes block to fq.

This method allows perisisting in-memory blocks during graceful shutdown, even if persistence is disabled.

func (*FastQueue) TryWriteBlock

func (fq *FastQueue) TryWriteBlock(block []byte) bool

TryWriteBlock tries writing block to fq.

false is returned if the block couldn't be written to fq when the in-memory queue is full and the persistent queue is disabled.

func (*FastQueue) UnblockAllReaders

func (fq *FastQueue) UnblockAllReaders()

UnblockAllReaders unblocks all the readers.