persistentqueue

Documentation

Index

• Constants
• type FastQueue
  • func MustOpenFastQueue(path, name string, maxInmemoryBlocks int, maxPendingBytes int64) *FastQueue
  • func (fq *FastQueue) Dirname() string
  • func (fq *FastQueue) GetInmemoryQueueLen() int
  • func (fq *FastQueue) GetPendingBytes() uint64
  • func (fq *FastQueue) MustClose()
  • func (fq *FastQueue) MustReadBlock(dst []byte) ([]byte, bool)
  • func (fq *FastQueue) MustWriteBlock(block []byte)
  • func (fq *FastQueue) UnblockAllReaders()

Constants

const DefaultChunkFileSize = (MaxBlockSize + 8) * 16

DefaultChunkFileSize represents default chunk file size

const MaxBlockSize = 32 * 1024 * 1024

MaxBlockSize is the maximum size of the block persistent queue can work with.

Types

type FastQueue

type FastQueue struct {
	// contains filtered or unexported fields
}

FastQueue is fast persistent queue, which prefers sending data via memory.

It falls back to sending data via file when readers don't catch up with writers.

func MustOpenFastQueue

func MustOpenFastQueue(path, name string, maxInmemoryBlocks int, maxPendingBytes int64) *FastQueue

MustOpenFastQueue opens persistent queue at the given path.

It holds up to maxInmemoryBlocks in memory before falling back to file-based persistence.

if maxPendingBytes is 0, then the queue size is unlimited. Otherwise its size is limited by maxPendingBytes. The oldest data is dropped when the queue reaches maxPendingSize.

func (*FastQueue) Dirname

func (fq *FastQueue) Dirname() string

Dirname returns the directory name for persistent queue.

func (*FastQueue) GetInmemoryQueueLen

func (fq *FastQueue) GetInmemoryQueueLen() int

GetInmemoryQueueLen returns the length of inmemory queue.

func (*FastQueue) GetPendingBytes

func (fq *FastQueue) GetPendingBytes() uint64

GetPendingBytes returns the number of pending bytes in the fq.

func (*FastQueue) MustClose

func (fq *FastQueue) MustClose()

MustClose unblocks all the readers.

It is expected no new writers during and after the call.

func (*FastQueue) MustReadBlock

func (fq *FastQueue) MustReadBlock(dst []byte) ([]byte, bool)

MustReadBlock reads the next block from fq to dst and returns it.

func (*FastQueue) MustWriteBlock

func (fq *FastQueue) MustWriteBlock(block []byte)

MustWriteBlock writes block to fq.

func (*FastQueue) UnblockAllReaders

func (fq *FastQueue) UnblockAllReaders()

UnblockAllReaders unblocks all the readers.