fs

Documentation

Overview

Package fs implements write-ahead-log on FileSystem.

Index

• Constants
• func NewAlignedReadWriteWAL(filePath string, maxBufferSize int64, syncDuration time.Duration, ...) (wal.WAL, error)
• func NewAlignedWriteOnlyWAL(id *partition.ID, filePath string, maxBufferSize int64, ...) (wal.WAL, error)
• func NewFSManager(vertexInstance *dfv1.VertexInstance, opts ...Option) wal.Manager
• type Option
  • func WithMaxBufferSize(size int64) Option
  • func WithStorePath(path string) Option
  • func WithSyncDuration(maxDuration time.Duration) Option

Constants

const (
	IEEE            = 0xedb88320
	EntryHeaderSize = 28
	SegmentPrefix   = "segment"
)

Functions

func NewAlignedReadWriteWAL

func NewAlignedReadWriteWAL(filePath string,
	maxBufferSize int64,
	syncDuration time.Duration,
	pipelineName string,
	vertexName string,
	replica int32) (wal.WAL, error)

NewAlignedReadWriteWAL creates a new alignedWAL instance for read-write. This will be used during boot up where we will be replaying the messages from the alignedWAL and then writing to it.

func NewAlignedWriteOnlyWAL

func NewAlignedWriteOnlyWAL(id *partition.ID,
	filePath string,
	maxBufferSize int64,
	syncDuration time.Duration,
	pipelineName string,
	vertexName string,
	replica int32) (wal.WAL, error)

NewAlignedWriteOnlyWAL creates a new alignedWAL instance for write-only. This will be used in happy path where we are only writing to the alignedWAL.

func NewFSManager

func NewFSManager(vertexInstance *dfv1.VertexInstance, opts ...Option) wal.Manager

NewFSManager is a FileSystem WAL Manager.

Types

type Option

type Option func(stores *fsManager)

func WithMaxBufferSize

func WithMaxBufferSize(size int64) Option

WithMaxBufferSize sets the alignedWAL buffer max size option

func WithStorePath

func WithStorePath(path string) Option

WithStorePath sets the alignedWAL store path

func WithSyncDuration

func WithSyncDuration(maxDuration time.Duration) Option

WithSyncDuration sets the alignedWAL sync duration option