dedup

Documentation

Index

• func DefaultPath() string
• type Dedup
  • func New(path string, fns ...OptFn) Dedup
• type KeyValue
• type OptFn
  • func WithClearDB() OptFn
  • func WithWindow(d time.Duration) OptFn

Functions

func DefaultPath

func DefaultPath() string

DefaultPath returns the default path for the deduplication service's badger DB

Types

type Dedup

type Dedup interface {
	// Set returns [true] if it was the first time the key was encountered, otherwise it returns [false] along with the previous value
	Set(kv KeyValue) (bool, int64)

	// Commit commits a list of previously set keys to the DB
	Commit(keys []string) error

	// Close closes the deduplication service
	Close()
}

Dedup is the interface for deduplication service

func New

func New(path string, fns ...OptFn) Dedup

New creates a new deduplication service. The service needs to be closed after use.

type KeyValue

type KeyValue struct {
	Key   string
	Value int64
}

type OptFn

type OptFn func(*badgerDB)

func WithClearDB

func WithClearDB() OptFn

WithClearDB clears the DB on startup

func WithWindow

func WithWindow(d time.Duration) OptFn

WithWindow sets the window for deduplication