legacyflumeoffset

Documentation

Overview

Package legacyflumeoffset implements the first iteration of an offset file. as implemented by https://github.com/flumedb/flumelog-offset Format

data is stored in a append only log, where the byte index of the start of a record is the primary key (offset). offset-><data.length (UInt32BE)> <data ...> <data.length (UInt32BE)> <file_length (UInt32BE or Uint48BE or Uint53BE)> by writing the length of the data both before and after each record it becomes possible to scan forward and backward (like a doubly linked list) It's very handly to be able to scan backwards, as often you want to see the last N items, and so you don't need an index for this.

Index

• type Log
  • func Open(path string, codec margaret.Codec) (*Log, error)
  • func (l *Log) Append(v interface{}) (int64, error)
  • func (l *Log) Changes() luigi.Observable
  • func (l *Log) Get(ofst int64) (interface{}, error)
  • func (log *Log) Query(specs ...margaret.QuerySpec) (luigi.Source, error)
  • func (l *Log) Seq() int64

Types

type Log

type Log struct {
	// contains filtered or unexported fields
}

func Open

func Open(path string, codec margaret.Codec) (*Log, error)

func (*Log) Append

func (l *Log) Append(v interface{}) (int64, error)

Append appends a new entry to the log

func (*Log) Changes

func (l *Log) Changes() luigi.Observable

func (*Log) Get

func (l *Log) Get(ofst int64) (interface{}, error)

Get returns the entry with sequence number seq

func (*Log) Query

func (log *Log) Query(specs ...margaret.QuerySpec) (luigi.Source, error)

Query returns a stream that is constrained by the passed query specification

func (*Log) Seq

func (l *Log) Seq() int64