offset2

Documentation

Overview

Package offset2 implements a margaret log as persisted sequence of data across multiple files.

Format Defintion

A log consists of three files: data, ofst and jrnl.

* data: a list of length-prefixed data chunks, size is a uint64 (size++[size]byte).

* ofst: a list of uint64, representing entry offsets in 'data'

* jrnl keeps track of the current sequence number, see checkJournal() for more

To read entry 5 in `data`, you follow these steps:

1. Seek to 5*(sizeof(uint64)=8)=40 in `ofset` and read the uint64 representing the offset in `data`

2. Seek to that offset in `data`, read the length-prefix (the uint64 for the size of the entry)

3. Finally, read that amount of data, which is your entry

All uint64's are encoded in BigEndian.

Index

• func LockSink(sink luigi.Sink) luigi.Sink
• type OffsetLog
  • func Open(name string, cdc margaret.Codec) (*OffsetLog, error)
  • func (log *OffsetLog) Append(v interface{}) (int64, error)
  • func (log *OffsetLog) Changes() luigi.Observable
  • func (log *OffsetLog) CheckConsistency() error
  • func (log *OffsetLog) Close() error
  • func (log *OffsetLog) FileName() string
  • func (log *OffsetLog) Get(seq int64) (interface{}, error)
  • func (log *OffsetLog) Null(seq int64) error
  • func (log *OffsetLog) Query(specs ...margaret.QuerySpec) (luigi.Source, error)
  • func (log *OffsetLog) Replace(seq int64, data []byte) error
  • func (log *OffsetLog) Seq() int64

Functions

func LockSink

func LockSink(sink luigi.Sink) luigi.Sink

Types

type OffsetLog

type OffsetLog struct {
	// contains filtered or unexported fields
}

func Open

func Open(name string, cdc margaret.Codec) (*OffsetLog, error)

Open returns a the offset log in the directory at `name`. If it is empty or does not exist, a new log will be created.

func (*OffsetLog) Append

func (log *OffsetLog) Append(v interface{}) (int64, error)

func (*OffsetLog) Changes

func (log *OffsetLog) Changes() luigi.Observable

func (*OffsetLog) CheckConsistency

func (log *OffsetLog) CheckConsistency() error

CheckConsistency is an fsck for the offset log.

func (*OffsetLog) Close

func (log *OffsetLog) Close() error

func (*OffsetLog) FileName

func (log *OffsetLog) FileName() string

func (*OffsetLog) Get

func (log *OffsetLog) Get(seq int64) (interface{}, error)

func (*OffsetLog) Null

func (log *OffsetLog) Null(seq int64) error

Null overwrites the entry at seq with zeros updating is kinda odd in append-only but in some cases you still might want to redact entries

func (*OffsetLog) Query

func (log *OffsetLog) Query(specs ...margaret.QuerySpec) (luigi.Source, error)

func (*OffsetLog) Replace

func (log *OffsetLog) Replace(seq int64, data []byte) error

Replace overwrites the seq entry with data. data has to be smaller then the current entry.

func (*OffsetLog) Seq

func (log *OffsetLog) Seq() int64