weave

Documentation

Overview

Package weave reads and writes multiple revisions of line-oriented data to a weave file.

A weave is a type of delta encoding that stores all of the revisions of the file linearily.

The weave sequence consists of 3 control messages interspersed with lines of plain data. The control messages are: Insert to begin data associated with a given delta, Delete to indicate a section no longer applies as of a given delta, and End to indicate a previous Insert or Delete no longer applies.

Each control message has an associated delta number, which begin with '1' and are increasing integers.

Index

• Variables
• func ReadDelta(nc NamingConvention, delta int, line func(text string) error) error
• func ReadGeneral(nc NamingConvention, delta int, sink Sink) error
• func TempFile(nc NamingConvention, compressed bool) (file *os.File, err error)
• type BytesReader
• type Delta
• type DeltaWriter
  • func NewDeltaWriter(nc NamingConvention, base int, name string, tags map[string]string) (*DeltaWriter, error)
  • func (w *DeltaWriter) Close() error
  • func (w *DeltaWriter) Write(p []byte) (n int, err error)
• type Header
  • func LoadHeader(r BytesReader) (*Header, error)
  • func NewHeader() Header
  • func ReadHeader(nc NamingConvention) (*Header, error)
  • func (h *Header) AddDelta(name string, tags map[string]string) int
  • func (h *Header) LatestDelta() int
  • func (h *Header) PenultimateDelta() (int, error)
  • func (h *Header) Save(w io.Writer) error
• type NamingConvention
• type NewWeaveWriter
  • func NewNewWeave(nc NamingConvention, name string, tags map[string]string) (*NewWeaveWriter, error)
  • func (w *NewWeaveWriter) Close() error
  • func (w *NewWeaveWriter) Write(p []byte) (n int, err error)
• type Parser
  • func NewParser(rd io.Reader, sink Sink, delta int) *Parser
  • func (p *Parser) ParseTo(lineno int) error
• type SimpleNaming
  • func (sn *SimpleNaming) BackupFile() string
  • func (sn *SimpleNaming) IsCompressed() bool
  • func (sn *SimpleNaming) MainFile() string
  • func (sn *SimpleNaming) MakeName(ext string, compressed bool) string
  • func (sn *SimpleNaming) TempFile(num int, compressed bool) string
• type Sink
• type WriteDelta
  • func NewWriteDelta(wr *bufio.Writer) WriteDelta
  • func (w WriteDelta) Delete(delta int) error
  • func (w WriteDelta) End(delta int) error
  • func (w WriteDelta) Insert(delta int) error
  • func (w WriteDelta) Plain(text string, keep bool) error
• type Writer
  • func (w Writer) Delete(delta int) error
  • func (w Writer) End(delta int) error
  • func (w Writer) Insert(delta int) error
  • func (w Writer) Plain(text string, keep bool) error

Variables

var ErrInvalidHeader = fmt.Errorf("invalid weave header")

ErrInvalidHeader indicates that the header in the weave file was malformed. This usually means this isn't actually a weave file.

Functions

func ReadDelta

func ReadDelta(nc NamingConvention, delta int, line func(text string) error) error

ReadDelta reads the contents of a given delta from a delta file. The 'line' function will be called on each line of text from the delta. If 'line' returns a non-nil error, it will be propagated up through the call to ReadDelta, otherwise, ReadDelta will return any error encountered in reading.

func ReadGeneral

func ReadGeneral(nc NamingConvention, delta int, sink Sink) error

ReadGeneral reads a delta using the specified Sink.

func TempFile

func TempFile(nc NamingConvention, compressed bool) (file *os.File, err error)

TempFile will create a tempfile, the file, and an error.

Types

type BytesReader

type BytesReader interface {
	ReadBytes(delim byte) ([]byte, error)
}

A BytesReader is something that can be read a line at a time.

type Delta

type Delta struct {
	Name   string            `json:"name"`
	Number int               `json:"number"`
	Tags   map[string]string `json:"tags"`
	Time   time.Time         `json:"time"`
}

A Delta describes a single version of the data stored in the weave file. They are numbered, ideally sequentially, starting with 1, have a name, a timestamp, and a set of tags.

type DeltaWriter

type DeltaWriter struct {
	// contains filtered or unexported fields
}

A DeltaWriter is used to write a new version to a weave file.

func NewDeltaWriter

func NewDeltaWriter(nc NamingConvention, base int, name string, tags map[string]string) (*DeltaWriter, error)

NewDeltaWriter create a new DeltaWriter. The contents should be written to the DeltaWriter, and the 'Close' method called. Note that most of the work is done as part of the Close method. 'base' should be set to the delta number that this change is based on. This will typically be the last delta written. Note that the tags should not be changed until after Close is called.

func (*DeltaWriter) Close

func (w *DeltaWriter) Close() error

Close closes the delta writer. This causes the delta to actually be generated, so it is important to check the error status from this method.

func (*DeltaWriter) Write

func (w *DeltaWriter) Write(p []byte) (n int, err error)

type Header

type Header struct {
	Version int      `json:"version"`
	Deltas  []*Delta `json:"deltas"`
}

A Header is at the beginning of ever weave file. It describes each of the deltas in the file. The version describes the version of the header, in this case, 1.

func LoadHeader

func LoadHeader(r BytesReader) (*Header, error)

LoadHeader reads the header from the stream.

func NewHeader

func NewHeader() Header

NewHeader creates a blank header describing zero deltas.

func ReadHeader

func ReadHeader(nc NamingConvention) (*Header, error)

ReadHeader reads the header from the weave file described by the naming convention.

func (*Header) AddDelta

func (h *Header) AddDelta(name string, tags map[string]string) int

AddDelta adds a new delta to this header, of a given name and tags. The tags will be copied, the Time filled in, and the number returned.

func (*Header) LatestDelta

func (h *Header) LatestDelta() int

LatestDelta returns the most recent delta in the store. Zero means there are no deltas (this is valid for a new blank header).

func (*Header) PenultimateDelta

func (h *Header) PenultimateDelta() (int, error)

PenultimateDelta returns the penultimate delta, if there is one. Will return an error if this it not the case.

func (*Header) Save

func (h *Header) Save(w io.Writer) error

Save writes the header to the stream in the format used in the weave files.

type NamingConvention

type NamingConvention interface {
	// Generate a temporary name.  The file may or may not exist
	// (not checked).  If the name is already present, this can be
	// called again with a different num to generate a different
	// name.  Compressed is a hint as to whether this name should
	// match compression.
	TempFile(num int, compressed bool) string

	// Return the pathname of the primary file.
	MainFile() string

	// Return the pathname of the backup file.
	BackupFile() string

	// Return if compression is requested on the main file.
	IsCompressed() bool
}

A NamingConvention determines the names of various temp files. The SCCS conventions are not followed, because they are not safe (this code will never write to a file that already exists).

type NewWeaveWriter

type NewWeaveWriter struct {
	// contains filtered or unexported fields
}

NewWeaveWriter implements io.Writer (buffered) to write to a new weave file. The client should call the Close method to finalize writing.

func NewNewWeave

func NewNewWeave(nc NamingConvention, name string, tags map[string]string) (*NewWeaveWriter, error)

NewNewWeave creates a new weave file. The file will be named based on the given naming convention. The 'name' will be used for the initial delta, and the tags will be recorded in that delta. Close must be called to finialize the weaving. Note that the underlying writer is buffered.

func (*NewWeaveWriter) Close

func (w *NewWeaveWriter) Close() error

Close closes the NewWeaveWriter, renaming the new file to the finalized name.

func (*NewWeaveWriter) Write

func (w *NewWeaveWriter) Write(p []byte) (n int, err error)

type Parser

type Parser struct {
	// The source of data for the reader.
	Source *bufio.Reader

	// A possible sync for the data read.  The reader will call
	// into the Sync for each record of the weave file.
	Sink Sink

	// Delta is the desired delta number to retrieve.  This
	// affects the ParseTo call, as well as the 'keep' argument
	// passed to Sink's Plain call.
	Delta int
	// contains filtered or unexported fields
}

Parser scans as input a weave file. The behavior is controlled by some of the fields.

func NewParser

func NewParser(rd io.Reader, sink Sink, delta int) *Parser

NewParser constructs a new reader that reads from the given Reader, targeting the given delta. Will call into sink for each record.

func (*Parser) ParseTo

func (p *Parser) ParseTo(lineno int) error

ParseTo runs the parser until we reach line 'lineno'. Lines are numbered from 1, so calling with a lineno of zero will run the parser until the end of the input. Returns an error if there is one, can also return io.EOF to indicate the end of parsing.

type SimpleNaming

type SimpleNaming struct {
	Path       string // The directory for the files to be written.
	Base       string // The base of the filename.
	Ext        string // The extension to use for the main name.
	Compressed bool   // Are these names to indicate compression.
}

The SimpleNaming is a NamingConvention that has a basename, with the main file having a specified extension, the backup file having a ".bak" extension, and the temp files using a numbered extension starting with ".0". If the names are intended to be compressed, a ".gz" suffix can also be added.

func (*SimpleNaming) BackupFile

func (sn *SimpleNaming) BackupFile() string

BackupFile returns the name of the backup file for this naming.

func (*SimpleNaming) IsCompressed

func (sn *SimpleNaming) IsCompressed() bool

IsCompressed returns whether this naming convention expects datafiles to be compressed.

func (*SimpleNaming) MainFile

func (sn *SimpleNaming) MainFile() string

MainFile returns the name of the primary file for this naming.

func (*SimpleNaming) MakeName

func (sn *SimpleNaming) MakeName(ext string, compressed bool) string

MakeName constructs a name with a given extention and possibility of being compressed.

func (*SimpleNaming) TempFile

func (sn *SimpleNaming) TempFile(num int, compressed bool) string

TempFile returns the name of a temp file, containing a given sequence number, and with the desired compression.

type Sink

type Sink interface {
	// Begin an insert sequence for the given delta.
	Insert(delta int) error
	// Begin a delete sequence for the given delta.
	Delete(delta int) error
	// End a previous Insert or Delete
	End(delta int) error
	// A line of plain text.  `keep` indicates whether or not the
	// text should be included in the Delta specified in the
	// Reader.
	Plain(text string, keep bool) error
}

A Sink accepts a weave data stream, and processes it in some manner. All return a possible error, which will propagate upward.

type WriteDelta

type WriteDelta struct {
	// contains filtered or unexported fields
}

WriteDelta is a Sink that merely writes the plaintext of a given delta to the given writer. Does not close the writer when it is finished.

func NewWriteDelta

func NewWriteDelta(wr *bufio.Writer) WriteDelta

NewWriteDelta creates a WriteDelta that just writes the plaintext to the given writer.

func (WriteDelta) Delete

func (w WriteDelta) Delete(delta int) error

Delete marks a delete, does nothing.

func (WriteDelta) End

func (w WriteDelta) End(delta int) error

End marks the end, does nothing.

func (WriteDelta) Insert

func (w WriteDelta) Insert(delta int) error

Insert marks an insert, does nothing.

func (WriteDelta) Plain

func (w WriteDelta) Plain(text string, keep bool) error

Plain possibly adds plaintext.

type Writer

type Writer struct {
	io.Writer
}

A Writer is a sync that outputs weave data to the given output. The particular delta is ignored, and the entire output will be emitted.

func (Writer) Delete

func (w Writer) Delete(delta int) error

Delete marks a new delete.

func (Writer) End

func (w Writer) End(delta int) error

End marks a new end.

func (Writer) Insert

func (w Writer) Insert(delta int) error

Insert marks a new insert.

func (Writer) Plain

func (w Writer) Plain(text string, keep bool) error

Plain marks a new line of plaintext.