delimited

Documentation

Overview

Package delimited implements a reader and writer for simple streams of length-delimited byte records. Each record is written as a varint-encoded length in bytes, followed immediately by the record itself.

A stream consists of a sequence of such records packed consecutively without additional padding. There are no checksums or compression.

Index

• func Copy(sink Sink, src Source) error
• type Reader
  • func NewReader(r io.Reader) *Reader
  • func (r *Reader) Next() ([]byte, error)
  • func (r *Reader) NextProto(pb proto.Message) error
• type Sink
• type Source
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w Writer) Put(record []byte) error
  • func (w Writer) PutProto(msg proto.Message) error
  • func (w Writer) WriteRecord(record []byte) (int, error)

Functions

func Copy

func Copy(sink Sink, src Source) error

Copy sequentially copies each record read from src to sink until src.Next() returns io.EOF or another error occurs.

Types

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader consumes length-delimited records from a byte source.

Usage:

rd := delimited.NewReader(r)
for {
  rec, err := rd.Next()
  if err == io.EOF {
    break
  } else if err != nil {
    log.Fatal(err)
  }
  doStuffWith(rec)
}

func NewReader

func NewReader(r io.Reader) *Reader

NewReader constructs a new delimited Reader for the records in r.

func (*Reader) Next

func (r *Reader) Next() ([]byte, error)

Next returns the next length-delimited record from the input, or io.EOF if there are no more records available. Returns io.ErrUnexpectedEOF if a short record is found, with a length of n but fewer than n bytes of data. Because there is no resynchronization mechanism, it is generally not possible to recover from a short record in this format.

The slice returned is valid only until a subsequent call to Next.

func (*Reader) NextProto

func (r *Reader) NextProto(pb proto.Message) error

NextProto consumes the next available record by calling r.Next, and decodes it into pb with proto.Unmarshal.

type Sink

type Sink interface {
	// Put delivers a record to the sink.
	Put([]byte) error
}

A Sink represents a receiver of records.

type Source

type Source interface {
	// Next returns the next record in the sequence, or io.EOF when no further
	// records are available. The slice returned by Next is only required to be
	// valid until a subsequent call to Next.
	Next() ([]byte, error)
}

A Source represents a sequence of records.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

A Writer outputs delimited records to an io.Writer.

Basic usage:

wr := delimited.NewWriter(w)
for record := range records {
   if err := wr.Put(record); err != nil {
     log.Fatal(err)
   }
}

func NewWriter

func NewWriter(w io.Writer) *Writer

NewWriter constructs a new delimited Writer that writes records to w.

func (Writer) Put

func (w Writer) Put(record []byte) error

Put writes the specified record to the writer. It equivalent to WriteRecord, but discards the number of bytes written.

func (Writer) PutProto

func (w Writer) PutProto(msg proto.Message) error

PutProto encodes and writes the specified proto.Message to the writer.

func (Writer) WriteRecord

func (w Writer) WriteRecord(record []byte) (int, error)

WriteRecord writes the specified record to the underlying writer, returning the total number of bytes written including the length tag.