dbin

README

dfuse dbin Library

dbin is a simple file storage format to pack a stream of protobuf messages. It is part of dfuse.

Usage

See example usage in merger

File format

First four magic bytes:

• 'd', 'b', 'i', 'n'

Next single byte:

• file format version, current is 0x00

Next three bytes:

• content type, like 'ETH', 'EOS', or whatever..

Next two bytes:

• 10-based string representation of content version: '00' for version 0, '99', for version 99

Rest of the file:

• Length-prefixed messages, with each length specified as 4 bytes big-endian uint32.
• Followed by message of that length, then start over.
• EOF reached when no more bytes exist after the last message boundary.

Contributing

Issues and PR in this repo related strictly to the dbin library.

Report any protocol-specific issues in their respective repositories

Please first refer to the general dfuse contribution guide, if you wish to contribute to this code base.

License

Apache 2.0

Documentation

Index

• type Reader
  • func NewFileReader(filename string) (*Reader, error)
  • func NewReader(r io.Reader) *Reader
  • func (r *Reader) Close() error
  • func (r *Reader) ReadHeader() (contentType string, version int32, err error)
  • func (r *Reader) ReadMessage() ([]byte, error)
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w *Writer) Close() error
  • func (w *Writer) WriteHeader(contentType string, version int) error
  • func (w *Writer) WriteMessage(msg []byte) error

Types

type Reader

type Reader struct {
	io.Reader
	// contains filtered or unexported fields
}

func NewFileReader

func NewFileReader(filename string) (*Reader, error)

func NewReader

func NewReader(r io.Reader) *Reader

func (*Reader) Close

func (r *Reader) Close() error

func (*Reader) ReadHeader

func (r *Reader) ReadHeader() (contentType string, version int32, err error)

func (*Reader) ReadMessage

func (r *Reader) ReadMessage() ([]byte, error)

ReadMessage reads next message from byte stream

type Writer

type Writer struct {
	io.Writer
	// contains filtered or unexported fields
}

func NewWriter

func NewWriter(w io.Writer) *Writer

func (*Writer) Close

func (w *Writer) Close() error

func (*Writer) WriteHeader

func (w *Writer) WriteHeader(contentType string, version int) error

func (*Writer) WriteMessage

func (w *Writer) WriteMessage(msg []byte) error