sereal

Documentation

Overview

Package sereal implements the Sereal serialization format

It follows the standard Go Marshal/Unmarshal interface.

For more information on Sereal, please see http://blog.booking.com/sereal-a-binary-data-serialization-format.html and http://github.com/Sereal/Sereal

Index

• Constants
• Variables
• func DecompressDocument(dst, b []byte) (r []byte, err error)
• func LooksLikeSereal(b []byte) bool
• func Marshal(body interface{}) ([]byte, error)
• func Unmarshal(b []byte, body interface{}) error
• type Decoder
  • func NewDecoder() *Decoder
  • func (d *Decoder) RegisterName(name string, value interface{})
  • func (d *Decoder) Unmarshal(b []byte, vbody interface{}) (err error)
  • func (d *Decoder) UnmarshalHeader(b []byte, vheader interface{}) (err error)
  • func (d *Decoder) UnmarshalHeaderBody(b []byte, vheader interface{}, vbody interface{}) (err error)
• type Encoder
  • func NewEncoder() *Encoder
  • func NewEncoderV2() *Encoder
  • func NewEncoderV3() *Encoder
  • func (e *Encoder) Marshal(body interface{}) (b []byte, err error)
  • func (e *Encoder) MarshalWithHeader(header interface{}, body interface{}) (b []byte, err error)
• type ErrCorrupt
  • func (c ErrCorrupt) Error() string
• type Merger
  • func NewMerger() *Merger
  • func NewMergerV2() *Merger
  • func NewMergerV3() *Merger
  • func (m *Merger) Append(b []byte) (int, error)
  • func (m *Merger) Finish() ([]byte, error)
• type PerlAlias
• type PerlFreeze
• type PerlObject
• type PerlRegexp
• type PerlUndef
  • func PerlCanonicalUndef() *PerlUndef
• type PerlWeakRef
• type SnappyCompressor
• type ZlibCompressor
• type ZstdCompressor

Constants

const (
	TopLevelArray topLevelElementType = iota
	TopLevelArrayRef
)

Top-level data structures for merged documents

const (
	ZlibBestSpeed          = zlib.BestSpeed
	ZlibBestCompression    = zlib.BestCompression
	ZlibDefaultCompression = zlib.DefaultCompression
)

Zlib constants

const (
	ZstdBestSpeed          = 1
	ZstdBestCompression    = 20
	ZstdDefaultCompression = 3
)

Zstd constants

const ProtocolVersion = 3

ProtocolVersion is a maximum version supported by the sereal package.

Variables

var (
	ErrBadHeaderUTF8 = errors.New("bad header: it seems your document was accidentally UTF-8 encoded")
	ErrBadHeader     = errors.New("bad header: not a valid Sereal document")
	ErrBadSnappy     = errors.New("snappy compression only valid for v1 documents")
	ErrBadZlibV3     = errors.New("zlib compression only valid for v3 documents and up")
	ErrBadZstdV4     = errors.New("zstd compression only valid for v4 documents and up")

	ErrHeaderPointer = errors.New("expected pointer for header")
	ErrBodyPointer   = errors.New("expected pointer for body")

	ErrTruncated  = errors.New("truncated document")
	ErrUnknownTag = errors.New("unknown tag byte")

	ErrTooLarge = errors.New("sereal: document too large to be compressed with snappy")

	ErrMaxRecursionLimit = errors.New("sereal: recursion limit exceeded")
)

Errors

Functions

func DecompressDocument

func DecompressDocument(dst, b []byte) (r []byte, err error)

DecompressDocument changes the compression type of a Sereal document without performing a full re-serialization

One use case is to use this before decoding, to avoid the implicit allocation.

func LooksLikeSereal

func LooksLikeSereal(b []byte) bool

LooksLikeSereal perofrms a quick and rudimentary check whether the buffer contains a Sereal document

func Marshal

func Marshal(body interface{}) ([]byte, error)

Marshal encodes body with the default encoder

func Unmarshal

func Unmarshal(b []byte, body interface{}) error

Unmarshal decodes b into body with the default decoder

Types

type Decoder

type Decoder struct {

	// PerlCompat produced a decoded data structure that preserves as much as possible the original Perl structure
	PerlCompat bool
	// DisableReferentialIntegrity produces a decoded data structure where Perl references are never decoded as Go pointers
	DisableReferentialIntegrity bool
	// contains filtered or unexported fields
}

A Decoder reads and decodes Sereal objects from an input buffer

func NewDecoder

func NewDecoder() *Decoder

NewDecoder returns a decoder with default flags

func (*Decoder) RegisterName

func (d *Decoder) RegisterName(name string, value interface{})

RegisterName registers the named class with an instance of 'value'. When the decoder finds a FREEZE tag with the given class, the binary data will be passed to value's UnmarshalBinary method.

func (*Decoder) Unmarshal

func (d *Decoder) Unmarshal(b []byte, vbody interface{}) (err error)

Unmarshal parses the Sereal-encoded buffer b and stores the result in the value pointed to by vbody

func (*Decoder) UnmarshalHeader

func (d *Decoder) UnmarshalHeader(b []byte, vheader interface{}) (err error)

UnmarshalHeader parses the Sereal-v2-encoded buffer b and stores the header data into the variable pointed to by vheader

func (*Decoder) UnmarshalHeaderBody

func (d *Decoder) UnmarshalHeaderBody(b []byte, vheader interface{}, vbody interface{}) (err error)

UnmarshalHeaderBody parses the Sereal-encoded buffer b extracts the header and body data into vheader and vbody, respectively

type Encoder

type Encoder struct {
	PerlCompat           bool       // try to mimic Perl's structure as much as possible
	Compression          compressor // optionally compress the main payload of the document using SnappyCompressor or ZlibCompressor
	CompressionThreshold int        // threshold in bytes above which compression is attempted: 1024 bytes by default
	DisableDedup         bool       // should we disable deduping of class names and hash keys
	DisableFREEZE        bool       // should we disable the FREEZE tag, which calls MarshalBinary
	ExpectedSize         uint       // give a hint to encoder about expected size of encoded data
	MaxRecursionDepth    int        // maximum recursion depth
	// contains filtered or unexported fields
}

An Encoder encodes Go data structures into Sereal byte streams

func NewEncoder

func NewEncoder() *Encoder

NewEncoder returns a new Encoder struct with default values

func NewEncoderV2

func NewEncoderV2() *Encoder

NewEncoderV2 returns a new Encoder that encodes version 2

func NewEncoderV3

func NewEncoderV3() *Encoder

NewEncoderV3 returns a new Encoder that encodes version 3

func (*Encoder) Marshal

func (e *Encoder) Marshal(body interface{}) (b []byte, err error)

Marshal returns the Sereal encoding of body

func (*Encoder) MarshalWithHeader

func (e *Encoder) MarshalWithHeader(header interface{}, body interface{}) (b []byte, err error)

MarshalWithHeader returns the Sereal encoding of body with header data

type ErrCorrupt

type ErrCorrupt struct{ Err string }

ErrCorrupt is returned if the sereal document was corrupt

func (ErrCorrupt) Error

func (c ErrCorrupt) Error() string

type Merger

type Merger struct {

	// TopLevelElement allows a user to choose what container will be used
	// at top level. Available options: array, arrayref, hash, hashref
	TopLevelElement topLevelElementType

	// optionally compress the main payload of the document using SnappyCompressor or ZlibCompressor
	// CompressionThreshold specifies threshold in bytes above which compression is attempted: 1024 bytes by default
	Compression          compressor
	CompressionThreshold int

	// If enabled, merger will deduplicate all strings it meets.
	// Otherwise, only hash key and class names will be deduplicated
	DedupeStrings bool

	// If enabled, KeepFlat keeps flat structture of final document.
	// Specifically, consider two arrays [A,B,C] and [D,E,F]:
	// - when KeepFlat == false, the result of merging is [[A,B,C],[D,E,F]]
	// - when KeepFlat == true, the result is [A,B,C,D,E,F]
	//   This mode is relevant only to top level elements
	KeepFlat bool

	// give a hint to encoder about expected size of encoded data
	ExpectedSize uint
	// contains filtered or unexported fields
}

Merger merges multiple sereal documents without reconstructing them

func NewMerger

func NewMerger() *Merger

NewMerger returns a merger using the latest sereal version

func NewMergerV2

func NewMergerV2() *Merger

NewMergerV2 returns a merger for processing sereal v2 documents

func NewMergerV3

func NewMergerV3() *Merger

NewMergerV3 returns a merger for processing sereal v3 documents

func (*Merger) Append

func (m *Merger) Append(b []byte) (int, error)

Append adds the sereal document b and returns the number of elements added to the top-level structure

func (*Merger) Finish

func (m *Merger) Finish() ([]byte, error)

Finish is called to terminate the merging process

type PerlAlias

type PerlAlias struct {
	Alias interface{}
}

PerlAlias represents an aliased value

type PerlFreeze

type PerlFreeze struct {
	Class string
	Data  []byte
}

PerlFreeze represents an object's custom Freeze implementation

type PerlObject

type PerlObject struct {
	Class     string
	Reference interface{}
}

PerlObject represents a perl blessed reference

type PerlRegexp

type PerlRegexp struct {
	Pattern   []byte
	Modifiers []byte
}

PerlRegexp represents a perl regular expression

type PerlUndef

type PerlUndef struct {
	// contains filtered or unexported fields
}

PerlUndef represents perl's "undef" value

func PerlCanonicalUndef

func PerlCanonicalUndef() *PerlUndef

PerlCanonicalUndef returns a value that represents perl's shared undef (PL_sv_undef).

For more details see https://github.com/Sereal/Sereal/blob/master/sereal_spec.pod#user-content-dealing-with-undefined-values

type PerlWeakRef

type PerlWeakRef struct {
	Reference interface{}
}

PerlWeakRef represents a weak reference

type SnappyCompressor

type SnappyCompressor struct {
	Incremental bool // enable incremental parsing
}

SnappyCompressor compresses a Sereal document using the Snappy format.

type ZlibCompressor

type ZlibCompressor struct {
	Level int // compression level, set to ZlibDefaultCompression by default
}

ZlibCompressor compresses a Sereal document using the zlib format.

type ZstdCompressor

type ZstdCompressor struct {
	Level int // compression level, set to ZstdDefaultCompression by default
}

ZstdCompressor compresses a Sereal document using the zstd format.