model

Documentation

Index

• Constants
• Variables
• type ObjectDecoder
  • func NewObjectDecoder(dataEncoding string) (ObjectDecoder, error)
• type SegmentDecoder
  • func MustNewSegmentDecoder(dataEncoding string) SegmentDecoder
  • func NewSegmentDecoder(dataEncoding string) (SegmentDecoder, error)

Constants

const CurrentEncoding = v1.Encoding

CurrentEncoding is a string representing the encoding that all new blocks should be created with

Variables

var AllEncodings = []string{
	v1.Encoding,
}

AllEncodings is used for testing

Types

type ObjectDecoder

type ObjectDecoder interface {
	// PrepareForRead converts the byte slice into a deepdb.Snapshot for reading. This can be very expensive
	//  and should only be used when surfacing a byte slice from deepdb and preparing it for reads.
	PrepareForRead(obj []byte) (*deeppb_tp.Snapshot, error)

	// FastRange returns the start and end unix epoch timestamp of the snapshot. If it's not possible to efficiently get these
	// values from the underlying encoding then it should return decoder.ErrUnsupported
	FastRange(obj []byte) (uint32, error)
}

ObjectDecoder is used to work with opaque byte slices that contain snapshot data in the backend

func NewObjectDecoder

func NewObjectDecoder(dataEncoding string) (ObjectDecoder, error)

NewObjectDecoder returns a Decoder given the passed string.

type SegmentDecoder

type SegmentDecoder interface {
	// PrepareForWrite takes a snapshot pointer and returns a record prepared for writing to an ingester
	PrepareForWrite(snapshot *deeppb_tp.Snapshot, start uint32) ([]byte, error)
	// PrepareForRead converts a set of segments created using PrepareForWrite. These segments
	//  are converted into a deepTP.Snapshot. This operation can be quite costly and should be called only for reading
	PrepareForRead(segment []byte) (*deeppb_tp.Snapshot, error)
	// ToObject converts a set of segments into an object ready to be written to the deepdb backend.
	//  The resultant byte slice can then be manipulated using the corresponding ObjectDecoder.
	//  ToObject is on the write path and should do as little as possible.
	ToObject(segments []byte) ([]byte, error)
	// FastRange returns the start and end unix epoch timestamp of the provided segment. If its not possible to efficiently get these
	// values from the underlying encoding then it should return decoder.ErrUnsupported
	FastRange(segment []byte) (uint32, error)
}

SegmentDecoder is used by the distributor/ingester to aggregate and pass segments of snapshots. The distributor creates the segments using PrepareForWrite which can then be consumed and organized by snapshotid in the ingester.

The ingester then holds these in memory until either:

• The snapshot id is queried. In this case it uses PrepareForRead to turn the segments into a deepTP.Snapshot for return on the query path.
• It needs to push them into deepdb. For this it uses ToObject() to create a single byte slice from the segments that is then completely handled by an ObjectDecoder of the same version

func MustNewSegmentDecoder

func MustNewSegmentDecoder(dataEncoding string) SegmentDecoder

MustNewSegmentDecoder creates a new encoding or it panics

func NewSegmentDecoder

func NewSegmentDecoder(dataEncoding string) (SegmentDecoder, error)

NewSegmentDecoder returns a Decoder given the passed string.