sync

README

Data synchronizer

Data Synchronizer updates application objects from a storage. A process checks if underlying storage object has been modified and if so it starts synchronization.

Large amount of data synchronization be both memory and CPU extensive, causing application performance hit got short duration. In case where only small amount of actual changes are submitted this package allows to selectively applies these changes with reusing non mutated state.

To allow selective changes, snapshoter and keyer has to be implemented to retrieve a record key from byte array. Make sure this function is substantially cheaper than the record decoding function.

Documentation

Index

• func CompositeKey(keys ...string) func(data []byte) (interface{}, error)
• func Hash(data []byte) complex128
• func IntKeyJSONExtractor(key string) func(data []byte) (interface{}, error)
• func StringKeyJSONExtractor(key string) func(data []byte) (interface{}, error)
• type CSVUnmarshaler
• type Checksumer
• type Handler
• type Keyer
• type Provider
• type Service
  • func New(fsOptions ...storage.Option) *Service
  • func (s *Service) Sync(ctx context.Context, sync *Synchronization) error
• type Snapshoter
• type Synchronization

Functions

func CompositeKey

func CompositeKey(keys ...string) func(data []byte) (interface{}, error)

func Hash

func Hash(data []byte) complex128

func IntKeyJSONExtractor

func IntKeyJSONExtractor(key string) func(data []byte) (interface{}, error)

IntKeyJSONExtractor returns function returning int key value

func StringKeyJSONExtractor

func StringKeyJSONExtractor(key string) func(data []byte) (interface{}, error)

StringKeyJSONExtractor returns function returning string key value

Types

type CSVUnmarshaler

type CSVUnmarshaler interface {
	UnmarshalCSV(text string) error
	SkipHeader() bool
}

CSVUnmarshaler represents CSV Unmarshaler

type Checksumer

type Checksumer func([]byte) complex128

Checksumer returns a record checksum

type Handler

type Handler func(target interface{}) (bool, error)

type Keyer

type Keyer func(encoded []byte) (interface{}, error)

Keyer returns an item key from byte steam

type Provider

type Provider func() interface{}

type Service

type Service struct {
	// contains filtered or unexported fields
}

func New

func New(fsOptions ...storage.Option) *Service

New creates a sync service

func (*Service) Sync

func (s *Service) Sync(ctx context.Context, sync *Synchronization) error

type Snapshoter

type Snapshoter func(key interface{}, targetPtr interface{}) bool

Snapshoter returns previous value into target key and return true, otherwise false

type Synchronization

type Synchronization struct {
	*resource.Asset
	Provider Provider
	Handler  Handler
	Keyer    Keyer
	Type     reflect.Type
	Snapshoter
	Checksumer Checksumer
	// contains filtered or unexported fields
}