Documentation ¶
Index ¶
- Variables
- type CollectTable
- type InspectCallback
- type Inspectable
- type KeyHashFunc
- type MemTable
- func (tbl *MemTable) Emit(kv saw.Datum) error
- func (tbl *MemTable) Inspect(key saw.DatumKey, callback InspectCallback) (int, error)
- func (tbl *MemTable) InspectAll(callback InspectCallback, concurrent bool) (int, error)
- func (tbl *MemTable) InspectSet(keys []saw.DatumKey, callback InspectCallback, concurrent bool) (int, error)
- func (tbl *MemTable) Result(ctx context.Context) (interface{}, error)
- type SimpleTable
- func (tbl *SimpleTable) Emit(kv saw.Datum) (err error)
- func (tbl *SimpleTable) Inspect(key saw.DatumKey, callback InspectCallback) (int, error)
- func (tbl *SimpleTable) InspectAll(callback InspectCallback, concurrent bool) (int, error)
- func (tbl *SimpleTable) InspectSet(keys []saw.DatumKey, callback InspectCallback, concurrent bool) (int, error)
- func (tbl *SimpleTable) Result(ctx context.Context) (interface{}, error)
- type TableItemFactory
- type TableResultMap
- type TableSpec
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidTableSpec = errors.New("saw.table: invalid table spec")
Functions ¶
This section is empty.
Types ¶
type CollectTable ¶
type CollectTable struct {
// contains filtered or unexported fields
}
Collect is a special table that it doesn't do any computation, but simply stores everything it receives.
func NewCollectTable ¶
func NewCollectTable(ctx context.Context, spec TableSpec) (table *CollectTable, err error)
Creates a new CollectTable, returns error when underling DatumWriter creation fails.
TableItemFactory and NumShards in TableSpec is no-op, # shards outputed is soley determined by PersistentResource.NumShards
type Inspectable ¶
type Inspectable interface { Inspect(key saw.DatumKey, callback InspectCallback) (inspected int, err error) InspectSet(keys []saw.DatumKey, callback InspectCallback, concurrent bool) (inspected int, err error) InspectAll(callback InspectCallback, concurrent bool) (inspected int, err error) }
Inspectable tables allows a callback to inspect one, a set of, or all saws in the table, table gurantees no concurrent Emit() for the saw being inspected, but other items not being currently being inspected could still receive Emit() concurrently, that means you cannot get a consistent snapshot of table using InspectAll().
InspectSet() and InspectAll() can inspect items in parallel when concurrent=true. All inspect functions return number of items been inspected without error, and one of the error callback returned if there's any, inspect functions stops as soon as it encounter error.
type KeyHashFunc ¶
type MemTable ¶
type MemTable struct {
// contains filtered or unexported fields
}
MemTable manages a set (spec.NumShards) of SimpleTables, provides concurrent safe Emit(), stores finaly result when Result() called if there is a spec.PersistentResource setting.
func NewMemTable ¶
func (*MemTable) InspectAll ¶
func (tbl *MemTable) InspectAll(callback InspectCallback, concurrent bool) (int, error)
func (*MemTable) InspectSet ¶
func (*MemTable) Result ¶
Returns TableResultMap, each item as Result() of item saw. nil item results are ignored.
When error presents in individual items Result(), it still tries to get results of all others, then a partial result and one of the item result error will be returned.
When tbl.spec.PersistentResource set, results will be write to persistent store, all items' Result() will still be called when persistent fails.
type SimpleTable ¶
type SimpleTable struct {
// contains filtered or unexported fields
}
SimpleTable is a in-memory, non-storable memory table, concurrent non-safe table --- it assumes you call all its methods sequentially. Good for handling small set of data in mini-batch --- aggregate stats for a single user session etc.
func NewSimpleTable ¶
func NewSimpleTable(spec TableSpec) *SimpleTable
func (*SimpleTable) Inspect ¶
func (tbl *SimpleTable) Inspect(key saw.DatumKey, callback InspectCallback) (int, error)
func (*SimpleTable) InspectAll ¶
func (tbl *SimpleTable) InspectAll(callback InspectCallback, concurrent bool) (int, error)
func (*SimpleTable) InspectSet ¶
func (tbl *SimpleTable) InspectSet( keys []saw.DatumKey, callback InspectCallback, concurrent bool) (int, error)
func (*SimpleTable) Result ¶
func (tbl *SimpleTable) Result(ctx context.Context) (interface{}, error)
Returns TableResultMap, each item as Result() of item saw. nil item results are ignored.
When error presents in individual items Result(), it still tries to get results of all others, then a partial result and one of the item result error will be returned.
type TableItemFactory ¶
func ItemFactoryOf ¶
func ItemFactoryOf(example saw.Saw) TableItemFactory
A simple item factory that creates zero value (not copy!) instance of saw type in paramter, panic if saw is not pointer receiver.
type TableResultMap ¶
SimpleTable and MemTable result type
type TableSpec ¶
type TableSpec struct { Name string ItemFactory TableItemFactory // KeyHashFunc assigns incoming Datum to one of its shard, defaults to fnv32 KeyHashFunc KeyHashFunc // Defaults to 127 NumShards int // When not empty, table state will be stored at external storage. PersistentResource storage.ResourceSpec // It depends on table type to determine what data get persistent and what // encoder to use. Defaults to verbatim (accepts and stores []byte) ValueEncoder saw.ValueEncoder // Implementation may pre-allocate and reuse buffer for encoding values, to avoid // frequent malloc, defaults to 4096 ValueEncodeBufferSize int }
TableSpec is shared configuration of Table implementations in this package.