idxmap

README

Concept

The idxmap package provides an enhanced mapping structure to help in the following use cases:

• Exposing read-only access to plugin local data for other plugins
• Secondary indexing
• Data caching for key-value store (such as etcd)

For more detailed description see the godoc.

Exposing plugin local information Use Case

App plugins often need to expose some structured information to other plugins inside the agent (see the following diagram).

Structured data stored in idxmap are available for (possibly concurrent) read access to other plugins:

1. either via lookup using primary keys or secondary indices;
2. or via watching for data changes in the map using channels or callbacks (subscribe for changes and receive notification once an item is added, changed or removed).

TODO: remove "<<cache>>" string from the image

Caching Use Case

It is useful to have the data from a key-value store cached when you need to:

• minimize the number of lookups into the key-value store
• execute lookups by secondary indexes for key-value stores that do not necessarily support secondary indexing (e.g. etcd)

CacheHelper turns idxmap (injected as field IDX) into an indexed local copy of remotely stored key-value data. CacheHelper watches the target key-value store for data changes and resync events. Received key-value pairs are transformed into the name-value (+ secondary indices if defined) pairs and stored into the injected idxmap instance. For a visual explanation, see the diagram below:

Packages that use index map. The constructor that combines CacheHelper with idxmap to build the cache from the example can be found there as well.

Examples

• Isolated and simplified examples can be found here:
  • lookup
  • watch

Documentation

Overview

Package idxmap defines a mapping structure which supports mapping change notifications and retrieval of items by fields in the value structure.

Primary Index                Item                                Secondary indexes

===================================================================================

Eth1              +---------------------+                 { "IP" : ["192.168.2.1", "10.0.0.8"],
                  |  Status: Enabled    |                   "Type" : ["ethernet"]
                  |  IP: 192.168.2.1    |                 }
                  |      10.0.0.8       |
                  |  Type: ethernet     |
                  |  Desc: something    |
                  +---------------------+

Function `Put` adds a value (item) into the mapping. In the function call the primary index(name) for the item is specified. The values of the primary index are unique, if the name already exists, then the item is overwritten. To retrieve an item identified by the primary index, use the `GetValue` function. An item can be removed from the mapping by calling the `Delete` function. The names that are currently registered can be retrieved by calling the `ListAllNames` function.

The constructor allows you to define a `createIndexes` function that extracts secondary indices from stored items. The function returns a map indexed by names of secondary indexes, and the values are the extracted values for the particular item. The values of secondary indexes are not necessarily unique. To retrieve items based on secondary indices use the `ListNames` function. In contrast to the lookup by primary index, the function may return multiple names.

`Watch` allows to define a callback that is called when a change in the mapping occurs. There is a helper function `ToChan` available, which allows to deliver notifications through a channel.

Index

• Constants
• func ToChan(ch chan NamedMappingGenericEvent, opts ...interface{}) func(dto NamedMappingGenericEvent)
• type NamedMapping
• type NamedMappingEvent
• type NamedMappingGenericEvent
• type NamedMappingRW

Constants

const DefaultNotifTimeout = 2 * time.Second

DefaultNotifTimeout for delivery of notification

Functions

func ToChan

func ToChan(ch chan NamedMappingGenericEvent, opts ...interface{}) func(dto NamedMappingGenericEvent)

ToChan creates a callback that can be passed to the Watch function in order to receive notifications through a channel. If the notification can not be delivered until timeout, it is dropped.

Types

type NamedMapping

type NamedMapping interface {
	// GetRegistryTitle returns the title assigned to the registry.
	GetRegistryTitle() string

	// GetValue retrieves a previously stored item identified by
	// <name>. If there is no item associated with the give name in the mapping,
	// the <exists> flag is returned as *false*.
	GetValue(name string) (value interface{}, exists bool)

	// ListNames looks up the items by a secondary index.
	// It returns the names of all indexes for which the value of a secondary
	// key <field> equals to <value>.
	ListNames(field string, value string) (names []string)

	// ListAllNames returns all names in the mapping.
	ListAllNames() (names []string)

	// Watch subscribes to receive notifications about the changes in the
	// mapping. To receive changes through a channel, ToChan utility can be used.
	//
	// Example usage:
	//
	//  map.Watch(plugin.PluginName, ToChan(myChannel))
	//
	//  map.Watch(plugin.PluginName, func(msgNamedMappingGenericEvent) {/*handle callback*/ return nil})
	//
	Watch(subscriber infra.PluginName, callback func(NamedMappingGenericEvent)) error
}

NamedMapping is the "user API" to the mapping. It provides read-only access.

type NamedMappingEvent

type NamedMappingEvent struct {
	// Logical name of the object
	Name string
	// Del denotes a type of change
	// - it is true if an item was removed
	// - it false if an item was added or updated
	Del bool
	// Update denotes a type of change
	// - it is true if and item metadata was updated
	// - it is false if an item was added or removed
	Update bool
	// RegistryTitle identifies the registry (NameToIndexMapping)
	RegistryTitle string
}

NamedMappingEvent is a part of the change notification. It is a generic part that does not contain metadata of type interface{} thus it can be reused in mapping with typed metadata.

type NamedMappingGenericEvent

type NamedMappingGenericEvent struct {
	NamedMappingEvent

	Value interface{}
}

NamedMappingGenericEvent represents a single change in the mapping. The structure is created when an item is inserted or removed from the mapping.

type NamedMappingRW

type NamedMappingRW interface {
	NamedMapping

	// Put registers a new item into the mapping under the given <name>.
	// Name is the primary unique key, if an item was registered before
	// it is overwritten.
	Put(name string, value interface{})

	// Update replaces a metadata value in an existing mapping under
	// provided <name>. If item is missing, it is NOT created
	// and false value is returned in such a case.
	Update(name string, value interface{}) (success bool)

	// Delete removes an item associated with the <name> from the mapping.
	Delete(name string) (value interface{}, exists bool)

	// Clear removes all entries from the mapping
	Clear()
}

NamedMappingRW is the "owner API" to the mapping. Using this API the owner can modify the content of the mapping.