mapstore

README

MapStore

Overview

MapStore is a simple API built on top of Kubernetes ConfigMaps that allows you to programmatically read, write and delete key value pairs. This package was created as a solution to store small amounts of data without needing a persistent volume or standalone database. Because this package is backed by a ConfigMap, we must remember that potentially every interaction will send API requests to the Kubernetes API server.

To get started, check out the examples page.

Caveats

Kubernetes can not guarantee exclusive access to a ConfigMap, so we need to be aware of some edge cases. The ideal usage of MapStore is to have a single process accessing the data to ensure no inconsistences. Using a leader election package to protect access is recommended.

Internal caching

MapStore has the ability to hold the data of the ConfigMap in memory for quick lookups and reducing unnecessary requests to the Kubernetes API. This should only be enabled when you can guarantee no other app or process is accessing the same ConfigMap.

cacheConfigMapInternally := true
mapStore, err := mapstore.New("my-test-cm", cacheConfigMapInternally)

Size limitations

Please be aware that ConfigMaps are limited in size. This package has no protective measures in place to ensure you are below the limit.

A ConfigMap is not designed to hold large chunks of data. The data stored in a ConfigMap cannot exceed 1 MiB. If you need to store settings that are larger than this limit, you may want to consider mounting a volume or use a separate database or file service.

Kubernetes ConfigMap documentation

Environment variables

There are a few environment variables that you can apply to your workload that will effect MapStore:

MAPSTORE_CLUSTER_CONFIG_PATH can be set if you are using this package outside of your cluster, but still want to interact with a ConfigMap on the cluster. You can define the path to your cluster config file and it will be used by MapStore when connecting to the cluster. This is also a handy variable when testing locally. By default this value is empty and MapStore uses the InClusterConfig for it's connection.

NAMESPACE is the namespace name that MapStore will use. If not set, MapStore will attempt to pull the current namespace from /var/run/secrets/kubernetes.io/serviceaccount/namespace.

Documentation

Overview

Package mapstore facilitates saving key value pairs into a Kubernetes ConfigMap.

package main

import (
    "fmt"
    "log"

    "github.com/unrolled/mapstore"
)

func main() {
    mapStore, err := mapstore.New("my-custom-config-map-name", false)
    if err != nil {
        log.Fatalf("error creating mapstore (possible rbac issue?): %v", err)
    }

    err = mapStore.Set("my-key", []byte("my value lives here"))
    if err != nil {
        log.Fatalf("error setting value: %v", err)
    }

    val, err := mapStore.Get("my-key")
    if err != nil {
        log.Fatalf("error getting value: %v", err)
    }

    fmt.Printf("Value from ConfigMap: %#v\n", val)
}

Index

• Variables
• func VerifyConnection(testMapNamespace, testMapName string) error
• type AdvancedInterface
• type Interface
• type Manager
  • func New(cmNamespace, cmName string, cacheInternally bool) (*Manager, error)
  • func (k *Manager) Delete(key string) error
  • func (k *Manager) ForceSet(key string, value []byte) error
  • func (k *Manager) Get(key string) ([]byte, error)
  • func (k *Manager) Keys() ([]string, error)
  • func (k *Manager) Raw() (map[string][]byte, error)
  • func (k *Manager) Set(key string, value []byte) error
  • func (k *Manager) Truncate() error

Variables

var (
	ErrNoValueFound = errors.New("no value found in env")
	ErrDataMismatch = errors.New("data is mismatched")
	ErrKeyNotFound  = errors.New("key was not found")
)

Functions

func VerifyConnection

func VerifyConnection(testMapNamespace, testMapName string) error

VerifyConnection is a helper function that creates a temporary ConfigMap to ensure cluster connectivity and RBAC settings.

Types

type AdvancedInterface

type AdvancedInterface interface {
	Interface
	Raw() (map[string][]byte, error)
	ForceSet(key string, value []byte) error
}

AdvancedInterface defines the required methods and a few optional methods for the Manager implementation.

type Interface

type Interface interface {
	Keys() ([]string, error)
	Get(key string) ([]byte, error)
	Set(key string, value []byte) error
	Delete(key string) error
	Truncate() error
}

Interface defines the required methods to satisfy the Manager implementation.

type Manager

type Manager struct {
	*sync.RWMutex
	// contains filtered or unexported fields
}

Manager is a thread safe key value store backed by a Kubernetes ConfigMap.

func New

func New(cmNamespace, cmName string, cacheInternally bool) (*Manager, error)

New returns a newly setup Manager instance.

func (*Manager) Delete

func (k *Manager) Delete(key string) error

Delete removes the given key from the underlying ConfigMap.

func (*Manager) ForceSet

func (k *Manager) ForceSet(key string, value []byte) error

ForceSet is the same as Set, but does not check if the values are equal first.

func (*Manager) Get

func (k *Manager) Get(key string) ([]byte, error)

Get uses the supplied key and attempts to return the corresponding value from the ConfigMap.

func (*Manager) Keys

func (k *Manager) Keys() ([]string, error)

Keys returns all the key names from the ConfigMap.

func (*Manager) Raw

func (k *Manager) Raw() (map[string][]byte, error)

Raw returns the actual underlying map data.

func (*Manager) Set

func (k *Manager) Set(key string, value []byte) error

Set checks if the value has changed before performing the underlying save call.

func (*Manager) Truncate

func (k *Manager) Truncate() error

Truncate removes all the data from the underlying ConfigMap.