cmap

README

Why this fork?

Basically, the original one is no longer maintained.

concurrent map

As explained here and here, the map type in Go doesn't support concurrent reads and writes. concurrent-map provides a high-performance solution to this by sharding the map with minimal time spent waiting for locks.

Prior to Go 1.9, there was no concurrent map implementation in the stdlib. In Go 1.9, sync.Map was introduced. The new sync.Map has a few key differences from this map. The stdlib sync.Map is designed for append-only scenarios. So if you want to use the map for something more like in-memory db, you might benefit from using our version. You can read more about it in the golang repo, for example here and here

Usage

Import the package:

import (
	"github.com/guerinoni/concurrent-map"
)

go get "github.com/guerinoni/concurrent-map"

The package is now imported under the "cmap" namespace.

Example

	// Create a new map.
	m := cmap.New[string, string]()

	// Sets item within map, sets "bar" under key "foo"
	m.Set("foo", "bar")

	// Retrieve item from map.
	bar, ok := m.Get("foo")

	// Removes item under key "foo"
	m.Remove("foo")

For more examples have a look at concurrent_map_test.go.

Running tests:

go test ./...

go test -bench=. ./...

Comparing benchmarks:

go test -bench=. -benchmem > before.txt
go test -bench=. -benchmem > after.txt

go install golang.org/x/perf/cmd/benchstat@latest

benchstat before.txt after.txt

Documentation

Index

• Variables
• type ConcurrentMap
  • func New[K comparable, V any]() *ConcurrentMap[K, V]
  • func NewWithCustomShardingFunction[K comparable, V any](sharding func(key K) uintptr) ConcurrentMap[K, V]
  • func (m ConcurrentMap[K, V]) Clear()
  • func (m ConcurrentMap[K, V]) Count() int
  • func (m ConcurrentMap[K, V]) Get(key K) (V, bool)
  • func (m ConcurrentMap[K, V]) GetShard(key K) *ConcurrentMapShared[K, V]
  • func (m ConcurrentMap[K, V]) Has(key K) bool
  • func (m ConcurrentMap[K, V]) IsEmpty() bool
  • func (m ConcurrentMap[K, V]) Items() map[K]V
  • func (m ConcurrentMap[K, V]) IterBuffered() <-chan Tuple[K, V]
  • func (m ConcurrentMap[K, V]) IterCb(fn IterCb[K, V])
  • func (m ConcurrentMap[K, V]) Keys() []K
  • func (m ConcurrentMap[K, V]) MSet(data map[K]V)
  • func (m ConcurrentMap[K, V]) MarshalJSON() ([]byte, error)
  • func (m ConcurrentMap[K, V]) Pop(key K) (v V, exists bool)
  • func (m ConcurrentMap[K, V]) Remove(key K)
  • func (m ConcurrentMap[K, V]) RemoveCb(key K, cb RemoveCb[K, V]) bool
  • func (m ConcurrentMap[K, V]) Set(key K, value V)
  • func (m ConcurrentMap[K, V]) SetIfAbsent(key K, value V) bool
  • func (m *ConcurrentMap[K, V]) UnmarshalJSON(b []byte) (err error)
  • func (m ConcurrentMap[K, V]) Upsert(key K, value V, cb UpsertCb[K, V]) (res V)
• type ConcurrentMapShared
• type IterCb
• type RemoveCb
• type Tuple
• type UpsertCb

Variables

var SHARD_COUNT = runtime.GOMAXPROCS(-1)

Types

type ConcurrentMap

type ConcurrentMap[K comparable, V any] struct {
	// contains filtered or unexported fields
}

ConcurrentMap is "thread" safe map for generic types. To avoid lock bottlenecks this map is dived to several (SHARD_COUNT) map shards.

func New

func New[K comparable, V any]() *ConcurrentMap[K, V]

New creates a new concurrent map with a generic key type.

func NewWithCustomShardingFunction

func NewWithCustomShardingFunction[K comparable, V any](sharding func(key K) uintptr) ConcurrentMap[K, V]

NewWithCustomShardingFunction creates a new concurrent map with a generic key type and a custom sharding function.

func (ConcurrentMap[K, V]) Clear

func (m ConcurrentMap[K, V]) Clear()

Clear removes all items from map.

func (ConcurrentMap[K, V]) Count

func (m ConcurrentMap[K, V]) Count() int

Count returns the number of elements within the map.

func (ConcurrentMap[K, V]) Get

func (m ConcurrentMap[K, V]) Get(key K) (V, bool)

Get retrieves an element from map under given key.

func (ConcurrentMap[K, V]) GetShard

func (m ConcurrentMap[K, V]) GetShard(key K) *ConcurrentMapShared[K, V]

GetShard returns shard under given key.

func (ConcurrentMap[K, V]) Has

func (m ConcurrentMap[K, V]) Has(key K) bool

Looks up an item under specified key.

func (ConcurrentMap[K, V]) IsEmpty

func (m ConcurrentMap[K, V]) IsEmpty() bool

IsEmpty checks if map is empty.

func (ConcurrentMap[K, V]) Items

func (m ConcurrentMap[K, V]) Items() map[K]V

Items returns all items as map[string]V.

func (ConcurrentMap[K, V]) IterBuffered

func (m ConcurrentMap[K, V]) IterBuffered() <-chan Tuple[K, V]

IterBuffered returns a buffered iterator which could be used in a for range loop.

func (ConcurrentMap[K, V]) IterCb

func (m ConcurrentMap[K, V]) IterCb(fn IterCb[K, V])

Callback based iterator, cheapest way to read all elements in a map.

func (ConcurrentMap[K, V]) Keys

func (m ConcurrentMap[K, V]) Keys() []K

Keys returns all keys as []string.

func (ConcurrentMap[K, V]) MSet

func (m ConcurrentMap[K, V]) MSet(data map[K]V)

func (ConcurrentMap[K, V]) MarshalJSON

func (m ConcurrentMap[K, V]) MarshalJSON() ([]byte, error)

Reviles ConcurrentMap "private" variables to json marshal.

func (ConcurrentMap[K, V]) Pop

func (m ConcurrentMap[K, V]) Pop(key K) (v V, exists bool)

Pop removes an element from the map and returns it.

func (ConcurrentMap[K, V]) Remove

func (m ConcurrentMap[K, V]) Remove(key K)

Remove removes an element from the map.

func (ConcurrentMap[K, V]) RemoveCb

func (m ConcurrentMap[K, V]) RemoveCb(key K, cb RemoveCb[K, V]) bool

RemoveCb locks the shard containing the key, retrieves its current value and calls the callback with those params If callback returns true and element exists, it will remove it from the map Returns the value returned by the callback (even if element was not present in the map).

func (ConcurrentMap[K, V]) Set

func (m ConcurrentMap[K, V]) Set(key K, value V)

Sets the given value under the specified key.

func (ConcurrentMap[K, V]) SetIfAbsent

func (m ConcurrentMap[K, V]) SetIfAbsent(key K, value V) bool

Sets the given value under the specified key if no value was associated with it.

func (*ConcurrentMap[K, V]) UnmarshalJSON

func (m *ConcurrentMap[K, V]) UnmarshalJSON(b []byte) (err error)

Reverse process of Marshal.

func (ConcurrentMap[K, V]) Upsert

func (m ConcurrentMap[K, V]) Upsert(key K, value V, cb UpsertCb[K, V]) (res V)

Insert or Update - updates existing element or inserts a new one using UpsertCb.

type ConcurrentMapShared

type ConcurrentMapShared[K comparable, V any] struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

type IterCb

type IterCb[K comparable, V any] func(key K, v V)

Iterator callbacalled for every key,value found in maps. RLock is held for all calls for a given shard therefore callback sess consistent view of a shard, but not across the shards.

type RemoveCb

type RemoveCb[K comparable, V any] func(key K, v V, exists bool) bool

RemoveCb is a callback executed in a map.RemoveCb() call, while Lock is held If returns true, the element will be removed from the map.

type Tuple

type Tuple[K comparable, V any] struct {
	Key K
	Val V
}

Used by the Iter & IterBuffered functions to wrap two variables together over a channel,.

type UpsertCb

type UpsertCb[K comparable, V any] func(exist bool, valueInMap V, newValue V) V

Callback to return new element to be inserted into the map It is called while lock is held, therefore it MUST NOT try to access other keys in same map, as it can lead to deadlock since Go sync.RWLock is not reentrant.