shardingmap

README

ShardingMap

中文

A simple and efficient sharding map (distributed hash map) implementation in Go. It allows splitting your key-value pairs across multiple shards to achieve thread-safe, high-performance access to large datasets.

Features

• Sharded: Distribute your data across multiple shards for better scalability and concurrency.
• Thread-Safe: Use of sync.RWMutex ensures concurrent reads and writes are handled safely.
• Customizable: Set custom shard count and sharding function.
• FNV Hashing: Default sharding function based on FNV-1a hash.

Installation

go get github.com/lishank0119/shardingmap

Usage

Creating a Sharded Map

package main

import (
	"fmt"
	"github.com/lishank0119/shardingmap"
)

func main() {
	// Create a new sharded map with custom options
	sm := shardingmap.New[int, string](
		shardingmap.WithShardCount[int, string](32),                                      // Set the number of shards
		shardingmap.WithShardingFunc[int, string](func(key int) int { return key }), // Custom sharding function
	)

	// Set some key-value pairs
	sm.Set(1, "value1")
	sm.Set(2, "value2")

	// Retrieve values
	if value, ok := sm.Get(1); ok {
		fmt.Println(value) // Output: value1
	}

	// Delete a key
	sm.Delete(2)

	// Iterate over all key-value pairs
	sm.ForEach(func(k int, v string) {
		fmt.Printf("%d: %s\n", k, v)
	})

	fmt.Println("Total items:", sm.Len()) // Output: Total items: 1
}

Methods

• New: Creates a new sharded map with configurable options (WithShardCount and WithShardingFunc).
• Set: Adds or updates a key-value pair in the map.
• Get: Retrieves the value associated with the key.
• Delete: Removes the key-value pair from the map.
• Len: Returns the total number of key-value pairs in the map.
• ForEach: Iterates over all key-value pairs.

Customization

You can customize the following:

• Shard Count: Define the number of shards (default is 16).
• Sharding Function: Provide a custom sharding function for more complex data distributions.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Documentation

Index

• type Option
  • func WithShardCount[K comparable, V any](count int) Option[K, V]
  • func WithShardingFunc[K comparable, V any](fn func(K) int) Option[K, V]
• type Shard
• type ShardingMap
  • func New[K comparable, V any](opts ...Option[K, V]) *ShardingMap[K, V]
  • func (sm *ShardingMap[K, V]) Delete(key K)
  • func (sm *ShardingMap[K, V]) ForEach(f func(K, V))
  • func (sm *ShardingMap[K, V]) Get(key K) (V, bool)
  • func (sm *ShardingMap[K, V]) Len() int
  • func (sm *ShardingMap[K, V]) Set(key K, value V)

Types

type Option

type Option[K comparable, V any] func(*ShardingMap[K, V])

Option defines the options for configuring the ShardingMap

func WithShardCount

func WithShardCount[K comparable, V any](count int) Option[K, V]

WithShardCount sets the number of shards

func WithShardingFunc

func WithShardingFunc[K comparable, V any](fn func(K) int) Option[K, V]

WithShardingFunc sets a custom sharding function

type Shard

type Shard[K comparable, V any] struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Shard defines the structure of each shard

type ShardingMap

type ShardingMap[K comparable, V any] struct {
	// contains filtered or unexported fields
}

ShardingMap defines the structure of the entire sharding map

func New

func New[K comparable, V any](opts ...Option[K, V]) *ShardingMap[K, V]

New creates a new ShardingMap with support for options

func (*ShardingMap[K, V]) Delete

func (sm *ShardingMap[K, V]) Delete(key K)

Delete removes the key

func (*ShardingMap[K, V]) ForEach

func (sm *ShardingMap[K, V]) ForEach(f func(K, V))

ForEach iterates over all key-value pairs

func (*ShardingMap[K, V]) Get

func (sm *ShardingMap[K, V]) Get(key K) (V, bool)

Get retrieves the value corresponding to the key

func (*ShardingMap[K, V]) Len

func (sm *ShardingMap[K, V]) Len() int

Len returns the total number of elements in the ShardingMap

func (*ShardingMap[K, V]) Set

func (sm *ShardingMap[K, V]) Set(key K, value V)

Set sets the key-value pair