ttl

README

Go Time-To-Live Containers

Introduction

ttl is golang package that implements time-to-live container types such that after a given amount of time, items in the map are deleted.

• The map key can be any comparable data type, via Generics.
• Any data type can be used as a map value, via Generics.

This is a fork of the awesome github.com/jftuga/TtlMap with a few enhancements and creature-comforts:

• Map is a generic, homogeneous map type
  • Meaning that both key and value are determined by Generics
  • Using any or interface{} as the value type will effectively emulate the original source package
• Map can be created with a context.Context
  • Map will automatically stop pruning expired items (equivalent to Map.Close()) if the context cancels to prevent goroutine leaks
  • Great for services
• The package name is simply ttl, in case other TTL-enabled types seem like a good idea
  • For example: a slice implementation
• The syntax is a little more idiomatic
• Methods have been renamed to be more familiar to Go standard library users
  • Load() and Store() instead of Get() and Set()
• Key/value pairs can use the default TTL for the Map, or have their own individual TTL by using StoreWithTTL
• Code is a little safer for concurrent use (at the time of the fork) and more performant in that use case
  • Use of sync.RWLock so that read-heavy applications block less
  • Use of atomic.Int64 for the timestamp so that it may be updated without a write lock
  • Addition of ttl.Map.Range() in place of the All() method
    • Modelled after sync.Map.Range()
    • Safe for concurrent access
• Replace internal time.Tick() with a time.Ticker to prevent leakage

Requirements

ttl requires Go v1.21.

Installation

To use this module in your project, run:

go get -u github.com/glenvan/ttl/v2

... and import the ttl package into your Go code using:

package main

import (
	"github.com/glenvan/ttl/v2"
)

func example() {
	tm := ttl.NewMap(...)
}

License

This project is licensed under the terms of the MIT License. It derives from previous work, also licensed under the terms of the MIT License.

Example

Full example using many data types

Small example:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/glenvan/ttl/v2"
)

func main() {
	defaultTTL := 300 * time.Millisecond    // a key's time to live
	startSize := 3                          // initial number of items in map
	pruneInterval := 100 * time.Millisecond // prune expired items each time pruneInterval elapses
	refreshOnLoad := true                   // update item's 'lastAccessTime' on ttl.Map.Load()

	// Any comparable data type such as int, uint64, pointers and struct types (if all field
	// types are comparable) can be used as the key type
	t := ttl.NewMap[string, string](defaultTTL, startSize, pruneInterval, refreshOnLoad)
	defer t.Close()

	// Populate the ttl.Map
	t.Store("hello", "world")
	t.Store("goodbye", "universe")

	fmt.Printf("ttl.Map length: %d\n", t.Length())

	t.Delete("goodbye")

	// Display all items in ttl.Map
	t.Range(func(key string, value string) bool {
		fmt.Printf("[%7s] '%v'\n", key, value)
		return true
	})

	sleepTime := defaultTTL + pruneInterval
	fmt.Printf("Sleeping %s, items should be expired and removed afterward\n", sleepTime)

	time.Sleep(sleepTime)

	v, ok := t.Load("hello")
	fmt.Printf("[%7s] '%v' (exists: %t)\n", "hello", v, ok)

	v, ok = t.Load("goodbye")
	fmt.Printf("[%7s] '%v' (exists: %t)\n", "goodbye", v, ok)

	fmt.Printf("ttl.Map length: %d\n", t.Length())
}

Output:

$ go run small.go

ttl.Map length: 2
[  hello] 'world'
Sleeping 400ms, items should be expired and removed afterward
[  hello] '' (exists: false)
[goodbye] '' (exists: false)
ttl.Map length: 0

API

See the package documentation.

Acknowledgments

As mentioned, this is a fork of the awesome github.com/jftuga/TtlMap package. All ideas in this derivative package flowed from that one.

Original Package Acknowledgements

• Adopted from: Map with TTL option in Go
  • Answer created by: OneOfOne
• /u/skeeto: suggestions for the New function
• @zhayes on the Go Discord: helping the original author with Go Generics

Disclosure Notification

This program was completely developed on my own personal time, for my own personal benefit, and on my personally owned equipment.

Documentation

Overview

Package ttl is set of "time-to-live" container type such that after a given amount of time, items in the container are deleted.

Index

• type Map
  • func NewMap[K comparable, V any](defaultTTL time.Duration, length int, pruneInterval time.Duration, ...) (m *Map[K, V])
  • func NewMapContext[K comparable, V any](ctx context.Context, defaultTTL time.Duration, length int, ...) (m *Map[K, V])
  • func (m *Map[K, V]) Clear()
  • func (m *Map[K, V]) Close()
  • func (m *Map[K, V]) Delete(key K)
  • func (m *Map[K, V]) DeleteFunc(del func(key K, value V) bool)
  • func (m *Map[K, V]) Length() int
  • func (m *Map[K, V]) Load(key K) (value V, ok bool)
  • func (m *Map[K, V]) LoadPassive(key K) (value V, ok bool)
  • func (m *Map[K, V]) Range(f func(key K, value V) bool)
  • func (m *Map[K, V]) Store(key K, value V)
  • func (m *Map[K, V]) StoreWithTTL(key K, value V, TTL time.Duration)

Examples

• Map
• Map.DeleteFunc
• Map.Load
• Map.Range

Types

type Map

type Map[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Map is a "time-to-live" map such that after a given amount of time, items in the map are deleted. Map is safe for concurrent use.

When a Map.Load or Map.Store occurs, the lastAccess time is set to the current time. Therefore, only items that are not called by Map.Load or Map.Store will be deleted after the TTL expires.

Map.LoadPassive can be used in which case the lastAccess time will *not* be updated.

Adapted from: https://stackoverflow.com/a/25487392/452281

Example

defaultTTL := 300 * time.Millisecond    // the default lifetime for a key/value pair
startSize := 3                          // initial number of items in map
pruneInterval := 100 * time.Millisecond // prune expired items each time pruneInterval elapses
refreshOnLoad := true                   // update item's 'lastAccessTime' on ttl.Map.Load()

// Any comparable data type such as int, uint64, pointers and struct types (if all field
// types are comparable) can be used as the key type
t := ttl.NewMap[string, string](defaultTTL, startSize, pruneInterval, refreshOnLoad)
defer t.Close()

// Populate the ttl.Map
t.Store("hello", "world")
t.Store("goodbye", "universe")

fmt.Printf("ttl.Map length: %d\n", t.Length())

t.Delete("goodbye")

// Display all items in ttl.Map
t.Range(func(key string, value string) bool {
	fmt.Printf("[%7s] '%v'\n", key, value)
	return true
})

sleepTime := defaultTTL + pruneInterval
fmt.Printf("Sleeping %s, items should be expired and removed afterward\n", sleepTime)

time.Sleep(sleepTime)

v, ok := t.Load("hello")
fmt.Printf("[%7s] '%v' (exists: %t)\n", "hello", v, ok)

v, ok = t.Load("goodbye")
fmt.Printf("[%7s] '%v' (exists: %t)\n", "goodbye", v, ok)

fmt.Printf("ttl.Map length: %d\n", t.Length())

Output:

ttl.Map length: 2
[  hello] 'world'
Sleeping 400ms, items should be expired and removed afterward
[  hello] '' (exists: false)
[goodbye] '' (exists: false)
ttl.Map length: 0

func NewMap

func NewMap[K comparable, V any](
	defaultTTL time.Duration,
	length int,
	pruneInterval time.Duration,
	refreshOnLoad bool,
) (m *Map[K, V])

NewMap returns a new Map with items expiring according to the defaultTTL specified if they have not been accessed within that duration. Access refresh can be overridden so that items expire after the TTL whether they have been accessed or not.

Map objects returned by NewMap must be closed with Map.Close when they're no longer needed.

func NewMapContext

func NewMapContext[K comparable, V any](
	ctx context.Context,
	defaultTTL time.Duration,
	length int,
	pruneInterval time.Duration,
	refreshOnLoad bool,
) (m *Map[K, V])

NewMapContext returns a new Map with items expiring according to the defaultTTL specified if they have not been accessed within that duration. Access refresh can be overridden so that items expire after the TTL whether they have been accessed or not.

NewMapContext accepts a context. If the context is cancelled, the pruning process will automatically stop whether you've called Map.Close or not. It's safe to use either approach.

context.Background() is perfectly acceptable as the default context, however you should Map.Close the Map yourself in that case.

Map objects returned by NewMapContext may still be closed with Map.Close when they're no longer needed or if the cancellation of the context is not guaranteed.

func (*Map[K, V]) Clear

func (m *Map[K, V]) Clear()

Clear will remove all key/value pairs from the Map. Clear is safe for concurrent use.

func (*Map[K, V]) Close

func (m *Map[K, V]) Close()

Close will terminate TTL pruning of the Map. If Close is not called on a Map after it's no longer needed, the Map will leak (unless the context has been cancelled).

Close may be called multiple times and is safe to call even if the context has been cancelled.

func (*Map[K, V]) Delete

func (m *Map[K, V]) Delete(key K)

Delete will remove a key and its value from the Map. Delete is safe for concurrent use.

func (*Map[K, V]) DeleteFunc

func (m *Map[K, V]) DeleteFunc(del func(key K, value V) bool)

DeleteFunc deletes any key/value pairs from the Map for which del returns true. DeleteFunc is safe for concurrent use.

Example

tm := ttl.NewMap[string, int](30*time.Second, 0, 2*time.Second, true)
defer tm.Close()

tm.Store("zero", 0)
tm.Store("one", 1)
tm.Store("two", 2)

// Delete all even keys
tm.DeleteFunc(func(key string, val int) bool {
	return val%2 == 0
})

tm.Range(func(key string, val int) bool {
	fmt.Printf("%s: %d\n", key, val)
	return true
})

Output:

one: 1

func (*Map[K, V]) Length

func (m *Map[K, V]) Length() int

Length returns the current length of the Map's internal map. Length is safe for concurrent use.

func (*Map[K, V]) Load

func (m *Map[K, V]) Load(key K) (value V, ok bool)

Load will retrieve a value from the Map, as well as a bool indicating whether the key was found. If the item was not found the value returned is undefined. Load is safe for concurrent use.

Example

tm := ttl.NewMap[string, string](30*time.Second, 0, 2*time.Second, true)
defer tm.Close()

tm.Store("hello", "world")

value, ok := tm.Load("hello")
if ok {
	fmt.Println(value)
}

Output:

world

func (*Map[K, V]) LoadPassive

func (m *Map[K, V]) LoadPassive(key K) (value V, ok bool)

LoadPassive will retrieve a value from the Map (without updating that value's time to live), as well as a bool indicating whether the key was found. If the item was not found the value returned is undefined. LoadPassive is safe for concurrent use.

func (*Map[K, V]) Range

func (m *Map[K, V]) Range(f func(key K, value V) bool)

Range calls f sequentially for each key and value present in the Map. If f returns false, Range stops the iteration.

Range is safe for concurrent use and supports modifying the value (assuming it's a reference type like a slice, map, or a pointer) within the range function. However, this requires a write lock on the Map – so you are not able to perform Map.Delete or Map.Store operations on the original Map directly within the range func, as that would cause a panic. Even an accessor like Map.Load or Map.LoadPassive would lock indefinitely.

If you need to perform operations on the original Map, do so in a new goroutine from within the range func – effectively deferring the operation until the Range completes.

If you just need to delete items with a certain key or value, use Map.DeleteFunc instead.

Example

tm := ttl.NewMap[string, string](30*time.Second, 0, 2*time.Second, true)
defer tm.Close()

tm.Store("hello", "world")
tm.Store("goodbye", "universe")

fmt.Printf("Length before: %d\n", tm.Length())

tm.Range(func(key string, val string) bool {
	if key == "goodbye" {
		// defer deletion in the original Map using a goroutine
		go func() {
			tm.Delete(key)
		}()

		return false // break
	}

	return true // continue
})

// Give the goroutine some time to complete
time.Sleep(20 * time.Millisecond)

fmt.Printf("Length after: %d\n", tm.Length())

Output:

Length before: 2
Length after: 1

func (*Map[K, V]) Store

func (m *Map[K, V]) Store(key K, value V)

Store will insert a value into the Map with the default tome to live. If the key/value pair already exists, the last access time will be updated, but the TTL will not be changed. This is important if the key/value pair was created with a non-default TTL using Map.StoreWithTTL. Store is safe for concurrent use.

func (*Map[K, V]) StoreWithTTL

func (m *Map[K, V]) StoreWithTTL(key K, value V, TTL time.Duration)

StoreWithTTL will insert a value into the Map with a custom time to live. If the key/value pair already exists, the last access time will be updated and the TTL will not be changed to the parameter value. Store is safe for concurrent use.