countstore

Documentation

Overview

Interface for fast atomic counters, and separate implementations using redis and in-process memory.

Index

• Constants
• type CountStore
• type MemCountStore
  • func NewMemCountStore() MemCountStore
  • func (s MemCountStore) GetCount(ctx context.Context, name, val, period string) (int, error)
  • func (s MemCountStore) GetCountDistinct(ctx context.Context, name, bucket, period string) (int, error)
  • func (s MemCountStore) Increment(ctx context.Context, name, val string) error
  • func (s MemCountStore) IncrementDistinct(ctx context.Context, name, bucket, val string) error
  • func (s MemCountStore) IncrementPeriod(ctx context.Context, name, val, period string) error
• type RedisCountStore
  • func NewRedisCountStore(redisURL string) (*RedisCountStore, error)
  • func (s *RedisCountStore) GetCount(ctx context.Context, name, val, period string) (int, error)
  • func (s *RedisCountStore) GetCountDistinct(ctx context.Context, name, val, period string) (int, error)
  • func (s *RedisCountStore) Increment(ctx context.Context, name, val string) error
  • func (s *RedisCountStore) IncrementDistinct(ctx context.Context, name, bucket, val string) error
  • func (s *RedisCountStore) IncrementPeriod(ctx context.Context, name, val, period string) error

Constants

const (
	PeriodTotal = "total"
	PeriodDay   = "day"
	PeriodHour  = "hour"
)

Types

type CountStore

type CountStore interface {
	GetCount(ctx context.Context, name, val, period string) (int, error)
	Increment(ctx context.Context, name, val string) error
	IncrementPeriod(ctx context.Context, name, val, period string) error
	// TODO: batch increment method
	GetCountDistinct(ctx context.Context, name, bucket, period string) (int, error)
	IncrementDistinct(ctx context.Context, name, bucket, val string) error
}

CountStore is an interface for storing incrementing event counts, bucketed into periods. It is implemented by MemCountStore and by RedisCountStore.

Period bucketing works on the basis of the current date (as determined mid-call). See the `Period*` consts for the available period types.

The "GetCount" and "Increment" methods perform actual counting. The "*Distinct" methods have a different behavior: "IncrementDistinct" marks a value as seen at least once, and "GetCountDistinct" asks how _many_ values have been seen at least once.

Incrementing -- both the "Increment" and "IncrementDistinct" variants -- increases a count in each supported period bucket size. In other words, one call to CountStore.Increment causes three increments internally: one to the count for the hour, one to the count for the day, and one to the all-time count. The "IncrementPeriod" method allows only incrementing a single period bucket. Care must be taken to match the "GetCount" period with the incremented period when using this variant.

The exact implementation and precision of the "*Distinct" methods may vary: in the MemCountStore implementation, it is precise (it's based on large maps); in the RedisCountStore implementation, it uses the Redis "pfcount" feature, which is based on a HyperLogLog datastructure which has probabilistic properties (see https://redis.io/commands/pfcount/ ).

Memory growth and availability of information over time also varies by implementation. The RedisCountStore implementation uses Redis's key expiration primitives; only the all-time counts go without expiration. The MemCountStore grows without bound (it's intended to be used in testing and other non-production operations).

type MemCountStore

type MemCountStore struct {
	// Counts is keyed by a string that is a munge of "{name}/{val}[/{period}]",
	// where period is either absent (meaning all-time total)
	// or a string describing that timeperiod (either "YYYY-MM-DD" or that plus a literal "T" and "HH").
	//
	// (Using a values for `name` and `val` with slashes in them is perhaps inadvisable, as it may be ambiguous.)
	Counts         *xsync.MapOf[string, int]
	DistinctCounts *xsync.MapOf[string, *xsync.MapOf[string, bool]]
}

func NewMemCountStore

func NewMemCountStore() MemCountStore

func (MemCountStore) GetCount

func (s MemCountStore) GetCount(ctx context.Context, name, val, period string) (int, error)

func (MemCountStore) GetCountDistinct

func (s MemCountStore) GetCountDistinct(ctx context.Context, name, bucket, period string) (int, error)

func (MemCountStore) Increment

func (s MemCountStore) Increment(ctx context.Context, name, val string) error

func (MemCountStore) IncrementDistinct

func (s MemCountStore) IncrementDistinct(ctx context.Context, name, bucket, val string) error

func (MemCountStore) IncrementPeriod

func (s MemCountStore) IncrementPeriod(ctx context.Context, name, val, period string) error

type RedisCountStore

type RedisCountStore struct {
	Client *redis.Client
}

func NewRedisCountStore

func NewRedisCountStore(redisURL string) (*RedisCountStore, error)

func (*RedisCountStore) GetCount

func (s *RedisCountStore) GetCount(ctx context.Context, name, val, period string) (int, error)

func (*RedisCountStore) GetCountDistinct

func (s *RedisCountStore) GetCountDistinct(ctx context.Context, name, val, period string) (int, error)

func (*RedisCountStore) Increment

func (s *RedisCountStore) Increment(ctx context.Context, name, val string) error

func (*RedisCountStore) IncrementDistinct

func (s *RedisCountStore) IncrementDistinct(ctx context.Context, name, bucket, val string) error

func (*RedisCountStore) IncrementPeriod

func (s *RedisCountStore) IncrementPeriod(ctx context.Context, name, val, period string) error

Variant of Increment() which only acts on a single specified time period. The intended us of this variant is to control the total number of counters persisted, by using a relatively short time period, for which the counters will expire.