sharding

package
v2.8.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 21, 2024 License: Apache-2.0 Imports: 13 Imported by: 3

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func InferShard

func InferShard() (int, error)

InferShard extracts the shard index based on its hostname.

Types

type ClusterFilterFunction added in v2.8.0

type ClusterFilterFunction func(c *v1alpha1.Cluster) bool

func GetClusterFilter

func GetClusterFilter(distributionFunction DistributionFunction, shard int) ClusterFilterFunction

GetClusterFilter returns a ClusterFilterFunction which is a function taking a cluster as a parameter and returns wheter or not the cluster should be processed by a given shard. It calls the distributionFunction to determine which shard will process the cluster, and if the given shard is equal to the calculated shard the function will return true.

type DistributionFunction added in v2.8.0

type DistributionFunction func(c *v1alpha1.Cluster) int

func GetDistributionFunction added in v2.8.0

func GetDistributionFunction(db db.ArgoDB, shardingAlgorithm string) DistributionFunction

GetDistributionFunction returns which DistributionFunction should be used based on the passed algorithm and the current datas.

func LegacyDistributionFunction added in v2.8.0

func LegacyDistributionFunction() DistributionFunction

LegacyDistributionFunction returns a DistributionFunction using a stable distribution algorithm: for a given cluster the function will return the shard number based on the cluster id. This function is lightweight and can be distributed easily, however, it does not ensure an homogenous distribution as some shards may get assigned more clusters than others. It is the legacy function distribution that is kept for compatibility reasons

func RoundRobinDistributionFunction added in v2.8.0

func RoundRobinDistributionFunction(db db.ArgoDB) DistributionFunction

RoundRobinDistributionFunction returns a DistributionFunction using an homogeneous distribution algorithm: for a given cluster the function will return the shard number based on the modulo of the cluster rank in the cluster's list sorted by uid on the shard number. This function ensures an homogenous distribution: each shards got assigned the same number of clusters +/-1 , but with the drawback of a reshuffling of clusters accross shards in case of some changes in the cluster list

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL