shuffle

Documentation

Index

• type Shuffle
  • func NewShuffle(vertexName string, partitionCount int) *Shuffle
  • func (s *Shuffle) Shuffle(keys []string) int32
  • func (s *Shuffle) ShuffleMessages(messages []*isb.Message) map[int32][]*isb.Message

Types

type Shuffle

type Shuffle struct {
	// contains filtered or unexported fields
}

Shuffle shuffles messages among ISB

func NewShuffle

func NewShuffle(vertexName string, partitionCount int) *Shuffle

NewShuffle accepts list of buffer identifiers(unique identifier of isb) and returns new shuffle instance. It uses vertex-name as seed, without a seed, we will end with the problem where Shuffling before the Vnth vertex creates a key to edge-buffer-index affinity, which will not change from Vn to Vn+1 Reduce vertices if there is no re-keying between these vertices causing idle partitions.

func (*Shuffle) Shuffle

func (s *Shuffle) Shuffle(keys []string) int32

Shuffle functions returns a shuffled identifier.

func (*Shuffle) ShuffleMessages

func (s *Shuffle) ShuffleMessages(messages []*isb.Message) map[int32][]*isb.Message

ShuffleMessages accepts list of isb messages and returns the mapping of isb to messages