lb

Documentation

Index

• type LB
  • func New(makeWorkers WorkerFactory, maxAttempts int, timeout time.Duration) (*LB, error)
  • func NewAsync(clients []mode.AsyncProtocolClient, maxAttempts int, ...) (*LB, error)
  • func NewSync(clients []mode.ProtocolClient, maxAttempts int, ...) (*LB, error)
  • func (m *LB) Close() error
  • func (m *LB) PublishEvent(signaler op.Signaler, opts outputs.Options, event common.MapStr) error
  • func (m *LB) PublishEvents(signaler op.Signaler, opts outputs.Options, events []common.MapStr) error
• type WorkerFactory
  • func AsyncClients(clients []mode.AsyncProtocolClient, waitRetry, maxWaitRetry time.Duration) WorkerFactory
  • func SyncClients(clients []mode.ProtocolClient, waitRetry, maxWaitRetry time.Duration) WorkerFactory

Types

type LB

type LB struct {
	// contains filtered or unexported fields
}

LB balances the sending of events between multiple connections.

The balancing algorithm is mostly pull-based, with multiple workers trying to pull some amount of work from a shared queue. Workers will try to get a new work item only if they have a working/active connection. Workers without active connection do not participate until a connection has been re-established. Due to the pull based nature the algorithm will load-balance events by random with workers having less latencies/turn-around times potentially getting more work items then other workers with higher latencies. Thusly the algorithm dynamically adapts to resource availability of server events are forwarded to.

Workers not participating in the load-balancing will continuously try to reconnect to their configured endpoints. Once a new connection has been established, these workers will participate in in load-balancing again.

If a connection becomes unavailable, the events are rescheduled for another connection to pick up. Rescheduling events is limited to a maximum number of send attempts. If events have not been send after maximum number of allowed attemps has been passed, they will be dropped.

Like network connections, distributing events to workers is subject to timeout. If no worker is available to pickup a message for sending, the message will be dropped internally after max_retries. If mode or message requires guaranteed send, message is retried infinitely.

func New

func New(
	makeWorkers WorkerFactory,
	maxAttempts int,
	timeout time.Duration,
) (*LB, error)

New create a new load balancer connection mode.

func NewAsync

func NewAsync(
	clients []mode.AsyncProtocolClient,
	maxAttempts int,
	waitRetry, timeout, maxWaitRetry time.Duration,
) (*LB, error)

func NewSync

func NewSync(
	clients []mode.ProtocolClient,
	maxAttempts int,
	waitRetry, timeout, maxWaitRetry time.Duration,
) (*LB, error)

func (*LB) Close

func (m *LB) Close() error

Close stops all workers and closes all open connections. In flight events are signaled as failed.

func (*LB) PublishEvent

func (m *LB) PublishEvent(
	signaler op.Signaler,
	opts outputs.Options,
	event common.MapStr,
) error

PublishEvent forwards the event to some load balancing worker.

func (*LB) PublishEvents

func (m *LB) PublishEvents(
	signaler op.Signaler,
	opts outputs.Options,
	events []common.MapStr,
) error

PublishEvents forwards events to some load balancing worker.

type WorkerFactory

type WorkerFactory interface {
	// contains filtered or unexported methods
}

func AsyncClients

func AsyncClients(
	clients []mode.AsyncProtocolClient,
	waitRetry, maxWaitRetry time.Duration,
) WorkerFactory

func SyncClients

func SyncClients(
	clients []mode.ProtocolClient,
	waitRetry, maxWaitRetry time.Duration,
) WorkerFactory