throttlegroup

README

Throttle Group

Package throttlegroup is a variation of the errgroup package (golang/x/sync/errgroup) that uses throttling to control the active number of goroutines at the same time. This is useful while running CPU and/or memory intensive code concurrently (e.g.: if you launch too many goroutines, you will end up of resources).

package main

import (
	"fmt"
	"time"

	"github.com/andreynering/goext/syncext/throttlegroup"
)

func main() {
	g := throttlegroup.WithThrottle(2)

	for i := 0; i < 10; i++ {
		i := i
		g.Go(func() error {
			fmt.Println(i)
			time.Sleep(time.Second)
			return nil
		})
	}
	if err := g.Wait(); err != nil {
		panic(err)
	}
}

Documentation

Overview

Package throttlegroup is a variation of the errgroup package (golang/x/sync/errgroup) that uses throttling to control the active number of goroutines at the same time. This is useful while running CPU and/or memory intensive code concurrently (e.g.: if you launch too many goroutines, you will end up of resources).

Index

• type Group
  • func Default() *Group
  • func DefaultWithContext(ctx context.Context) (*Group, context.Context)
  • func WithContext(ctx context.Context, num int) (g *Group, resultCtx context.Context)
  • func WithThrottle(num int) *Group
  • func (g *Group) Go(f func() error)

Examples

• Group

Types

type Group

type Group struct {
	*errgroup.Group
	// contains filtered or unexported fields
}

Group is a throttle group. The zero value should not be used.

Example

package main

import (
	"fmt"
	"time"

	"github.com/andreynering/goext/syncext/throttlegroup"
)

func main() {
	g := throttlegroup.WithThrottle(2)

	for i := 0; i < 10; i++ {
		i := i
		g.Go(func() error {
			fmt.Println(i)
			time.Sleep(time.Second)
			return nil
		})
	}
	if err := g.Wait(); err != nil {
		panic(err)
	}
}

func Default

func Default() *Group

Default returns a throttle group sized by the number of CPU cores of the device.

func DefaultWithContext

func DefaultWithContext(ctx context.Context) (*Group, context.Context)

DefaultWithContext returns a throttle group and a context, given a parent context. The size of the group will be the number of the CPU cores of the device.

func WithContext

func WithContext(ctx context.Context, num int) (g *Group, resultCtx context.Context)

WithContext returns a throttle group and a context, given a parent context and a throttle size.

func WithThrottle

func WithThrottle(num int) *Group

New returns a throttle group sized by the given number. E.g.: throttle of 4 means 4 active goroutines at the same time.

func (*Group) Go

func (g *Group) Go(f func() error)

Go overrides the parent errgroup.Group Go method, to make the throttling automatic.