circuitbreaker

package
v1.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 14, 2021 License: Apache-2.0 Imports: 10 Imported by: 44

Documentation

Overview

Package circuitbreaker implements the circuit breaker pattern, which provides stability and prevents cascading failures in distributed systems.

Sentinel circuit breaker module supports three strategies:

  1. SlowRequestRatio: the ratio of slow response time entry(entry's response time is great than max slow response time) exceeds the threshold. The following entry to resource will be broken. In SlowRequestRatio strategy, user must set max response time.
  2. ErrorRatio: the ratio of error entry exceeds the threshold. The following entry to resource will be broken.
  3. ErrorCount: the number of error entry exceeds the threshold. The following entry to resource will be broken.

Sentinel converts each circuit breaking Rule into a CircuitBreaker. Each CircuitBreaker has its own statistical structure.

Sentinel circuit breaker is implemented based on state machines. There are three states:

  1. Closed: all entries could pass checking.
  2. Open: the circuit breaker is broken, all entries are blocked. After retry timeout, circuit breaker switches state to Half-Open and allows one entry to probe whether the resource returns to its expected state.
  3. Half-Open: the circuit breaker is in a temporary state of probing, only one entry is allowed to access resource, others are blocked.

Sentinel circuit breaker provides the listener to observe events of state changes. 

type StateChangeListener interface {
	OnTransformToClosed(prev State, rule Rule)

	OnTransformToOpen(prev State, rule Rule, snapshot interface{})

	OnTransformToHalfOpen(prev State, rule Rule)
}

Here is the example code to use circuit breaker: 

 type stateChangeTestListener struct {}

 func (s *stateChangeTestListener) OnTransformToClosed(prev circuitbreaker.State, rule circuitbreaker.Rule) {
 	fmt.Printf("rule.steategy: %+v, From %s to Closed, time: %d\n", rule.Strategy, prev.String(), util.CurrentTimeMillis())
 }

 func (s *stateChangeTestListener) OnTransformToOpen(prev circuitbreaker.State, rule circuitbreaker.Rule, snapshot interface{}) {
 	fmt.Printf("rule.steategy: %+v, From %s to Open, snapshot: %.2f, time: %d\n", rule.Strategy, prev.String(), snapshot, util.CurrentTimeMillis())
 }

 func (s *stateChangeTestListener) OnTransformToHalfOpen(prev circuitbreaker.State, rule circuitbreaker.Rule) {
 	fmt.Printf("rule.steategy: %+v, From %s to Half-Open, time: %d\n", rule.Strategy, prev.String(), util.CurrentTimeMillis())
 }

 func main() {
 	err := sentinel.InitDefault()
 	if err != nil {
 		log.Fatal(err)
 	}
 	ch := make(chan struct{})
 	// Register a state change listener so that we could observer the state change of the internal circuit breaker.
 	circuitbreaker.RegisterStateChangeListeners(&stateChangeTestListener{})

 	_, err = circuitbreaker.LoadRules([]*circuitbreaker.Rule{
 	// Statistic time span=10s, recoveryTimeout=3s, slowRtUpperBound=50ms, maxSlowRequestRatio=50%
 		{
 			Resource:         "abc",
 			Strategy:         circuitbreaker.SlowRequestRatio,
 			RetryTimeoutMs:   3000,
 			MinRequestAmount: 10,
 			StatIntervalMs:   10000,
 			MaxAllowedRtMs:   50,
 			Threshold:        0.5,
 		},
 		// Statistic time span=10s, recoveryTimeout=3s, maxErrorRatio=50%
 		{
 			Resource:         "abc",
 			Strategy:         circuitbreaker.ErrorRatio,
 			RetryTimeoutMs:   3000,
 			MinRequestAmount: 10,
 			StatIntervalMs:   10000,
 			Threshold:        0.5,
 		},
 	})
 	if err != nil {
 		log.Fatal(err)
 	}

 	fmt.Println("Sentinel Go circuit breaking demo is running. You may see the pass/block metric in the metric log.")
 	go func() {
 		for {
 			e, b := sentinel.Entry("abc")
 			if b != nil {
 				//fmt.Println("g1blocked")
 				time.Sleep(time.Duration(rand.Uint64()%20) * time.Millisecond)
 			} else {
 				if rand.Uint64()%20 > 9 {
 					// Record current invocation as error.
 					sentinel.TraceError(e, errors.New("biz error"))
 				}
 				//fmt.Println("g1passed")
 				time.Sleep(time.Duration(rand.Uint64()%80+10) * time.Millisecond)
 				e.Exit()
 			}
 		}
		}()

 	go func() {
 		for {
 			e, b := sentinel.Entry("abc")
 			if b != nil {
 				//fmt.Println("g2blocked")
 				time.Sleep(time.Duration(rand.Uint64()%20) * time.Millisecond)
 			} else {
 				//fmt.Println("g2passed")
 				time.Sleep(time.Duration(rand.Uint64()%80) * time.Millisecond)
 				e.Exit()
 			}
 		}
 	}()
 	<-ch
 }

Index

Constants

View Source 
const (
	RuleCheckSlotOrder = 5000
)
View Source 
const (
	StatSlotOrder = 5000
)

Variables

View Source 
var (
	DefaultMetricStatSlot = &MetricStatSlot{}
)
View Source 
var (
	DefaultSlot = &Slot{}
)

Functions

func ClearRules added in v0.4.0 

func ClearRules() error

ClearRules clear all the previous rules.

func ClearRulesOfResource added in v1.0.3 

func ClearRulesOfResource(res string) error

ClearRulesOfResource clears resource level rules in circuitBreaker module.

func ClearStateChangeListeners added in v0.6.0 

func ClearStateChangeListeners()

ClearStateChangeListeners clears the all StateChangeListener Note: this function is not thread-safe.

func IsValidRule added in v1.0.3 

func IsValidRule(r *Rule) error

func LoadRules 

func LoadRules(rules []*Rule) (bool, error)

LoadRules replaces old rules with the given circuit breaking rules.

return value:

bool: was designed to indicate whether the internal map has been changed error: was designed to indicate whether occurs the error.

func LoadRulesOfResource added in v1.0.3 

func LoadRulesOfResource(res string, rules []*Rule) (bool, error)

LoadRulesOfResource loads the given resource's circuitBreaker rules to the rule manager, while all previous resource's rules will be replaced. the first returned value indicates whether do real load operation, if the rules is the same with previous resource's rules, return false

func RegisterStateChangeListeners 

func RegisterStateChangeListeners(listeners ...StateChangeListener)

RegisterStateChangeListeners registers the global state change listener for all circuit breakers Note: this function is not thread-safe.

func RemoveCircuitBreakerGenerator 

func RemoveCircuitBreakerGenerator(s Strategy) error

func SetCircuitBreakerGenerator 

func SetCircuitBreakerGenerator(s Strategy, generator CircuitBreakerGenFunc) error

SetCircuitBreakerGenerator sets the circuit breaker generator for the given strategy. Note that modifying the generator of default strategies is not allowed.

Types

type CircuitBreaker 

type CircuitBreaker interface {
	// BoundRule returns the associated circuit breaking rule.
	BoundRule() *Rule
	// BoundStat returns the associated statistic data structure.
	BoundStat() interface{}
	// TryPass acquires permission of an invocation only if it is available at the time of invocation.
	TryPass(ctx *base.EntryContext) bool
	// CurrentState returns current state of the circuit breaker.
	CurrentState() State
	// OnRequestComplete record a completed request with the given response time as well as error (if present),
	// and handle state transformation of the circuit breaker.
	// OnRequestComplete is called only when a passed invocation finished.
	OnRequestComplete(rtt uint64, err error)
}

CircuitBreaker is the basic interface of circuit breaker

type CircuitBreakerGenFunc 

type CircuitBreakerGenFunc func(r *Rule, reuseStat interface{}) (CircuitBreaker, error)

type MetricStatSlot 

type MetricStatSlot struct {
}

MetricStatSlot records metrics for circuit breaker on invocation completed. MetricStatSlot must be filled into slot chain if circuit breaker is alive.

func (*MetricStatSlot) OnCompleted 

func (c *MetricStatSlot) OnCompleted(ctx *base.EntryContext)

func (*MetricStatSlot) OnEntryBlocked 

func (c *MetricStatSlot) OnEntryBlocked(_ *base.EntryContext, _ *base.BlockError)

func (*MetricStatSlot) OnEntryPassed 

func (c *MetricStatSlot) OnEntryPassed(_ *base.EntryContext)

func (*MetricStatSlot) Order added in v1.0.0 

func (s *MetricStatSlot) Order() uint32

type Rule 

type Rule struct {
	// unique id
	Id string `json:"id,omitempty"`
	// resource name
	Resource string   `json:"resource"`
	Strategy Strategy `json:"strategy"`
	// RetryTimeoutMs represents recovery timeout (in milliseconds) before the circuit breaker opens.
	// During the open period, no requests are permitted until the timeout has elapsed.
	// After that, the circuit breaker will transform to half-open state for trying a few "trial" requests.
	RetryTimeoutMs uint32 `json:"retryTimeoutMs"`
	// MinRequestAmount represents the minimum number of requests (in an active statistic time span)
	// that can trigger circuit breaking.
	MinRequestAmount uint64 `json:"minRequestAmount"`
	// StatIntervalMs represents statistic time interval of the internal circuit breaker (in ms).
	// Currently the statistic interval is collected by sliding window.
	StatIntervalMs uint32 `json:"statIntervalMs"`
	// StatSlidingWindowBucketCount represents the bucket count of statistic sliding window.
	// The statistic will be more precise as the bucket count increases, but the memory cost increases too.
	// The following must be true — “StatIntervalMs % StatSlidingWindowBucketCount == 0”,
	// otherwise StatSlidingWindowBucketCount will be replaced by 1.
	// If it is not set, default value 1 will be used.
	StatSlidingWindowBucketCount uint32 `json:"statSlidingWindowBucketCount"`
	// MaxAllowedRtMs indicates that any invocation whose response time exceeds this value (in ms)
	// will be recorded as a slow request.
	// MaxAllowedRtMs only takes effect for SlowRequestRatio strategy
	MaxAllowedRtMs uint64 `json:"maxAllowedRtMs"`
	// Threshold represents the threshold of circuit breaker.
	// for SlowRequestRatio, it represents the max slow request ratio
	// for ErrorRatio, it represents the max error request ratio
	// for ErrorCount, it represents the max error request count
	Threshold float64 `json:"threshold"`
	//ProbeNum is number of probes required when the circuit breaker is half-open.
	//when the probe num are set  and circuit breaker in the half-open state.
	//if err occurs during the probe, the circuit breaker is opened immediately.
	//otherwise,the circuit breaker is closed only after the number of probes is reached
	ProbeNum uint64 `json:"probeNum"`
}

Rule encompasses the fields of circuit breaking rule.

func GetRules added in v1.0.0 

func GetRules() []Rule

GetRules returns all the rules based on copy. It doesn't take effect for circuit breaker module if user changes the rule. GetRules need to compete circuit breaker module's global lock and the high performance losses of copy, 

reduce or do not call GetRules if possible

func GetRulesOfResource added in v1.0.0 

func GetRulesOfResource(resource string) []Rule

GetRulesOfResource returns specific resource's rules based on copy. It doesn't take effect for circuit breaker module if user changes the rule. GetRulesOfResource need to compete circuit breaker module's global lock and the high performance losses of copy, 

reduce or do not call GetRulesOfResource frequently if possible

func (*Rule) ResourceName added in v0.6.0 

func (r *Rule) ResourceName() string

func (*Rule) String added in v0.6.0 

func (r *Rule) String() string

type Slot added in v0.5.0 

type Slot struct {
}

func (*Slot) Check added in v0.5.0 

func (b *Slot) Check(ctx *base.EntryContext) *base.TokenResult

func (*Slot) Order added in v1.0.0 

func (s *Slot) Order() uint32

type State 

type State int32
 Circuit Breaker State Machine:

                                switch to open based on rule
				+-----------------------------------------------------------------------+
				|                                                                       |
				|                                                                       v
		+----------------+                   +----------------+      Probe      +----------------+
		|                |                   |                |<----------------|                |
		|                |   Probe succeed   |                |                 |                |
		|     Closed     |<------------------|    HalfOpen    |                 |      Open      |
		|                |                   |                |   Probe failed  |                |
		|                |                   |                +---------------->|                |
		+----------------+                   +----------------+                 +----------------+

const (
	Closed State = iota
	HalfOpen
	Open
)

func (*State) String 

func (s *State) String() string

type StateChangeListener 

type StateChangeListener interface {
	// OnTransformToClosed is triggered when circuit breaker state transformed to Closed.
	// Argument rule is copy from circuit breaker's rule, any changes of rule don't take effect for circuit breaker
	// Copying rule has a performance penalty and avoids invalid listeners as much as possible
	OnTransformToClosed(prev State, rule Rule)

	// OnTransformToOpen is triggered when circuit breaker state transformed to Open.
	// The "snapshot" indicates the triggered value when the transformation occurs.
	// Argument rule is copy from circuit breaker's rule, any changes of rule don't take effect for circuit breaker
	// Copying rule has a performance penalty and avoids invalid listeners as much as possible
	OnTransformToOpen(prev State, rule Rule, snapshot interface{})

	// OnTransformToHalfOpen is triggered when circuit breaker state transformed to HalfOpen.
	// Argument rule is copy from circuit breaker's rule, any changes of rule don't take effect for circuit breaker
	// Copying rule has a performance penalty and avoids invalid listeners as much as possible
	OnTransformToHalfOpen(prev State, rule Rule)
}

StateChangeListener listens on the circuit breaker state change event

type Strategy 

type Strategy uint32

Strategy represents the strategy of circuit breaker. Each strategy is associated with one rule type. 

const (
	// SlowRequestRatio strategy changes the circuit breaker state based on slow request ratio
	SlowRequestRatio Strategy = iota
	// ErrorRatio strategy changes the circuit breaker state based on error request ratio
	ErrorRatio
	// ErrorCount strategy changes the circuit breaker state based on error amount
	ErrorCount
)

func (Strategy) String 

func (s Strategy) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.