Documentation
¶
Overview ¶
Package msgrate allows estimating the throughput of peers for more balanced syncs.
Index ¶
- type Tracker
- type Trackers
- func (t *Trackers) Capacity(id string, kind uint64, targetRTT time.Duration) int
- func (t *Trackers) MeanCapacities() map[uint64]float64
- func (t *Trackers) MedianRoundTrip() time.Duration
- func (t *Trackers) TargetRoundTrip() time.Duration
- func (t *Trackers) TargetTimeout() time.Duration
- func (t *Trackers) Track(id string, tracker *Tracker) error
- func (t *Trackers) Untrack(id string) error
- func (t *Trackers) Update(id string, kind uint64, elapsed time.Duration, items int)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Tracker ¶
type Tracker struct {
// contains filtered or unexported fields
}
Tracker estimates the throughput capacity of a peer with regard to each data type it can deliver. The goal is to dynamically adjust request sizes to max out network throughput without overloading either the peer or th elocal node.
By tracking in real time the latencies and bandiwdths peers exhibit for each packet type, it's possible to prevent overloading by detecting a slowdown on one type when another type is pushed too hard.
Similarly, real time measurements also help avoid overloading the local net connection if our peers would otherwise be capable to deliver more, but the local link is saturated. In that case, the live measurements will force us to reduce request sizes until the throughput gets stable.
Lastly, message rate measurements allows us to detect if a peer is unusually slow compared to other peers, in which case we can decide to keep it around or free up the slot so someone closer.
Since throughput tracking and estimation adapts dynamically to live network conditions, it's fine to have multiple trackers locally track the same peer in different subsystem. The throughput will simply be distributed across the two trackers if both are highly active.
func NewTracker ¶
NewTracker creates a new message rate tracker for a specific peer. An initial RTT is needed to avoid a peer getting marked as an outlier compared to others right after joining. It's suggested to use the median rtt across all peers to init a new peer tracker.
func (*Tracker) Capacity ¶
Capacity calculates the number of items the peer is estimated to be able to retrieve within the allotted time slot. The method will round up any division errors and will add an additional overestimation ratio on top. The reason for overshooting the capacity is because certain message types might not increase the load proportionally to the requested items, so fetching a bit more might still take the same RTT. By forcefully overshooting by a small amount, we can avoid locking into a lower-that-real capacity.
func (*Tracker) Update ¶
Update modifies the peer's capacity values for a specific data type with a new measurement. If the delivery is zero, the peer is assumed to have either timed out or to not have the requested data, resulting in a slash to 0 capacity. This avoids assigning the peer retrievals that it won't be able to honour.
type Trackers ¶
type Trackers struct {
// The fields below can be used to override certain default values. Their
// purpose is to allow quicker tests. Don't use them in production.
OverrideTTLLimit time.Duration
// contains filtered or unexported fields
}
Trackers is a set of message rate trackers across a number of peers with the goal of aggregating certain measurements across the entire set for outlier filtering and newly joining initialization.
func NewTrackers ¶
NewTrackers creates an empty set of trackers to be filled with peers.
func (*Trackers) Capacity ¶
Capacity is a helper function to access a specific tracker without having to track it explicitly outside.
func (*Trackers) MeanCapacities ¶
MeanCapacities returns the capacities averaged across all the added trackers. The purpos of the mean capacities are to initialize a new peer with some sane starting values that it will hopefully outperform. If the mean overshoots, the peer will be cut back to minimal capacity and given another chance.
func (*Trackers) MedianRoundTrip ¶
MedianRoundTrip returns the median RTT across all known trackers. The purpose of the median RTT is to initialize a new peer with sane statistics that it will hopefully outperform. If it seriously underperforms, there's a risk of dropping the peer, but that is ok as we're aiming for a strong median.
func (*Trackers) TargetRoundTrip ¶
TargetRoundTrip returns the current target round trip time for a request to complete in.The returned RTT is slightly under the estimated RTT. The reason is that message rate estimation is a 2 dimensional problem which is solvable for any RTT. The goal is to gravitate towards smaller RTTs instead of large messages, to result in a stabler download stream.
func (*Trackers) TargetTimeout ¶
TargetTimeout returns the timeout allowance for a single request to finish under. The timeout is proportional to the roundtrip, but also takes into consideration the tracker's confidence in said roundtrip and scales it accordingly. The final value is capped to avoid runaway requests.
Source Files