Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var (
Error = errs.Class("tally error")
)
Error is a standard error class for this package.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct { Interval time.Duration `help:"how frequently the tally service should run" releaseDefault:"1h" devDefault:"30s"` SaveRollupBatchSize int `help:"how large of batches SaveRollup should process at a time" default:"1000"` ReadRollupBatchSize int `help:"how large of batches GetBandwidthSince should process at a time" default:"10000"` }
Config contains configurable values for the tally service.
type Observer ¶ added in v0.23.0
type Observer struct { Now time.Time Log *zap.Logger Node map[storj.NodeID]float64 Bucket map[metabase.BucketLocation]*accounting.BucketTally }
Observer observes metainfo and adds up tallies for nodes and buckets.
func NewObserver ¶ added in v0.23.0
NewObserver returns an metainfo loop observer that adds up totals for buckets and nodes. The now argument controls when the observer considers pointers to be expired.
func (*Observer) InlineSegment ¶ added in v0.23.0
InlineSegment is called for each inline segment.
type Service ¶
Service is the tally service for data stored on each storage node
architecture: Chore
func New ¶
func New(log *zap.Logger, sdb accounting.StoragenodeAccounting, pdb accounting.ProjectAccounting, liveAccounting accounting.Cache, metainfoLoop *metainfo.Loop, interval time.Duration) *Service
New creates a new tally Service.
func (*Service) SetNow ¶ added in v1.3.2
SetNow allows tests to have the Service act as if the current time is whatever they want. This avoids races and sleeping, making tests more reliable and efficient.
func (*Service) Tally ¶
Tally calculates data-at-rest usage once.
How live accounting is calculated:
At the beginning of the tally iteration, we get a map containing the current project totals from the cache- initialLiveTotals (our current estimation of the project totals). At the end of the tally iteration, we have the totals from what we saw during the metainfo loop.
However, data which was uploaded during the loop may or may not have been seen in the metainfo loop. For this reason, we also read the live accounting totals again at the end of the tally iteration- latestLiveTotals.
The difference between latest and initial indicates how much data was uploaded during the metainfo loop and is assigned to delta. However, again, we aren't certain how much of the delta is accounted for in the metainfo totals. For the reason we make an assumption that 50% of the data is accounted for. So to calculate the new live accounting totals, we sum the metainfo totals and 50% of the deltas.