Documentation ¶
Overview ¶
Package gc contains the functions needed to run garbage collection.
The gc.PieceTracker implements the metainfo loop Observer interface allowing us to subscribe to the loop to get information for every segment in the metainfo database.
The gc.PieceTracker handling functions are used by the gc.Service to periodically account for all existing pieces on storage nodes and create "retain requests" which contain a bloom filter of all pieces that possibly exist on a storage node.
The gc.Service will send that request to the storagenode after a full metaloop iteration, and the storage node will use that request to delete the "garbage" pieces that are not in the bloom filter.
See storj/docs/design/garbage-collection.md for more info.
Index ¶
- Variables
- type Config
- type PieceTracker
- func (pieceTracker *PieceTracker) InlineSegment(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
- func (pieceTracker *PieceTracker) RemoteObject(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
- func (pieceTracker *PieceTracker) RemoteSegment(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
- type RetainInfo
- type Service
Constants ¶
This section is empty.
Variables ¶
var ( // Error defines the gc service errors class Error = errs.Class("gc service error") )
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct { Interval time.Duration `help:"the time between each send of garbage collection filters to storage nodes" releaseDefault:"120h" devDefault:"10m"` Enabled bool `help:"set if garbage collection is enabled or not" releaseDefault:"true" devDefault:"true"` // value for InitialPieces currently based on average pieces per node InitialPieces int `` /* 139-byte string literal not displayed */ FalsePositiveRate float64 `help:"the false positive rate used for creating a garbage collection bloom filter" releaseDefault:"0.1" devDefault:"0.1"` ConcurrentSends int `help:"the number of nodes to concurrently send garbage collection bloom filters to" releaseDefault:"1" devDefault:"1"` }
Config contains configurable values for garbage collection
type PieceTracker ¶
type PieceTracker struct {
// contains filtered or unexported fields
}
PieceTracker implements the metainfo loop observer interface for garbage collection
func NewPieceTracker ¶
func NewPieceTracker(log *zap.Logger, config Config, pieceCounts map[storj.NodeID]int) *PieceTracker
NewPieceTracker instantiates a new gc piece tracker to be subscribed to the metainfo loop
func (*PieceTracker) InlineSegment ¶
func (pieceTracker *PieceTracker) InlineSegment(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
InlineSegment returns nil because we're only doing gc for storage nodes for now
func (*PieceTracker) RemoteObject ¶
func (pieceTracker *PieceTracker) RemoteObject(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
RemoteObject returns nil because gc does not interact with remote objects
func (*PieceTracker) RemoteSegment ¶
func (pieceTracker *PieceTracker) RemoteSegment(ctx context.Context, path storj.Path, pointer *pb.Pointer) (err error)
RemoteSegment takes a remote segment found in metainfo and adds pieces to bloom filters
type RetainInfo ¶
type RetainInfo struct { Filter *bloomfilter.Filter CreationDate time.Time Count int }
RetainInfo contains info needed for a storage node to retain important data and delete garbage data