Documentation
¶
Overview ¶
Package gc contains the GarbageCollector which allows cacheable resources to register themselves for disposal when we run low on resources.
The idea is that cacheable resources like cache folders, docker images, downloaded artifacts. Any resource that can be recreated if needed, but that we would like to keep around for as long as possible, assuming it doesn't affect system resources.
To do this we need to be able to dispose resources when the system runs low on disk space or memory. This easily gets complicated, especially as we may want to optimize by diposing least-recently-used first. So to simplify it cacheable resources should implement the Disposable interface and be registered with the GarbageCollector, so it prioritize disposal, even when we have different types of cacheable resources.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrDisposableSizeNotSupported is used by Disposable to indicate that a resource // doesn't support a size function like MemorySize() or DiskSize() ErrDisposableSizeNotSupported = errors.New("Disposable instance doesn't support") // ErrDisposableInUse is used by Disposable to indicate that a resource is currently in // use and cannot be disposed ErrDisposableInUse = errors.New("Disposable is currently in use") )
Functions ¶
This section is empty.
Types ¶
type Disposable ¶
type Disposable interface {
// Get memory size used by resource, return ErrDisposableSizeNotSupported
// if you don't know how much memory it uses, but knows that it uses some
// non-trivial amount. If it just uses a small fixed amount, you can return
// 1 for simplicity.
MemorySize() (uint64, error)
// Get disk space used by resource, return ErrDisposableSizeNotSupported
// if you don't know how much disk space it uses, but knows that it uses some
// non-trivial amount. If it just uses a small fixed amount, you can return
// 1 for simplicity.
DiskSize() (uint64, error)
// Clean up after this resource, return ErrDisposableInUse if the resource is
// in use. Note that implementors should use this method to also remove any
// references to this resource from the resource manager, which typically has
// an internal list of cached resources.
//
// Warning, return an error other than ErrDisposableInUse and the worker will
// panic, abort tasks, alert operation and crash.
Dispose() error
// Last time the cache was used
LastUsed() time.Time
}
The Disposable Interface to be implemented by resources that can be managed by the GarbageCollector. To have the resource garbage collected you must call GarbageCollector.Register(&resource). The garbage collector will call resource.Dispose() when it wants to remove your resource, you must check that the resource isn't in use and return ErrDisposableInUse if it is.
Warning, return any error other than ErrDisposableSizeNotSupported or ErrDisposableInUse and it will result in a internal error aborting all tasks, reporting to operators and crashing the worker. This is because we can't have workers leak resources, so errors freeing resources has to be critical feel free to do retries and other things to avoid returning an error that we can't handle.
Note, all methods on this must be thread-safe, an implementation of most of these methods can be obtained by extending DisposableResource.
type DisposableResource ¶
type DisposableResource struct {
// contains filtered or unexported fields
}
The DisposableResource implements as thread-safe reference counted resource for the Disposable interface. Such that Acquire/Release updates the LastUsed time stamp.
Implementors of the Disposable interface using this base struct should implement Dispose such that it returns ErrDisposableSizeNotSupported if currently in use. This can be easily actieved using CanDispose.
Implementor should also implement EstimateMemorySize() and EstimateDiskSize(), but these are unrelated to this trait.
func (*DisposableResource) Acquire ¶
func (r *DisposableResource) Acquire()
Acquire the resource incrementing the reference count by one
func (*DisposableResource) CanDispose ¶
func (r *DisposableResource) CanDispose() error
CanDispose returns ErrDisposableInUse if the resource is currently being used. This is intended to be used by implementors of Dispose.
func (*DisposableResource) DiskSize ¶ added in v0.0.2
func (r *DisposableResource) DiskSize() (uint64, error)
DiskSize is the stub implementation of Disposable.DiskSize returning ErrDisposableSizeNotSupported, implementors really ought to overwrite this.
func (*DisposableResource) LastUsed ¶
func (r *DisposableResource) LastUsed() time.Time
LastUsed returns the last time this resource was used, as an implementation for the Disposable interface
func (*DisposableResource) MemorySize ¶ added in v0.0.2
func (r *DisposableResource) MemorySize() (uint64, error)
MemorySize is the stub implementation of Disposable.MemorySize returning ErrDisposableSizeNotSupported, implementors really ought to overwrite this.
func (*DisposableResource) Release ¶
func (r *DisposableResource) Release()
Release the resource decrementing the reference count by one and updating the lastUsed time stamp to now.
type GarbageCollector ¶
type GarbageCollector struct {
// contains filtered or unexported fields
}
GarbageCollector can be used register Disposable resources which will then be diposed when not in use and the system is low on available disk space or memory.
func New ¶ added in v0.0.2
func New(storageFolder string, minimumDiskSpace, minimumMemory int64) *GarbageCollector
New creates a GarbageCollector which uses storageFolder to test for available diskspace and tries to ensure that minimumDiskSpace and minimumMemory is satisfied after each call to Collect()
func (*GarbageCollector) Collect ¶
func (gc *GarbageCollector) Collect() error
Collect runs garbage collection and reclaims resources, attempting to satisfy minimumMemory and minimumDiskSpace, if possible.
func (*GarbageCollector) CollectAll ¶ added in v0.0.2
func (gc *GarbageCollector) CollectAll() error
CollectAll disposes all resources that can be disposed.
All resources not returning: ErrDisposableInUse. This is useful for testing when implementing resources.
func (*GarbageCollector) Register ¶
func (gc *GarbageCollector) Register(resource Disposable)
Register takes a Disposable resource for the GarbageCollector to manage.
GarbageCollector will attempt to to call resource.Dispose() at any time, you should return ErrDisposableInUse if the resource is in use.
func (*GarbageCollector) Unregister ¶
func (gc *GarbageCollector) Unregister(resource Disposable) bool
Unregister will inform the GarbageCollector to stop tracking the given resource. Returns true if the resource was tracked, notice that the resource maybe have been disposed of while waiting for the GC lock. Say if the GC was running when you made this call.
Note, you don't have to use this method. When you resouce is in a state where you don't want it to be disposed just ensure that Dispose() returns ErrDisposableInUse.
type ResourceTracker ¶ added in v0.0.2
type ResourceTracker interface {
Register(resource Disposable)
Unregister(resource Disposable) bool
}
A ResourceTracker is an object capable of tracking resources.
This is the interface for the GarbageCollector that should be exposed to engines and plugins. So they can't initiate garbage collection.
Notes ¶
Bugs ¶
Not sure this base struct is useful at all...
Source Files