Documentation
¶
Overview ¶
Package chancacher implements a pipeline of channels (in->out) that provides internal buffering (via a simple buffered channel), and caching data to disk.
Index ¶
Constants ¶
const MaxDepth = 1000000
The maximum channel depth, which is also used when the channel depth is set to 0. We could set this to MaxInt but we'd likely just run out of memory without a clean way to triage. It's best to just enforce a sensible maximum.
const TIMEOUT = time.Second
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ChanCacher ¶
type ChanCacher struct {
In chan interface{}
Out chan interface{}
// contains filtered or unexported fields
}
A ChanCacher is a pipeline of channels with a variable-sized internal buffer. The buffer can also cache to disk. The user is expected to connect ChanCacher.In and ChanCacher.Out.
func NewChanCacher ¶
func NewChanCacher(maxDepth int, cachePath string) *ChanCacher
Create a new ChanCacher with maximum depth, and optional backing file. If maxDepth == 0, the ChanCacher will be unbuffered. If maxDepth == -1, the ChanCacher depth will be set to MaxDepth. To enable a backing store, provide a path to backingPath. chancachers create two files using this prefix named cache_a and cache_b.
When a new ChanCacher is made, if cachePath points to existing cache files, the ChanCacher will immediately attempt to drain them from disk. In this way, you can recover data sent to disk on a crash or previous use of Commit().
func (*ChanCacher) BufferSize ¶
func (c *ChanCacher) BufferSize() int
Returns the number of elements on the internal buffer.
func (*ChanCacher) CacheHasData ¶
func (c *ChanCacher) CacheHasData() bool
Return if the cache has outstanding data not written to the output channel.
func (*ChanCacher) CacheStop ¶
func (c *ChanCacher) CacheStop()
Stop a running cache. Calling Stop() will prevent the ChanCacher from writing any new data to the backing file, but will not stop it from reading (draining) the cache to the output channel.
func (*ChanCacher) Commit ¶
func (c *ChanCacher) Commit()
Commit drains the buffer to the backing file and shuts down the cache. Commit should be called after closing the input channel if the buffer needs to be saved. Commit will block until the In channel is closed. The ChanCacher will not close the output channel until it's empty, so a typical production would look like:
close(c.In) drainSomeDataFrom(c.Out) // commit the rest of my data to disk c.Commit() // c.Out is now closed
Once Commit() is called, draining the cache cannot be restarted, though writing to the cache will still work. Commit should only be used for teardown scenarios.
func (*ChanCacher) Drain ¶
func (c *ChanCacher) Drain()
Drain blocks until the internal buffer is empty. It's possible that new data is still being consumed, so care should be taken when using Drain(). You probably don't want to use Drain(), but instead close ChanCacher.In and wait for the ChanCacher.Out to close, which does carry guarantees that the internal buffers and cache are fully drained.
Source Files