Documentation ¶
Overview ¶
Package tarcache supports the creation and running of a pipeline that receives files, tars up the contents, and uploads everything when the tarfile is big enough or the contents are old enough.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type TarCache ¶
type TarCache struct {
// contains filtered or unexported fields
}
TarCache contains everything you need to incrementally create a tarfile. Once enough time has passed since the first file was added OR the resulting tar file has become big enough, it will call the uploadAndDelete() method. To upload a lot of tarfiles, you should only have to create one TarCache. The TarCache takes care of creating each tarfile and getting it uploaded.
func New ¶
func New(rootDirectory filename.System, datatype string, ratio float64, metadata *flagx.KeyValue, sizeThreshold bytecount.ByteCount, ageThreshold memoryless.Config, uploader uploader.Uploader) (*TarCache, chan<- filename.System)
New creates a new TarCache object and returns a pointer to it and the channel used to send data to the TarCache.
func (*TarCache) ListenForever ¶
ListenForever waits for new files and then uploads them. Using this approach allows us to ensure that all file processing happens in this single thread, no matter whether the processing is happening due to age thresholds or size thresholds.