file_cache

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 20, 2023 License: MIT Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// Check for file expiry in below number of seconds
	CacheTimeoutCheckInterval = 5

	// Check for disk usage in below number of minutes
	DiskUsageCheckInterval = 1
)
View Source 
const DefaultEvictTime = 10
View Source 
const (
	MB = 1024 * 1024
)

Variables

This section is empty.

Functions

func NewFileCacheComponent 

func NewFileCacheComponent() internal.Component

Pipeline will call this method to create your object, initialize your variables here << DO NOT DELETE ANY AUTO GENERATED CODE HERE >>

func NewLFUPolicy 

func NewLFUPolicy(cfg cachePolicyConfig) cachePolicy

func NewLRUPolicy 

func NewLRUPolicy(cfg cachePolicyConfig) cachePolicy

Types

type FileCache 

type FileCache struct {
	internal.BaseComponent
	// contains filtered or unexported fields
}

Common structure for Component

func (*FileCache) Chmod 

func (fc *FileCache) Chmod(options internal.ChmodOptions) error

Chmod : Update the file with its new permissions

func (*FileCache) Chown 

func (fc *FileCache) Chown(options internal.ChownOptions) error

Chown : Update the file with its new owner and group

func (*FileCache) CloseFile 

func (fc *FileCache) CloseFile(options internal.CloseFileOptions) error

CloseFile: Flush the file and invalidate it from the cache.

func (*FileCache) Configure 

func (c *FileCache) Configure(_ bool) error

Configure : Pipeline will call this method after constructor so that you can read config and initialize yourself 

Return failure if any config is not valid to exit the process

func (*FileCache) CreateFile 

func (fc *FileCache) CreateFile(options internal.CreateFileOptions) (*handlemap.Handle, error)

CreateFile: Create the file in local cache.

func (*FileCache) DeleteDir 

func (fc *FileCache) DeleteDir(options internal.DeleteDirOptions) error

DeleteDir: Recursively invalidate the directory and its children

func (*FileCache) DeleteFile 

func (fc *FileCache) DeleteFile(options internal.DeleteFileOptions) error

DeleteFile: Invalidate the file in local cache.

func (*FileCache) FileUsed 

func (fc *FileCache) FileUsed(name string) error

func (*FileCache) FlushFile 

func (fc *FileCache) FlushFile(options internal.FlushFileOptions) error

FlushFile: Flush the local file to storage

func (*FileCache) GetAttr 

func (fc *FileCache) GetAttr(options internal.GetAttrOptions) (*internal.ObjAttr, error)

GetAttr: Consolidate attributes from storage and local cache

func (*FileCache) GetPolicyConfig 

func (c *FileCache) GetPolicyConfig(conf FileCacheOptions) cachePolicyConfig

func (*FileCache) IsDirEmpty 

func (fc *FileCache) IsDirEmpty(options internal.IsDirEmptyOptions) bool

IsDirEmpty: Whether or not the directory is empty

func (*FileCache) Name 

func (c *FileCache) Name() string

func (*FileCache) OnConfigChange 

func (c *FileCache) OnConfigChange()

OnConfigChange : If component has registered, on config file change this method is called

func (*FileCache) OpenFile 

func (fc *FileCache) OpenFile(options internal.OpenFileOptions) (*handlemap.Handle, error)

OpenFile: Makes the file available in the local cache for further file operations.

func (*FileCache) Priority 

func (c *FileCache) Priority() internal.ComponentPriority

func (*FileCache) ReadDir 

func (fc *FileCache) ReadDir(options internal.ReadDirOptions) ([]*internal.ObjAttr, error)

ReadDir: Consolidate entries in storage and local cache to return the children under this path.

func (*FileCache) ReadFile 

func (fc *FileCache) ReadFile(options internal.ReadFileOptions) ([]byte, error)

ReadFile: Read the local file

func (*FileCache) ReadInBuffer 

func (fc *FileCache) ReadInBuffer(options internal.ReadInBufferOptions) (int, error)

ReadInBuffer: Read the local file into a buffer

func (*FileCache) RenameDir 

func (fc *FileCache) RenameDir(options internal.RenameDirOptions) error

RenameDir: Recursively invalidate the source directory and its children

func (*FileCache) RenameFile 

func (fc *FileCache) RenameFile(options internal.RenameFileOptions) error

RenameFile: Invalidate the file in local cache.

func (*FileCache) SetName 

func (c *FileCache) SetName(name string)

func (*FileCache) SetNextComponent 

func (c *FileCache) SetNextComponent(nc internal.Component)

func (*FileCache) Start 

func (c *FileCache) Start(ctx context.Context) error

Start : Pipeline calls this method to start the component functionality 

this shall not block the call otherwise pipeline will not start

func (*FileCache) StatFs 

func (c *FileCache) StatFs() (*common.Statfs_t, bool, error)

func (*FileCache) Stop 

func (c *FileCache) Stop() error

Stop : Stop the component functionality and kill all threads started

func (*FileCache) StreamDir 

func (fc *FileCache) StreamDir(options internal.StreamDirOptions) ([]*internal.ObjAttr, string, error)

StreamDir : Add local files to the list retrieved from storage container

func (*FileCache) SyncFile 

func (fc *FileCache) SyncFile(options internal.SyncFileOptions) error

func (*FileCache) TempCacheCleanup 

func (c *FileCache) TempCacheCleanup() error

func (*FileCache) TruncateFile 

func (fc *FileCache) TruncateFile(options internal.TruncateFileOptions) error

TruncateFile: Update the file with its new size.

func (*FileCache) WriteFile 

func (fc *FileCache) WriteFile(options internal.WriteFileOptions) (int, error)

WriteFile: Write to the local file

type FileCacheOptions 

type FileCacheOptions struct {
	// e.g. var1 uint32 `config:"var1"`
	TmpPath string `config:"path" yaml:"path,omitempty"`
	Policy  string `config:"policy" yaml:"policy,omitempty"`

	Timeout     uint32 `config:"timeout-sec" yaml:"timeout-sec,omitempty"`
	MaxEviction uint32 `config:"max-eviction" yaml:"max-eviction,omitempty"`

	MaxSizeMB     float64 `config:"max-size-mb" yaml:"max-size-mb,omitempty"`
	HighThreshold uint32  `config:"high-threshold" yaml:"high-threshold,omitempty"`
	LowThreshold  uint32  `config:"low-threshold" yaml:"low-threshold,omitempty"`

	CreateEmptyFile bool `config:"create-empty-file" yaml:"create-empty-file,omitempty"`
	AllowNonEmpty   bool `config:"allow-non-empty-temp" yaml:"allow-non-empty-temp,omitempty"`
	CleanupOnStart  bool `config:"cleanup-on-start" yaml:"cleanup-on-start,omitempty"`

	EnablePolicyTrace bool `config:"policy-trace" yaml:"policy-trace,omitempty"`
	OffloadIO         bool `config:"offload-io" yaml:"offload-io,omitempty"`

	// v1 support
	V1Timeout     uint32 `config:"file-cache-timeout-in-seconds" yaml:"-"`
	EmptyDirCheck bool   `config:"empty-dir-check" yaml:"-"`
	SyncToFlush   bool   `config:"sync-to-flush" yaml:"sync-to-flush"`
	SyncNoOp      bool   `config:"ignore-sync" yaml:"ignore-sync,omitempty"`

	RefreshSec uint32 `config:"refresh-sec" yaml:"refresh-sec,omitempty"`
	HardLimit  bool   `config:"hard-limit" yaml:"hard-limit,omitempty"`
}

Structure defining your config parameters

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.