Documentation ¶
Overview ¶
Bit operations. Used for arranging bits for better compression
Simple file based Fixed size Record Storage implementation, syncronized copy on write+ filesync
Memory based loop For testing, volatile storage etc...
Misc util functions
Index ¶
- Constants
- type FileStorage
- func (p *FileStorage) GetFirst(nRecords int64) ([]byte, error)
- func (p *FileStorage) GetLatest(nRecords int64) ([]byte, error)
- func (p *FileStorage) Len() (int64, error)
- func (p *FileStorage) Read(arr []byte) (n int, err error)
- func (p *FileStorage) ReadAll() ([]byte, error)
- func (p *FileStorage) Seek(offset int64, whence int) (int64, error)
- func (p *FileStorage) Write(raw []byte) (n int, err error)
- type FileStorageConf
- func (p *FileStorageConf) BaseFileName() string
- func (p *FileStorageConf) CheckErrors() error
- func (p *FileStorageConf) GetNumberRangeOnDisk() (int64, int64, int64, error)
- func (p *FileStorageConf) InitFileStorage() (FileStorage, error)
- func (p *FileStorageConf) ReadFileWithNumber(fileNumber int64) ([]byte, error)
- type FixRegSto
- type Memloop
- func (p *Memloop) GetFirst(nRecords int64) ([]byte, error)
- func (p *Memloop) GetLatest(nRecords int64) ([]byte, error)
- func (p *Memloop) Len() (int64, error)
- func (p *Memloop) Read(arr []byte) (n int, err error)
- func (p *Memloop) ReadAll() ([]byte, error)
- func (p *Memloop) Seek(offset int64, whence int) (int64, error)
- func (p *Memloop) Write(raw []byte) (n int, err error)
- type MemloopConf
Constants ¶
const COMPRESSIONMETHOD_GZ = "gz" //TODO other compression methods?
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FileStorage ¶
type FileStorage struct {
// contains filtered or unexported fields
}
FileStorage, includes conf and cached data
func (*FileStorage) GetFirst ¶
func (p *FileStorage) GetFirst(nRecords int64) ([]byte, error)
GetFirst nRecords without moving seek cursor
func (*FileStorage) Len ¶
func (p *FileStorage) Len() (int64, error)
Len returns how many records are stored
func (*FileStorage) Read ¶
func (p *FileStorage) Read(arr []byte) (n int, err error)
Read implements Reader interface. Except only array length must be multiple of recordsize for normal operation
func (*FileStorage) ReadAll ¶
func (p *FileStorage) ReadAll() ([]byte, error)
ReadAll gets all content. Use with caution, small storages
type FileStorageConf ¶
type FileStorageConf struct { Name string //Numbering _0, _1,_2 etc.. RecordSize int64 //One entry is this long, prefer power of two MaxFileCount int64 //TODO if 0? no at least 1 FileMaxSize int64 //How many bytes. Prefer multiple of 512 (erase blocks size optimal) Path string //Compression settings CompressionMethod string //Empty or "gz" BitSlices []int //Empty array no slicing.Else give bitlengths (usually bit size of each variable in record) }
FileStorageConf tells what kind of FileStorage instance is going to be created Name, is prefix for filename. Remember not to use same name (and path) in other databases or files RecordSize, is what your application needs (size of struct in bytes?) MaxFileCount, how many storage files exists on disk (plus "work file" without number) FileMaxSize, how many bytes one file takes up. Prefer some multiple of minimum file size on filesystem Path, path to directory where data is stored
func (*FileStorageConf) BaseFileName ¶
func (p *FileStorageConf) BaseFileName() string
func (*FileStorageConf) CheckErrors ¶
func (p *FileStorageConf) CheckErrors() error
CheckErrors tell is there problems with configuration
func (*FileStorageConf) GetNumberRangeOnDisk ¶
func (p *FileStorageConf) GetNumberRangeOnDisk() (int64, int64, int64, error)
getNumberRangeOnDisk, function gets minimum and maximum number in storage files and count of files (count is important if missing files in between? Also decides is delete needed)
func (*FileStorageConf) InitFileStorage ¶
func (p *FileStorageConf) InitFileStorage() (FileStorage, error)
InitFileStorage, Call this method after creating FileStorageConf. This creates dir if required
func (*FileStorageConf) ReadFileWithNumber ¶
func (p *FileStorageConf) ReadFileWithNumber(fileNumber int64) ([]byte, error)
Uses conf. Does not include state like FileStorage
type FixRegSto ¶
type FixRegSto interface { Write(raw []byte) (n int, err error) //size must be recordsize*N Len() (int64, error) //Number of records GetLatest(nRecords int64) ([]byte, error) //Without chancing read pointer GetFirst(nRecords int64) ([]byte, error) //Without chancing read pointer Seek(offset int64, whence int) (int64, error) Read(arr []byte) (n int, err error) ReadAll() ([]byte, error) }
FixRegSto implements ReadWriteSeeker interface
type Memloop ¶
type Memloop struct {
// contains filtered or unexported fields
}
type MemloopConf ¶
func (*MemloopConf) InitMemLoop ¶
func (p *MemloopConf) InitMemLoop() (Memloop, error)