file

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2022 License: MPL-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package file deals with low level file storage and transaction management.

StorageFile

StorageFile models a logical storage file which stores fixed size records on disk. Each record has a unique record id. On disk this logical storage file might be split into several smaller files. StorageFiles can be reused after they were closed if the transaction management has been disabled. This is not the case otherwise.

Record

A record is a byte slice of a StorageFile. It is a wrapper data structure for a byte array which provides read and write methods for several data types.

TransactionManager

TransactionManager provides an optional transaction management for StorageFile.

When used each record which is released from use is added to an in memory transaction log. Once the client calls Flush() on the StorageFile the in memory transaction is written to a transaction log on disk. The in-memory log is kept. The in-memory transaction log is written to the actual StorageFile once maxTrans is reached or the StorageFile is closed.

Should the process crash during a transaction, then the transaction log is written to the StorageFile on the next startup using the recover() function.

Index

Constants

View Source 
const (
	SizeByte          = 1
	SizeUnsignedShort = 2
	SizeShort         = 2
	SizeThreeByteInt  = 3
	SizeUnsignedInt   = 4
	SizeInt           = 4
	SizeSixByteLong   = 6
	SizeLong          = 8
)

Size constants for a record

View Source 
const DefaultFileSize = 0x2540BE401 // 10000000001 Bytes

DefaultFileSize is the default size of a physical file (10GB)

View Source 
const DefaultRecordSize = 4096

DefaultRecordSize is the default size of a record in bytes

View Source 
const DefaultTransInLog = 10

DefaultTransInLog is the default number of transactions which should be kept in memory (affects how often we sync the log from memory)

View Source 
const DefaultTransSize = 10

DefaultTransSize is the default number of records in a single transaction (affects how many record pointers are allocated at first per transaction)

View Source 
const LogFileSuffix = "tlg"

LogFileSuffix is the file suffix for transaction log files

Variables

View Source 
var (
	ErrAlreadyInUse  = errors.New("Record is already in-use")
	ErrNotInUse      = errors.New("Record was not in-use")
	ErrInUse         = errors.New("Records are still in-use")
	ErrTransDisabled = errors.New("Transactions are disabled")
	ErrInTrans       = errors.New("Records are still in a transaction")
	ErrNilData       = errors.New("Record has nil data")
)

Common storage file related errors.

View Source 
var (
	ErrBadMagic = fmt.Errorf("Bad magic for transaction log")
)

Common TransactionManager related errors

View Source 
var TransactionLogHeader = []byte{0x66, 0x42}

TransactionLogHeader is the magic number to identify transaction log files

Functions

This section is empty.

Types

type LogFile 

type LogFile interface {
	io.Writer
	io.Closer
	Sync() error
}

LogFile is the abstract interface for an transaction log file.

type Record 

type Record struct {
	// contains filtered or unexported fields
}

Record data structure

func NewRecord 

func NewRecord(id uint64, data []byte) *Record

NewRecord creates a new Record and returns a pointer to it.

func ReadRecord 

func ReadRecord(ior io.Reader) (*Record, error)

ReadRecord decodes a record by reading from an io.Reader.

func (*Record) ClearData 

func (r *Record) ClearData()

ClearData removes all stored data from a Record.

func (*Record) ClearDirty 

func (r *Record) ClearDirty()

ClearDirty clears the dirty flag of a Record.

func (*Record) Data 

func (r *Record) Data() []byte

Data returns the raw data of a Record.

func (*Record) DecTransCount 

func (r *Record) DecTransCount()

DecTransCount decrements the transaction count which means the record has been written to disk.

func (*Record) Dirty 

func (r *Record) Dirty() bool

Dirty returns the dirty flag of a Record.

func (*Record) ID 

func (r *Record) ID() uint64

ID returns the id of a Record.

func (*Record) InTransaction 

func (r *Record) InTransaction() bool

InTransaction returns if the Record is used in a transaction.

func (*Record) IncTransCount 

func (r *Record) IncTransCount()

IncTransCount increments the transaction count which means the record is in the log but not yet in the data file.

func (*Record) MarshalBinary 

func (r *Record) MarshalBinary() (data []byte, err error)

MarshalBinary returns a binary representation of a Record.

func (*Record) PageView 

func (r *Record) PageView() interface{}

PageView returns the view on this record. The view determines how the record is being used.

func (*Record) ReadInt16 

func (r *Record) ReadInt16(pos int) int16

ReadInt16 reads a 16-bit signed integer from a Record.

func (*Record) ReadInt32 

func (r *Record) ReadInt32(pos int) int32

ReadInt32 reads a 32-bit signed integer from a Record.

func (*Record) ReadRecord 

func (r *Record) ReadRecord(ior io.Reader) error

ReadRecord decodes a record by reading from an io.Reader.

func (*Record) ReadSingleByte 

func (r *Record) ReadSingleByte(pos int) byte

ReadSingleByte reads a byte from a Record.

func (*Record) ReadUInt16 

func (r *Record) ReadUInt16(pos int) uint16

ReadUInt16 reads a 16-bit unsigned integer from a Record.

func (*Record) ReadUInt32 

func (r *Record) ReadUInt32(pos int) uint32

ReadUInt32 reads a 32-bit unsigned integer from a Record.

func (*Record) ReadUInt64 

func (r *Record) ReadUInt64(pos int) uint64

ReadUInt64 reads a 64-bit unsigned integer from a Record.

func (*Record) SetDirty 

func (r *Record) SetDirty()

SetDirty sets the dirty flag of a Record.

func (*Record) SetID 

func (r *Record) SetID(id uint64) error

SetID changes the id of a Record.

func (*Record) SetPageView 

func (r *Record) SetPageView(view interface{})

SetPageView sets the view on this record.

func (*Record) String 

func (r *Record) String() string

String prints a string representation the Record.

func (*Record) UnmarshalBinary 

func (r *Record) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes a record from a binary blob.

func (*Record) WriteInt16 

func (r *Record) WriteInt16(pos int, value int16)

WriteInt16 writes a 16-bit signed integer to a Record.

func (*Record) WriteInt32 

func (r *Record) WriteInt32(pos int, value int32)

WriteInt32 writes a 32-bit signed integer to a Record.

func (*Record) WriteRecord 

func (r *Record) WriteRecord(iow io.Writer) error

WriteRecord writes a record to an io.Writer.

func (*Record) WriteSingleByte 

func (r *Record) WriteSingleByte(pos int, value byte)

WriteSingleByte writes a byte to a Record.

func (*Record) WriteUInt16 

func (r *Record) WriteUInt16(pos int, value uint16)

WriteUInt16 writes a 16-bit unsigned integer to a Record.

func (*Record) WriteUInt32 

func (r *Record) WriteUInt32(pos int, value uint32)

WriteUInt32 writes a 32-bit unsigned integer to a Record.

func (*Record) WriteUInt64 

func (r *Record) WriteUInt64(pos int, value uint64)

WriteUInt64 writes a 64-bit unsigned integer to a Record.

type StorageFile 

type StorageFile struct {
	// contains filtered or unexported fields
}

StorageFile data structure

func NewDefaultStorageFile 

func NewDefaultStorageFile(name string, transDisabled bool) (*StorageFile, error)

NewDefaultStorageFile creates a new storage file with default record size and returns a reference to it.

func NewStorageFile 

func NewStorageFile(name string, recordSize uint32, transDisabled bool) (*StorageFile, error)

NewStorageFile creates a new storage file and returns a reference to it.

func (*StorageFile) Close 

func (s *StorageFile) Close() error

Close commits all data and closes all physical files.

func (*StorageFile) Discard 

func (s *StorageFile) Discard(record *Record)

Discard a given record.

func (*StorageFile) Flush 

func (s *StorageFile) Flush() error

Flush commits the current transaction by flushing all dirty records to the transaction log on disk. If transactions are disabled it simply writes all dirty records to disk.

func (*StorageFile) Get 

func (s *StorageFile) Get(id uint64) (*Record, error)

Get returns a record from the file. Other components can write to this record. Any write operation should set the dirty flag on the record. Dirty records will be written back to disk when the file is flushed after which the dirty flag is cleared. Get panics if a record is requested which is still in-use.

func (*StorageFile) Name 

func (s *StorageFile) Name() string

Name returns the name of this storage file.

func (*StorageFile) RecordSize 

func (s *StorageFile) RecordSize() uint32

RecordSize returns the size of records which can be storerd or retrieved.

func (*StorageFile) ReleaseInUse 

func (s *StorageFile) ReleaseInUse(record *Record)

ReleaseInUse releases a record from the in-use map. ReleaseInUse panics if the record was not in use.

func (*StorageFile) ReleaseInUseID 

func (s *StorageFile) ReleaseInUseID(id uint64, dirty bool) error

ReleaseInUseID releases a record given by its id from the in-use map. The client code may indicate if the record is not dirty.

func (*StorageFile) Rollback 

func (s *StorageFile) Rollback() error

Rollback cancels the current transaction by discarding all dirty records.

func (*StorageFile) String 

func (s *StorageFile) String() string

String returns a string representation of a StorageFile.

func (*StorageFile) Sync 

func (s *StorageFile) Sync()

Sync syncs all physical files.

type StorageFileError added in v1.2.0 

type StorageFileError struct {
	Type     error
	Detail   string
	Filename string
}

StorageFileError is a storage file related error.

func NewStorageFileError added in v1.2.0 

func NewStorageFileError(sfeType error, sfeDetail string, sfeFilename string) *StorageFileError

NewStorageFileError returns a new StorageFile specific error.

func (*StorageFileError) Error added in v1.2.0 

func (e *StorageFileError) Error() string

Error returns a string representation of the error.

type TransactionManager 

type TransactionManager struct {
	// contains filtered or unexported fields
}

TransactionManager data structure

func NewTransactionManager 

func NewTransactionManager(owner *StorageFile, doRecover bool) (*TransactionManager, error)

NewTransactionManager creates a new transaction manager and returns a reference to it.

func (*TransactionManager) String 

func (t *TransactionManager) String() string

String returns a string representation of a TransactionManager.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.