db19

package
v0.0.0-...-c9bb408 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 19, 2024 License: MIT Imports: 36 Imported by: 0

Documentation

Index

Constants

View Source
const MaxTrans = 500 // ???

MaxTrans is the maximum number of outstanding/overlapping update transactions

Variables

View Source
var MakeSuTran func(tran *UpdateTran) *SuTran

MakeSuTran is injected by dbms to avoid import cycle

View Source
var MaxAge = 20

MaxAge is the maximum number of ticks that a transaction can be outstanding. Transactions are aborted if they exceed this limit.

Functions

func CheckDatabase

func CheckDatabase(dbfile string) (ec error)

CheckDatabase is called by -check and -repair

func CheckOtherIndex

func CheckOtherIndex(ixcols []string, ix *index.Overlay, nrows int, sumPrev uint64)

func MakeLess

func MakeLess(store *stor.Stor, is *ixkey.Spec) func(x, y uint64) bool

MakeLess handles _lower! but not rules. It is used for indexes (which don't support rules).

func OffToRec

func OffToRec(store *stor.Stor, off uint64) core.Record

func OffToRecCk

func OffToRecCk(store *stor.Stor, off uint64) core.Record

OffToRecCk verifies the checksum following the record

func Repair

func Repair(dbfile string, err error) (string, error)

func StartConcur

func StartConcur(db *Database, persistInterval time.Duration)

StartConcur starts the database pipeline - starting the goroutines and connecting them with channels.

checker -> merger

persist is called by merger every persistInterval

To stop we close the checker channel, and then each following stage closes its output channel. Finally the merger closes the allDone channel so we know the shutdown has finished.

func StartTimestamps

func StartTimestamps()

StartTimestamp is called by gsuneido.go openDbms

func Timestamp

func Timestamp() SuDate

Timestamp is the backend. See also Thread.Timestamp

Types

type Check

type Check struct {
	// contains filtered or unexported fields
}

Check holds the data for the transaction conflict checker. Checking is designed to be single threaded i.e. run in its own goroutine. It is intended to run asynchronously, i.e. callers not waiting for results. This allow more concurrency (overlap) with user code. Actions are checked as they are done, incrementally. A conflict with a completed transaction aborts the current transaction. A conflict with an outstanding (not completed) transaction randomly aborts one of the two transactions. The checker serializes transaction commits. A single sequence counter is used to assign unique start and end values. See CheckCo for the concurrent channel based interface to Check. See Checker for the common interface to Check and CheckCo

func NewCheck

func NewCheck(db *Database) *Check

func (*Check) Abort

func (ck *Check) Abort(t *CkTran, reason string) bool

Abort cancels a transaction. It returns false if the transaction is not found (e.g. already aborted).

func (*Check) AddExclusive

func (ck *Check) AddExclusive(table string) bool

AddExclusive is used for creating indexes on existing tables

func (*Check) Commit

func (ck *Check) Commit(ut *UpdateTran) bool

Commit finishes a transaction. It returns false if the transaction is not found (e.g. already aborted). No additional checking required since actions have already been checked.

func (*Check) Delete

func (ck *Check) Delete(t *CkTran, table string, off uint64, keys []string) bool

Delete adds a delete action. Outputs only need to be checked against outstanding reads. The keys are parallel with the indexes i.e. keys[i] is for indexes[i].

func (*Check) EndExclusive

func (ck *Check) EndExclusive(table string)

func (*Check) Final

func (ck *Check) Final() int

Final returns the count of committed transactions overlapping with outstanding

func (*Check) Output

func (ck *Check) Output(t *CkTran, table string, keys []string) bool

Output adds an output action. Outputs only need to be checked against outstanding reads. The keys are parallel with the indexes i.e. keys[i] is for indexes[i].

func (*Check) Persist

func (ck *Check) Persist() *DbState

Persist is just for tests, it doesn't actually persist

func (*Check) Read

func (ck *Check) Read(t *CkTran, table string, index int, from, to string) bool

Read adds a read action. Will conflict if another transaction has a write within the range.

func (*Check) ReadCount

func (ck *Check) ReadCount(t *CkTran) int

func (*Check) Run

func (ck *Check) Run(fn func() error) error

func (*Check) RunEndExclusive

func (ck *Check) RunEndExclusive(table string, fn func()) (err any)

func (*Check) RunExclusive

func (ck *Check) RunExclusive(table string, fn func()) any

func (*Check) StartTran

func (ck *Check) StartTran() *CkTran

func (*Check) Stop

func (ck *Check) Stop()

func (*Check) Transactions

func (ck *Check) Transactions() []int

Transactions returns a list of the active update transactions

func (*Check) Update

func (ck *Check) Update(t *CkTran,
	table string, oldoff uint64, oldkeys, newkeys []string) bool

Update adds a delete of oldoff/oldkeys and an output of newkeys.

type CheckCo

type CheckCo struct {
	// contains filtered or unexported fields
}

CheckCo is the concurrent, channel based interface to Check

func StartCheckCo

func StartCheckCo(db *Database, mergeChan chan todo, allDone chan void) *CheckCo

func (*CheckCo) Abort

func (ck *CheckCo) Abort(t *CkTran, reason string) bool

func (*CheckCo) AddExclusive

func (ck *CheckCo) AddExclusive(table string) bool

AddExclusive is used by load table and add index

func (*CheckCo) Commit

func (ck *CheckCo) Commit(ut *UpdateTran) bool

func (*CheckCo) Delete

func (ck *CheckCo) Delete(t *CkTran, table string, off uint64, keys []string) bool

func (*CheckCo) EndExclusive

func (ck *CheckCo) EndExclusive(table string)

func (*CheckCo) Final

func (ck *CheckCo) Final() int

func (*CheckCo) Output

func (ck *CheckCo) Output(t *CkTran, table string, keys []string) bool

func (*CheckCo) Persist

func (ck *CheckCo) Persist() *DbState

func (*CheckCo) Read

func (ck *CheckCo) Read(t *CkTran, table string, index int, from, to string) bool

func (*CheckCo) ReadCount

func (ck *CheckCo) ReadCount(t *CkTran) int

func (*CheckCo) RunEndExclusive

func (ck *CheckCo) RunEndExclusive(table string, fn func()) any

func (*CheckCo) RunExclusive

func (ck *CheckCo) RunExclusive(table string, fn func()) any

func (*CheckCo) StartTran

func (ck *CheckCo) StartTran() *CkTran

func (*CheckCo) Stop

func (ck *CheckCo) Stop()

func (*CheckCo) Transactions

func (ck *CheckCo) Transactions() []int

func (*CheckCo) Update

func (ck *CheckCo) Update(t *CkTran, table string, oldoff uint64, oldkeys, newkeys []string) bool

type Checker

type Checker interface {
	StartTran() *CkTran
	Read(t *CkTran, table string, index int, from, to string) bool
	Output(t *CkTran, table string, keys []string) bool
	Delete(t *CkTran, table string, off uint64, keys []string) bool
	Update(t *CkTran, table string, oldoff uint64, oldkeys, newkeys []string) bool
	ReadCount(t *CkTran) int
	Abort(t *CkTran, reason string) bool
	Commit(t *UpdateTran) bool
	Persist() *DbState
	Stop()
	Transactions() []int
	Final() int
	AddExclusive(table string) bool
	EndExclusive(table string)
	RunEndExclusive(table string, fn func()) any
	RunExclusive(table string, fn func()) any
}

Checker is the interface for Check and CheckCo

type CkTran

type CkTran struct {
	// contains filtered or unexported fields
}

func (*CkTran) Failed

func (t *CkTran) Failed() bool

func (*CkTran) String

func (t *CkTran) String() string

type Database

type Database struct {
	Store *stor.Stor
	// contains filtered or unexported fields
}

func CreateDatabase

func CreateDatabase(filename string) (*Database, error)

CreateDatabase creates an empty database in the named file. NOTE: The returned Database does not have a checker.

func CreateDb

func CreateDb(store *stor.Stor) (*Database, error)

CreateDb creates an empty database in the store. NOTE: The returned Database does not have a checker.

func OpenDatabase

func OpenDatabase(filename string) (*Database, error)

OpenDatabase opens the database in the named file for read & write. NOTE: The returned Database does not have a checker yet.

func OpenDb

func OpenDb(filename string, mode stor.Mode, check bool) (db *Database, err error)

OpenDb opens the database in the named file. NOTE: The returned Database does not have a checker.

func OpenDbStor

func OpenDbStor(store *stor.Stor, mode stor.Mode, check bool) (db *Database, err error)

OpenDbStor opens the database in the store. NOTE: The returned Database does not have a checker.

func (*Database) AddExclusive

func (db *Database) AddExclusive(table string)

func (*Database) AddNewTable

func (db *Database) AddNewTable(ts *meta.Schema, ti *meta.Info)

AddNewTable is used by compact and loading entire database. It panics if the table already exists.

func (*Database) AddView

func (db *Database) AddView(name, def string) bool

func (*Database) AlterCreate

func (db *Database) AlterCreate(sch *schema.Schema)

AlterCreate creates columns or indexes

func (*Database) AlterDrop

func (db *Database) AlterDrop(schema *schema.Schema) bool

AlterDrop removes columns or indexes

func (*Database) AlterRename

func (db *Database) AlterRename(table string, from, to []string) bool

AlterRename renames columns

func (*Database) CallTrigger

func (t *Database) CallTrigger(th *Thread, tran *UpdateTran, table string,
	oldrec, newrec Record)

func (*Database) Check

func (db *Database) Check() (ec error)

Check is called by the builtin Database.Check()

func (*Database) CheckAllFkeys

func (db *Database) CheckAllFkeys()

CheckAllFkeys is used after loading an entire database.

func (*Database) CheckerSync

func (db *Database) CheckerSync()

CheckerSync is for tests. It assigns a synchronous transaction checker to the database.

func (*Database) Close

func (db *Database) Close()

Close closes the database store, writing the current size to the start.

func (*Database) CloseKeepMapped

func (db *Database) CloseKeepMapped()

func (*Database) Closed

func (db *Database) Closed() bool

func (*Database) CommitMerge

func (db *Database) CommitMerge(ut *UpdateTran)

CommitMerge is for tests. It merges synchronously after each commit.

func (*Database) Corrupt

func (db *Database) Corrupt()

Corrupt marks the database as corrupted.

func (*Database) Corrupted

func (db *Database) Corrupted() bool

func (*Database) Create

func (db *Database) Create(schema *schema.Schema)

func (*Database) DisableTrigger

func (t *Database) DisableTrigger(table string)

func (*Database) Drop

func (db *Database) Drop(table string) error

Drop removes a table or view

func (*Database) EnableTrigger

func (t *Database) EnableTrigger(table string)

func (*Database) EndExclusive

func (db *Database) EndExclusive(table string)

func (*Database) Ensure

func (db *Database) Ensure(sch *schema.Schema)

func (*Database) Final

func (db *Database) Final() int

func (*Database) GetState

func (db *Database) GetState() *DbState

GetState returns a snapshot of the state as of a point in time. This state must be treated as read-only and must not be modified. To modify the state use UpdateState.

GetState is atomic, it is not blocked by UpdateState.

func (*Database) GetView

func (db *Database) GetView(name string) string

func (*Database) HaveUsers

func (db *Database) HaveUsers() bool

func (*Database) Merge

func (db *Database) Merge(fn mergefn, merges *mergeList)

Merge updates the base ixbuf's with the ones from transactions It is called by concur.go merger.

func (*Database) MustCheck

func (db *Database) MustCheck()

func (*Database) NewReadTran

func (db *Database) NewReadTran() *ReadTran

func (*Database) NewUpdateTran

func (db *Database) NewUpdateTran() *UpdateTran

func (*Database) OverwriteTable

func (db *Database) OverwriteTable(ts *meta.Schema, ti *meta.Info)

OverwriteTable is used when loading a single table. If the table already exists it is replaced.

func (*Database) Persist

func (db *Database) Persist() *DbState

Persist forces a persist and returns a persisted state, with all ixbuf layer entries merged into the btree. This is used by dump, load, and checkdb.

func (*Database) PersistSync

func (db *Database) PersistSync()

PersistSync is for tests

func (*Database) QuickCheck

func (db *Database) QuickCheck() (err error)

QuickCheck is the default partial checking done at start up.

func (*Database) RenameTable

func (db *Database) RenameTable(from, to string) bool

func (*Database) RunEndExclusive

func (db *Database) RunEndExclusive(table string, fn func())

func (*Database) RunExclusive

func (db *Database) RunExclusive(table string, fn func())

func (*Database) Schema

func (db *Database) Schema(table string) string

func (*Database) Size

func (db *Database) Size() uint64

func (*Database) Transactions

func (db *Database) Transactions() []int

Transactions only returns the update transactions. It returns nil if corrupted.

func (*Database) UpdateState

func (db *Database) UpdateState(fn func(*DbState))

UpdateState applies the given update function to a copy of theState and sets theState to the result. Guarded by stateMutex so only one thread can execute at a time. Note: the state passed to the update function is a *shallow* copy, it is up to the function to make copies of any nested containers.

UpdateState is guarded by a mutex

type DbState

type DbState struct {
	Meta *meta.Meta
	Asof int64  // unix milli time
	Off  uint64 // offset of this state
	// contains filtered or unexported fields
}

func NextState

func NextState(store *stor.Stor, off uint64) *DbState

func PrevState

func PrevState(store *stor.Stor, off uint64) *DbState

func ReadState

func ReadState(st *stor.Stor, off uint64) *DbState

func StateAsof

func StateAsof(store *stor.Stor, asof int64) *DbState

StateAsof returns the state <= asof, or the initial state.

func (*DbState) Write

func (state *DbState) Write() uint64

type ErrCorrupt

type ErrCorrupt struct {
	// contains filtered or unexported fields
}

func (*ErrCorrupt) Error

func (ec *ErrCorrupt) Error() string

func (*ErrCorrupt) Table

func (ec *ErrCorrupt) Table() string

type Ranges

type Ranges = ranges.Ranges

type ReadTran

type ReadTran struct {
	// contains filtered or unexported fields
}

func (*ReadTran) Abort

func (t *ReadTran) Abort() string

func (*ReadTran) Asof

func (t *ReadTran) Asof(asof int64) int64

func (*ReadTran) ColToFld

func (t *ReadTran) ColToFld(table, col string) int

func (*ReadTran) Complete

func (t *ReadTran) Complete() string

func (*ReadTran) Delete

func (t *ReadTran) Delete(*core.Thread, string, uint64)

func (*ReadTran) GetAllInfo

func (t *ReadTran) GetAllInfo() []*meta.Info

func (*ReadTran) GetAllSchema

func (t *ReadTran) GetAllSchema() []*meta.Schema

func (*ReadTran) GetAllViews

func (t *ReadTran) GetAllViews() []string

func (*ReadTran) GetIndex

func (t *ReadTran) GetIndex(table string, cols []string) *index.Overlay

func (*ReadTran) GetIndexI

func (t *ReadTran) GetIndexI(table string, iIndex int) *index.Overlay

func (*ReadTran) GetInfo

func (t *ReadTran) GetInfo(table string) *meta.Info

GetInfo returns read-only Info for the table or nil if not found

func (*ReadTran) GetRecord

func (t *ReadTran) GetRecord(off uint64) core.Record

func (*ReadTran) GetSchema

func (t *ReadTran) GetSchema(table string) *schema.Schema

func (*ReadTran) GetStore

func (t *ReadTran) GetStore() *stor.Stor

func (*ReadTran) GetView

func (t *ReadTran) GetView(name string) string

func (*ReadTran) Lookup

func (t *ReadTran) Lookup(table string, iIndex int, key string) *core.DbRec

Lookup returns the DbRec for a key, or nil if not found

func (*ReadTran) MakeLess

func (t *ReadTran) MakeLess(is *ixkey.Spec) func(x, y uint64) bool

func (*ReadTran) Num

func (t *ReadTran) Num() int

func (*ReadTran) Output

func (t *ReadTran) Output(*core.Thread, string, core.Record)

func (*ReadTran) RangeFrac

func (t *ReadTran) RangeFrac(table string, iIndex int, org, end string) float64

func (*ReadTran) Read

func (t *ReadTran) Read(string, int, string, string)

func (*ReadTran) ReadCount

func (t *ReadTran) ReadCount() int

func (*ReadTran) String

func (t *ReadTran) String() string

func (*ReadTran) Update

func (t *ReadTran) Update(*core.Thread, string, uint64, core.Record) uint64

func (*ReadTran) WriteCount

func (t *ReadTran) WriteCount() int

type Set

type Set = ordset.Set

Set needs to be an ordered set so that reads can check for a range

type UpdateTran

type UpdateTran struct {
	ReadTran
	// contains filtered or unexported fields
}

func (*UpdateTran) Abort

func (t *UpdateTran) Abort() string

Abort returns "" if it succeeds or if the transaction was already aborted.

func (*UpdateTran) Commit

func (t *UpdateTran) Commit()

Commit is used by tests. It panics on error.

func (*UpdateTran) Complete

func (t *UpdateTran) Complete() string

Complete returns "" on success, otherwise an error

func (*UpdateTran) Delete

func (t *UpdateTran) Delete(th *core.Thread, table string, off uint64)

func (*UpdateTran) GetAllInfo

func (t *UpdateTran) GetAllInfo() []*meta.Info

func (*UpdateTran) GetAllSchema

func (t *UpdateTran) GetAllSchema() []*meta.Schema

func (*UpdateTran) GetAllViews

func (t *UpdateTran) GetAllViews() []string

func (*UpdateTran) GetInfo

func (t *UpdateTran) GetInfo(table string) *meta.Info

GetInfo returns read-only Info for the table or nil if not found

func (*UpdateTran) GetSchema

func (t *UpdateTran) GetSchema(table string) *schema.Schema

func (*UpdateTran) GetStore

func (t *UpdateTran) GetStore() *stor.Stor

func (*UpdateTran) GetView

func (t *UpdateTran) GetView(name string) string

func (*UpdateTran) Lookup

func (t *UpdateTran) Lookup(table string, iIndex int, key string) *core.DbRec

Lookup returns the DbRec for a key, or nil if not found

func (*UpdateTran) Num

func (t *UpdateTran) Num() int

func (*UpdateTran) Output

func (t *UpdateTran) Output(th *core.Thread, table string, rec core.Record)

func (*UpdateTran) Read

func (t *UpdateTran) Read(table string, iIndex int, from, to string)

Read adds a transaction read event to the checker

func (*UpdateTran) ReadCount

func (t *UpdateTran) ReadCount() int

func (*UpdateTran) String

func (t *UpdateTran) String() string

func (*UpdateTran) Update

func (t *UpdateTran) Update(th *core.Thread, table string, oldoff uint64, newrec core.Record) uint64

func (*UpdateTran) WriteCount

func (t *UpdateTran) WriteCount() int

Directories

Path Synopsis
Package filelock provides a platform-independent API for advisory file locking.
Package filelock provides a platform-independent API for advisory file locking.
Package index implements gSuneido database indexes.
Package index implements gSuneido database indexes.
ixbuf
Package ixbuf defines an ordered list ixbuf.T with a mutating Insert and a immutable persistent Merge.
Package ixbuf defines an ordered list ixbuf.T with a mutating Insert and a immutable persistent Merge.
ixkey
Package ixkey handles specifying and encoding index key strings that are directly comparable.
Package ixkey handles specifying and encoding index key strings that are directly comparable.
Package meta handles the database metadata.
Package meta handles the database metadata.
schema
Package schema is a separate package so it can be used by query parsing.
Package schema is a separate package so it can be used by query parsing.
Package stor is used to access physical storage, normally by memory mapped file access.
Package stor is used to access physical storage, normally by memory mapped file access.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL