zodb

package
v1.8.2-0...-ee23551 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 18, 2024 License: GPL-2.0, GPL-3.0 Imports: 24 Imported by: 1

Documentation

Overview

Package zodb provides API to work with ZODB databases.

ZODB (http://zodb.org) was originally created in Python world by Jim Fulton et al. Data model and API this package provides are partly based on ZODB/py (https://github.com/zopefoundation/ZODB) to maintain compatibility in between Python and Go implementations.

Data model

A ZODB database is conceptually modeled as transactional log of changes to objects. Oid identifies an object and Tid - a transaction. A transaction can change several objects and also has metadata, like user and description, associated with it. If an object is changed by transaction, it is said that there is revision of the object with particular object state committed by that transaction. Object revision is the same as tid of the transaction that modified the object. The combination of object identifier and particular revision (serial) uniquely addresses corresponding data record.

Tids of consecutive database transactions are monotonically increasing and are connected with time when transaction in question was committed. This way, besides identifying a transaction with changes, Tid can also be used to specify whole database state constructed by all cumulated transaction changes from database beginning up to, and including, transaction specified by it. Xid is "extended" oid that specifies particular object state: it is (oid, at) pair that is mapped to object's latest revision with serial ≤ at.

Object state data is generally opaque, but is traditionally based on Python pickles in ZODB/py world.

An object can reference other objects in the database by their oid.

Storage layer

The storage layer provides access to a ZODB database in terms of database records with raw bytes payload.

At storage level a ZODB database can be opened with Open. Once opened IStorage interface is returned that represents access to the database. Please see IStorage, and interfaces it embeds, for details.

Application layer

The application layer provides access to a ZODB database in terms of in-RAM application-level objects whose in-RAM state is synchronized with data in the database. For the synchronization to work, objects must be explicitly activated before access (contrary to zodb/py where activation is implicit, hooked into __getattr__), for example: 

var obj *MyObject // *MyObject must implement IPersistent (see below)
... // init obj pointer, usually by traversing from another persistent object.

// make sure object's in-RAM data is present.
//
// ZODB will load corresponding data and decode it into obj.
// On success, obj will be live and application can use its state.
err := obj.PActivate(ctx)
if err != nil {
	return ... // handle error
}

obj.xxx // use object.
if ... {
	obj.PModify() // let persistency layer know we are going to modify the object.
	obj.xxx++     // change the object.
}

// tell persistency layer we no longer need obj's in-RAM data to be present.
// if obj was not modified, its in-RAM state might go away after.
obj.PDeactivate()

IPersistent interface describes the details of the activation protocol.

For MyObject to implement IPersistent it must embed Persistent type. MyObject also has to register itself to persistency machinery with RegisterClass.

In-RAM application objects are handled in groups. During the scope of corresponding in-progress transaction(*), a group corresponds to particular view of the database (at) and has isolation guarantee from further database transactions, and from in-progress changes to in-RAM objects in other groups.

If object₁ references object₂ in the database, the database reference will be represented with corresponding reference between in-RAM application objects. If there are multiple database references to one object, it will be represented by the same number of references to only one in-RAM application object. An in-RAM application object can have reference to another in-RAM application object only from the same group(+). Reference cycles are also allowed. In general objects graph in the database is isomorphly mapped to application objects graph in RAM.

A particular view of the database together with corresponding group of application objects isolated for modifications is represented by Connection. Connection is also sometimes called a "jar" in ZODB terminology.

DB represents a handle to database at application level and contains pool of connections. DB.Open opens database connection. The connection will be automatically put back into DB pool for future reuse after corresponding transaction is complete. DB thus provides service to maintain live objects cache and reuse live objects from transaction to transaction.

Note that it is possible to have several DB handles to the same database. This might be useful if application accesses distinctly different sets of objects in different transactions and knows beforehand which set it will be next time. Then, to avoid huge live cache misses, it makes sense to keep DB handles opened for every possible case of application access.

All DB, Connection and object activation protocol is safe to access from multiple goroutines simultaneously.

--------

(*) see package lab.nexedi.com/kirr/neo/go/transaction. (+) if both objects are from the same database.

Python data

To maintain database data compatibility with ZODB/py, ZODB/go provides first class support for Python data. At storage-level PyData provides way to treat raw data record content as serialized by ZODB/py, and at application level types that are registered with state type providing PyStateful (see RegisterClass) are automatically (de)serialized as Python pickles(*).

An example of application-level type with ZODB/py compatibility can be seen in package lab.nexedi.com/kirr/neo/go/zodb/btree which provides BTree handling for ZODB/go.

--------

(*) for pickle support package github.com/kisielk/og-rek is used.

Storage drivers

To implement a ZODB storage one need to provide IStorageDriver interface and register it to ZODB with RegisterDriver. Package lab.nexedi.com/kirr/neo/go/zodb/wks links-in and registers support for well-known ZODB storages, such as FileStorage and ZEO.

Miscellaneous

See also package lab.nexedi.com/kirr/neo/go/zodb/zodbtools and associated zodb command that provide tools for managing ZODB databases.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ClassOf 

func ClassOf(obj IPersistent) string

ClassOf returns ZODB class of a Go object.

The following is returned:

  • if obj's type was registered (RegisterClass) -- corresponding class.
  • for Broken objects -- ZODB.Broken("<broken-class>").
  • else -- ZODB.Go("<fully-qualified-type(obj)>")

func OpenDriver 

func OpenDriver(ctx context.Context, zurl string, opt *DriverOptions) (_ IStorageDriver, at0 Tid, err error)

OpenDriver opens ZODB storage driver by URL.

It is similar to Open but returns low-level IStorageDriver instead of IStorage. Most users should use Open.

func RegisterClass 

func RegisterClass(class string, typ, stateType reflect.Type)

RegisterClass registers ZODB class to correspond to Go type.

Only registered classes can be saved to database, and are converted to corresponding application-level objects on load. When ZODB loads an object whose class is not known, it will represent it as Broken object.

class is a full class path for registered class, e.g. "BTrees.LOBTree.LOBucket". typ is Go type corresponding to class.

typ must embed Persistent; *typ must implement IPersistent.

typ must be convertible to stateType; stateType must implement Ghostable and either Stateful or PyStateful(*).

RegisterClass must be called from global init().

(*) the rationale for stateType coming separately is that this way for application types it is possible not to expose Ghostable and Stateful methods in their public API.

func RegisterClassAlias 

func RegisterClassAlias(alias, class string)

RegisterClassAlias registers alias for a ZODB class.

When ZODB loads an object whose class is alias, it will be handled like object with specified ZODB class.

Class aliases are useful for backward compatibility - sometimes class name of an object changes, but to support loading previously-saved objects, the old class name has to be also supported.

func RegisterDriver 

func RegisterDriver(scheme string, opener DriverOpener)

RegisterDriver registers opener to be used for URLs with scheme.

Types

type Broken 

type Broken struct {
	Persistent
	// contains filtered or unexported fields
}

Broken objects are used to represent loaded ZODB objects with classes that were not registered to zodb Go package.

See RegisterClass for details.

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache provides RAM caching layer that can be used over a storage.

func NewCache 

func NewCache(loader interface {
	Loader
	URL() string
}, sizeMax int) *Cache

NewCache creates new cache backed up by loader.

The cache will use not more than ~ sizeMax bytes of RAM for cached data.

func (*Cache) Load 

func (c *Cache) Load(ctx context.Context, xid Xid) (buf *mem.Buf, serial Tid, err error)

Load loads data from database via cache.

If data is already in cache - cached content is returned.

func (*Cache) Prefetch 

func (c *Cache) Prefetch(ctx context.Context, xid Xid)

Prefetch arranges for data to be eventually present in cache.

If data is not yet in cache loading for it is started in the background. Prefetch is not blocking operation and does not wait for loading, if any was started, to complete.

Prefetch does not return any error.

func (*Cache) SetSizeMax 

func (c *Cache) SetSizeMax(sizeMax int)

SetSizeMax adjusts how much RAM cache can use for cached data.

type Committer 

type Committer interface {
}

Committer provides functionality to commit transactions.

type ConnOptions 

type ConnOptions struct {
	At     Tid  // if !0, open Connection bound to `at` view of database; not latest.
	NoSync bool // don't sync with storage to get its last tid.

	// don't put connection back into DB pool after transaction ends.
	//
	// This is low-level option that allows to inspect/use connection's
	// LiveCache after transaction finishes, and to manually resync the
	// connection onto another database view. See Connection.Resync for
	// details.
	NoPool bool
}

ConnOptions describes options to DB.Open .

func (*ConnOptions) String 

func (opt *ConnOptions) String() string

String represents connection options in human-readable form.

For example: 

(@head, sync)

type Connection 

type Connection struct {
	// contains filtered or unexported fields
}

Connection represents application-level view of a ZODB database.

The view is represented by IPersistent objects associated with the connection. Connection changes are private and are isolated from changes in other Connections. Connection's view corresponds to particular database state and is thus isolated from further database transactions.

Connection is safe to access from multiple goroutines simultaneously.

Connection and objects obtained from it must be used by application only inside transaction where Connection was opened.

Use DB.Open to open a connection.

func (*Connection) At 

func (conn *Connection) At() Tid

At returns database state corresponding to the connection.

func (*Connection) Cache 

func (conn *Connection) Cache() *LiveCache

Cache returns connection's cache of live objects.

func (*Connection) DB 

func (conn *Connection) DB() *DB

DB returns database handle under which the connection was opened.

func (*Connection) Get 

func (conn *Connection) Get(ctx context.Context, oid Oid) (_ IPersistent, err error)

Get returns in-RAM object corresponding to specified ZODB object according to current database view.

If there is already in-RAM object that corresponds to oid, that in-RAM object is returned. Otherwise new in-RAM object is created and filled with object's class loaded from the database.

The scope of the object returned is the Connection.

The object's data is not necessarily loaded after Get returns. Use PActivate to make sure the object is fully loaded.

func (*Connection) Resync 

func (conn *Connection) Resync(ctx context.Context, at Tid) (err error)

Resync resyncs the connection onto different database view and transaction.

Connection's objects pinned in live cache are guaranteed to stay in live cache, even if maybe in ghost state (e.g. if they have to be invalidated due to database changes).

Resync can be used many times.

Resync must be used only under the following conditions:

  • the connection was initially opened with NoPool flag;
  • previous transaction, under which connection was previously opened/resynced, must be already complete;
  • contrary to DB.Open, at cannot be 0.

Note: new at can be both higher and lower than previous connection at.

Note: if new at is already covered by DB.Head Resync will be non-blocking operation. However if at is > current DB.Head Resync, similarly to DB.Open, will block waiting for DB.Head to become ≥ at.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB represents a handle to database at application level and contains pool of connections. DB.Open opens database connection. The connection will be automatically put back into DB pool for future reuse after corresponding transaction is complete. DB thus provides service to maintain live objects cache and reuse live objects from transaction to transaction.

Note that it is possible to have several DB handles to the same database. This might be useful if application accesses distinctly different sets of objects in different transactions and knows beforehand which set it will be next time. Then, to avoid huge cache misses, it makes sense to keep DB handles opened for every possible case of application access.

DB is safe to access from multiple goroutines simultaneously.

func NewDB 

func NewDB(stor IStorage, opt *DBOptions) *DB

NewDB creates new database handle.

Created database handle must be closed when no longer needed.

func (*DB) Close 

func (db *DB) Close() error

Close closes database handle.

After Close DB.Open calls will return error. However it is ok to continue to use connections opened prior to Close.

func (*DB) Open 

func (db *DB) Open(ctx context.Context, opt *ConnOptions) (_ *Connection, err error)

Open opens new connection to the database.

By default the connection is opened to current latest database state; opt.At can be specified to open connection bound to particular view of the database.

Open must be called under transaction. Opened connection must be used only under the same transaction and only until that transaction is complete(*).

(*) unless NoPool option is used.

func (*DB) Storage 

func (db *DB) Storage() IStorage

Storage returns storage this database handle was created for.

type DBOptions 

type DBOptions struct {
	// CacheControl, if !nil, is set as default live cache control for
	// newly created connections.
	CacheControl LiveCacheControl
}

DBOptions describes options to NewDB.

type DataInfo 

type DataInfo struct {
	Oid  Oid
	Tid  Tid    // changed by this transaction
	Data []byte // new object data; nil if object becomes deleted

	// DataTidHint is optional hint from a storage that the same data was
	// already originally committed in earlier transaction, for example in
	// case of undo. It is 0 if there is no such hint.
	//
	// Storages are not obliged to provide this hint, and in particular it
	// is valid for a storage to always return this as zero.
	//
	// In ZODB/py world this originates from
	// https://github.com/zopefoundation/ZODB/commit/2b0c9aa4.
	DataTidHint Tid
}

DataInfo is information about one object change.

type DriverOpener 

type DriverOpener func(ctx context.Context, u *url.URL, opt *DriverOptions) (_ IStorageDriver, at0 Tid, _ error)

DriverOpener is a function to open a storage driver.

at₀ gives database state at open time. The driver will send to Watchq (see DriverOptions) only and all events in (at₀, +∞] range.

type DriverOptions 

type DriverOptions struct {
	ReadOnly bool // whether to open storage as read-only

	// Channel where storage events have to be delivered.
	//
	// Watchq can be nil to ignore such events. However if Watchq != nil, the events
	// have to be consumed or else the storage driver will misbehave - e.g.
	// it can get out of sync with the on-disk database file, or deadlock
	// on any user-called operation.
	//
	// The storage driver closes !nil Watchq when the driver is closed.
	//
	// The storage driver will send only and all events in (at₀, +∞] range,
	// where at₀ is at returned by driver open.
	//
	// The storage driver will stop sending events after call to Close.
	// In particular the following example is valid and safe from deadlock:
	//
	//	watchq := make(chan zodb.Event)
	//	stor, at0, err := zodb.OpenDriver(..., &DriverOptions{Watchq: watchq})
	//	defer stor.Close()
	//
	//	for {
	//		select {
	//		case <-ctx.Done():
	//			return ctx.Err()
	//
	//		case <-watchq:
	//			...
	//		}
	//	}
	Watchq chan<- Event
}

DriverOptions describes options for DriverOpener.

func (*DriverOptions) String 

func (opt *DriverOptions) String() string

String represents DriverOptions in human-readable form.

For example 

(read-only, watch)

type Event 

type Event interface {
	// contains filtered or unexported methods
}

Event represents one database event.

Possible events are:

  • EventError an error happened
  • EventCommit a transaction was committed

type EventCommit 

type EventCommit struct {
	Tid     Tid   // ID of committed transaction
	Changev []Oid // ID of objects changed by committed transaction
}

EventCommit is event describing one observed database commit.

func (*EventCommit) String 

func (e *EventCommit) String() string

type EventError 

type EventError struct {
	Err error
}

EventError is event describing an error observed by watcher.

func (*EventError) String 

func (e *EventError) String() string

String formats events into human-readable form.

type Ghostable 

type Ghostable interface {
	// DropState should discard in-RAM object state.
	//
	// It is called by persistency machinery only on non-ghost objects,
	// i.e. when the objects has its in-RAM state.
	DropState()
}

Ghostable is the interface describing in-RAM object who can release its in-RAM state.

type IDataIterator 

type IDataIterator interface {
	// NextData yields information about next storage data record.
	// returned data stays valid until next call to NextData().
	// end of iteration is indicated with io.EOF
	NextData(ctx context.Context) (*DataInfo, error)
}

IDataIterator is the interface to iterate data records.

type IPersistent 

type IPersistent interface {
	PJar() *Connection // Connection this in-RAM object is part of.
	POid() Oid         // object ID in the database.

	// object serial in the database as of particular Connection (PJar) view.
	// InvalidTid if not yet loaded.
	PSerial() Tid

	// PActivate brings object to live state.
	//
	// It requests to persistency layer that in-RAM object data to be present.
	// If object state was not in RAM - it is loaded from the database.
	//
	// On successful return the object data is either the same as in the
	// database or, if this data was previously modified by user of
	// object's jar, that modified data.
	//
	// Object data must be accessed only after corresponding PActivate
	// call, which marks that object's data as being in use.
	PActivate(ctx context.Context) error

	// PDeactivate indicates that corresponding PActivate caller finished access to object's data.
	//
	// As PActivate makes sure object's data is present in-RAM, PDeactivate
	// tells persistency layer that this data is no longer used by
	// corresponding PActivate caller.
	//
	// Note that it is valid to have several concurrent uses of object
	// data, each protected with corresponding PActivate/PDeactivate pair:
	// as long as there is still any PActivate not yet compensated with
	// corresponding PDeactivate, object data will assuredly stay alive in RAM.
	//
	// Besides exotic cases, the caller thus must not use object's data
	// after PDeactivate call.
	PDeactivate()

	// PInvalidate requests in-RAM object data to be discarded.
	//
	// Irregardless of whether in-RAM object data is the same as in the
	// database, or it was modified, that in-RAM data must be forgotten.
	//
	// PInvalidate must not be called while there is any in-progress
	// object's data use (PActivate till PDeactivate).
	//
	// In practice this means that:
	//
	//	- application must make sure to finish all objects accesses
	//	  before transaction boundary: at transaction boundary - either
	//	  at abort or commit, the persistency layer will sync to
	//	  database and process invalidations.
	//
	//	- if PInvalidate is explicitly called by application, the
	//	  application must care to make sure it does not access the
	//	  object data simultaneously.
	PInvalidate()
	// contains filtered or unexported methods
}

IPersistent is the interface that every in-RAM object representing any database object implements.

It is based on IPersistent from ZODB/py: 

https://github.com/zopefoundation/ZODB/blob/3.10.7-4-gb8d7a8567/src/persistent/interfaces.py#L22

but is not exactly equal to it.

It is safe to access IPersistent from multiple goroutines simultaneously.

Use Persistent as the base for application-level types that need to provide persistency.

func NewPersistent 

func NewPersistent(typ reflect.Type, jar *Connection) IPersistent

NewPersistent creates new instance of persistent type.

typ must embed Persistent and must be registered with RegisterClass.

Created instance will be associated with jar, but will have no oid assigned until transaction commit.

type IStorage 

type IStorage interface {
	// same as in IStorageDriver
	URL() string
	Close() error
	Loader
	Iterator

	// Sync syncs to storage and updates current view of it.
	//
	// After Sync, Head is guaranteed to give ID of last transaction
	// committed to storage data as observed from some time _afterwards_
	// Sync call was made. In particular for client-server case, Sync
	// cannot retain cached view of storage and has to perform round-trip
	// to the server.
	Sync(context.Context) error

	// Head returns ID of last committed transaction.
	//
	// Returned head is ID of last committed transaction as observed from
	// some time _before_ Head call returns. In particular for
	// client-sever case, Head can return cached view of storage that was
	// learned some time ago.
	//
	// Head is ↑=.
	//
	// Head is 0 if no transactions have been committed yet.
	//
	// Use Sync to synchronize with the storage.
	Head() Tid

	// additional to IStorageDriver
	Prefetcher
	Watcher
}

IStorage is the interface provided by opened ZODB storage.

func Open 

func Open(ctx context.Context, zurl string, opt *OpenOptions) (_ IStorage, err error)

Open opens ZODB storage by URL.

Only URL schemes registered to zodb package are handled. Users should import in storage packages they use or zodb/wks package to get support for well-known storages.

Storage authors should register their storages with RegisterDriver.

type IStorageDriver 

type IStorageDriver interface {
	// URL returns URL of how the storage was opened
	URL() string

	// Close closes storage
	Close() error

	// Sync syncs to storage and returns ID of last committed transaction.
	//
	// Returned head is ID of last transaction committed to storage data as
	// observed from some time _afterwards_ Sync call was made. In particular
	// for client-server case, Sync cannot return cached view of storage
	// and has to perform round-trip to the server.
	//
	// Head is ↑=.
	//
	// Head is 0 if no transactions have been committed yet.
	Sync(ctx context.Context) (head Tid, _ error)

	Loader
	Iterator
}

IStorageDriver is the raw interface provided by ZODB storage drivers.

type ITxnIterator 

type ITxnIterator interface {
	// NextTxn yields information about next database transaction:
	// 1. transaction metadata, and
	// 2. iterator over transaction's data records.
	// transaction metadata stays valid until next call to NextTxn().
	// end of iteration is indicated with io.EOF
	NextTxn(ctx context.Context) (*TxnInfo, IDataIterator, error)
}

ITxnIterator is the interface to iterate transactions.

type Iterator 

type Iterator interface {
	// Iterate creates iterator to iterate storage in [tidMin, tidMax] range.
	//
	// Iterate does not return any error. If there was error when setting
	// iteration up - it will be returned on first NextTxn call.
	//
	// TODO allow iteration both ways (forward & backward)
	Iterate(ctx context.Context, tidMin, tidMax Tid) ITxnIterator
}

Iterator provides functionality to iterate through storage transactions sequentially.

type List 

type List struct {
	Persistent

	// XXX it is not possible to embed slice - see Map for similar issue and more details.
	Data []interface{}
}

List is equivalent of persistent.list.PersistentList in ZODB/py.

type LiveCache 

type LiveCache struct {
	sync.Mutex
	// contains filtered or unexported fields
}

LiveCache keeps registry of live in-RAM objects for a Connection.

It semantically consists of 

{} oid -> obj

but does not hold strong reference to cached objects.

LiveCache is not safe to use from multiple goroutines simultaneously.

Use .Lock() / .Unlock() to serialize access.

func (*LiveCache) Get 

func (cache *LiveCache) Get(oid Oid) IPersistent

Get lookups object corresponding to oid in the cache.

If object is found, it is guaranteed to stay in live cache while the caller keeps reference to it. LiveCacheControl can be used to extend that guarantee.

func (*LiveCache) SetControl 

func (cache *LiveCache) SetControl(c LiveCacheControl)

SetControl installs c to handle cache decisions.

Any previously installed cache control is uninstalled. Passing nil sets the cache to have no control installed at all.

It is not safe to call SetControl simultaneously to other cache operations.

type LiveCacheControl 

type LiveCacheControl interface {
	// PCacheClassify is called to classify an object and returns live
	// cache policy that should be used for this object.
	PCacheClassify(obj IPersistent) PCachePolicy
}

LiveCacheControl is the interface that allows applications to influence Connection's decisions with respect to Connection's LiveCache.

See Connection.Cache and LiveCache.SetControl for how to install LiveCacheControl on a connection's live cache.

type Loader 

type Loader interface {
	// Load loads object data addressed by xid from database.
	//
	// Returned are:
	//
	//	- if there is data to load: buf is non-empty, serial indicates
	//	  transaction which matched xid criteria and err=nil.
	//
	//	  caller must not modify buf memory.
	//
	// otherwise buf=nil, serial=0 and err is *OpError with err.Err
	// describing the error cause:
	//
	//	- *NoObjectError if there is no such object in database at all,
	//	- *NoDataError   if object exists in database but there is no
	//	                 its data matching xid,
	//	- some other error indicating e.g. IO problem.
	//
	//
	// NOTE 1: ZODB/py provides 2 entrypoints in IStorage for loading:
	// loadSerial and loadBefore but in ZODB/go we have only Load which is
	// a bit different from both:
	//
	//	- Load loads object data for object at database state specified by xid.At
	//	- loadBefore loads object data for object at database state previous to xid.At
	//	  it is thus equivalent to Load(..., xid.At-1)
	//	- loadSerial loads object data from revision exactly modified
	//	  by transaction with tid = xid.At.
	//	  it is thus equivalent to Load(..., xid.At) with followup
	//	  check that returned serial is exactly xid.At(*)
	//
	// (*) loadSerial is used only in a few places in ZODB/py - mostly in
	//     conflict resolution code where plain Load semantic - without
	//     checking object was particularly modified at that revision - would
	//     suffice.
	//
	// NOTE 2: in ZODB/py loadBefore, in addition to serial, also returns
	// serial_next, which constraints storage implementations unnecessarily
	// and is used only in client cache.
	//
	// In ZODB/go Cache shows that it is possible to build efficient client
	// cache without serial_next returned from Load. For this reason in ZODB/go
	// Load specification comes without specifying serial_next return.
	Load(ctx context.Context, xid Xid) (buf *mem.Buf, serial Tid, err error)
}

Loader provides functionality to load objects.

type Map 

type Map struct {
	Persistent

	// XXX it is not possible to embed map. And even if we embed a map via
	// another type = map, then it is not possible to use indexing and
	// range over Map. -> just provide access to the map as .Data .
	Data map[interface{}]interface{}
}

Map is equivalent of persistent.mapping.PersistentMapping in ZODB/py.

type NoDataError 

type NoDataError struct {
	Oid Oid

	// DeletedAt explains object state wrt used search criteria:
	// - 0:  object was not created at time of searched xid.At
	// - !0: object was deleted by transaction with tid=DeletedAt
	DeletedAt Tid
}

NoDataError is the error which tells that object exists in the database, but there is no its non-empty revision satisfying search criteria.

func (*NoDataError) Error 

func (e *NoDataError) Error() string

type NoObjectError 

type NoObjectError struct {
	Oid Oid
}

NoObjectError is the error which tells that there is no such object in the database at all.

func (*NoObjectError) Error 

func (e *NoObjectError) Error() string

type ObjectState 

type ObjectState int

ObjectState describes state of in-RAM object. 

const (
	GHOST    ObjectState = -1 // object data is not yet loaded from the database
	UPTODATE ObjectState = 0  // object is live and in-RAM data is the same as in database
	CHANGED  ObjectState = 1  // object is live and in-RAM data was changed

)

type Oid 

type Oid uint64

Oid is object identifier.

In ZODB objects are uniquely identified by 64-bit integer. An object can have several revisions - each committed in different transaction. The combination of object identifier and particular transaction (serial) uniquely addresses corresponding data record.

0 is valid Oid and represents root database object. InvalidOid represents an invalid Oid.

See also: Xid. 

const InvalidOid Oid = 1<<64 - 1 // 0xffffffffffffffff

func ParseOid 

func ParseOid(s string) (Oid, error)

ParseOid parses oid from string.

See also: Oid.String .

func (Oid) String 

func (oid Oid) String() string

String converts oid to string.

Default oid string representation is 16-character hex string, e.g.: 

0000000000000001

See also: ParseOid.

func (Oid) XFmtString 

func (oid Oid) XFmtString(b []byte) []byte

type OpError 

type OpError struct {
	URL  string      // URL of the storage
	Op   string      // operation that failed
	Args interface{} // operation arguments, if any
	Err  error       // actual error that occurred during the operation
}

OpError is the error returned by IStorageDriver operations.

func (*OpError) Cause 

func (e *OpError) Cause() error

func (*OpError) Error 

func (e *OpError) Error() string

func (*OpError) Unwrap 

func (e *OpError) Unwrap() error

type OpenOptions 

type OpenOptions struct {
	ReadOnly bool // whether to open storage as read-only
	NoCache  bool // don't use cache for read/write operations; prefetch will be noop
}

OpenOptions describes options for Open.

func (*OpenOptions) String 

func (opt *OpenOptions) String() string

String represents OpenOptions in human-readable form.

For example: 

(read-only, no-cache)

type PCachePolicy 

type PCachePolicy int

PCachePolicy describes live caching policy for a persistent object.

It is | combination of PCache* flags with 0 meaning "use default policy".

See LiveCacheControl for how to apply a caching policy. 

const (
	// keep object pinned into cache, even if in ghost state.
	//
	// This allows to rely on object being never evicted from live cache.
	//
	// Note: object's state can still be discarded and the object can go
	// into ghost state. Use PCacheKeepState to prevent such automatic
	// state eviction until state discard is semantically required.
	PCachePinObject PCachePolicy = 1 << iota

	// don't keep object in cache.
	//
	// The object will be discarded from the cache completely as soon as it
	// is semantically valid to do so.
	PCacheDropObject

	// keep object state in cache.
	//
	// This prevents object state to go away when !dirty object is no
	// longer used. However the object itself can go away unless it is
	// pinned into cache via PCachePinObject.
	//
	// Note: on invalidation, state of invalidated objects is discarded
	// unconditionally.
	PCacheKeepState

	// don't keep object state.
	//
	// Data access is likely non-temporal and object's state will be used
	// once and then won't be used for a long time. Don't pollute cache
	// with state of this object.
	PCacheDropState
)

type Persistent 

type Persistent struct {
	// contains filtered or unexported fields
}

Persistent is common base IPersistent implementation for in-RAM representation of database objects.

To use - a class needs to embed Persistent and register itself additionally providing Ghostable and (Py)Stateful methods. For example: 

type MyObject struct {
	Persistent
	...
}

type myObjectState MyObject

func (o *myObjectState) DropState() { ... }
func (o *myObjectState) GetState() *mem.Buf { ... }
func (o *myObjectState) SetState(state *mem.Buf) error { ... }

func init() {
	t := reflect.TypeOf
	zodb.RegisterClass("mymodule.MyObject", t(MyObject{}), t(myObjectState))
}

A new instance of the class that embeds Persistent must be created via NewPersistent, for example: 

obj := zodb.NewPersistent(reflect.TypeOf(MyObject{}), jar).(*MyObject)

func (*Persistent) PActivate 

func (obj *Persistent) PActivate(ctx context.Context) (err error)

PActivate implements IPersistent.

func (*Persistent) PDeactivate 

func (obj *Persistent) PDeactivate()

PDeactivate implements IPersistent.

func (*Persistent) PInvalidate 

func (obj *Persistent) PInvalidate()

PInvalidate() implements IPersistent.

func (*Persistent) PJar 

func (obj *Persistent) PJar() *Connection

func (*Persistent) POid 

func (obj *Persistent) POid() Oid

func (*Persistent) PSerial 

func (obj *Persistent) PSerial() Tid

type Prefetcher 

type Prefetcher interface {
	// Prefetch prefetches object addressed by xid.
	//
	// If data is not yet in cache loading for it is started in the background.
	// Prefetch is not blocking operation and does not wait for loading, if any was
	// started, to complete.
	//
	// Prefetch does not return any error.
	// Prefetch is noop if storage was opened with NoCache option.
	Prefetch(ctx context.Context, xid Xid)
}

Prefetcher provides functionality to prefetch objects.

type PyData 

type PyData []byte

PyData represents raw data stored into ZODB by Python applications.

The format is based on python pickles. Basically every serialized object has two parts: pickle with class description and pickle with object state. See 

https://github.com/zopefoundation/ZODB/blob/a89485c1/src/ZODB/serialize.py

for format description.

func (PyData) ClassName 

func (d PyData) ClassName() string

ClassName returns fully-qualified python class name used for object type.

The format is "module.class". If pickle decoding fails - "?.?" is returned.

type PyStateful 

type PyStateful interface {
	// PyGetState should return state of the in-RAM object as Python data.
	//
	// It is analog of __getstate__() in Python.
	//
	// It is called by persistency machinery only on non-ghost objects,
	// i.e. when the object has its in-RAM state.
	PyGetState() interface{}

	// PySetState should set state of the in-RAM object from Python data.
	//
	// It is analog of __setstate__() in Python.
	//
	// It is called by persistency machinery only on ghost objects, i.e.
	// when the objects does not yet have its in-RAM state.
	//
	// The error returned does not need to have object/setstate prefix -
	// persistency machinery is adding such prefix automatically.
	PySetState(pystate interface{}) error
}

PyStateful is the interface describing in-RAM object whose data state can be exchanged as Python data.

type Stateful 

type Stateful interface {
	// GetState should return state of the in-RAM object as raw data.
	//
	// It is called by persistency machinery only on non-ghost objects,
	// i.e. when the object has its in-RAM state.
	//
	// GetState should return a new buffer reference.
	GetState() *mem.Buf

	// SetState should set state of the in-RAM object from raw data.
	//
	// It is called by persistency machinery only on ghost objects, i.e.
	// when the objects does not yet have its in-RAM state.
	//
	// state ownership is not passed to SetState, so if state needs to be
	// retained after SetState returns it needs to be incref'ed.
	//
	// The error returned does not need to have object/setstate prefix -
	// persistent machinery is adding such prefix automatically.
	SetState(state *mem.Buf) error
}

Stateful is the interface describing in-RAM object whose data state can be exchanged as raw bytes.

type Tid 

type Tid uint64

Tid is transaction identifier.

In ZODB transaction identifiers are unique 64-bit integers corresponding to time when transaction in question was committed.

This way tid can also be used to specify whole database state constructed by all cumulated transaction changes from database beginning up to, and including, transaction specified by tid.

0 is invalid Tid, but canonical invalid Tid value is InvalidTid. 

const (
	TidMax     Tid = 1<<63 - 1 // 0x7fffffffffffffff
	InvalidTid Tid = 1<<64 - 1 // 0xffffffffffffffff
)

func ParseTid 

func ParseTid(s string) (Tid, error)

ParseTid parses tid from string.

See also: Tid.String .

func ParseTidRange 

func ParseTidRange(s string) (tidMin, tidMax Tid, err error)

ParseTidRange parses string of form "<tidmin>..<tidmax>" into tidMin, tidMax pair.

Both <tidmin> and <tidmax> can be empty, in which case defaults 0 and TidMax are used.

See `zodb help tidrange` for accepted tidrange syntax.

XXX also check tidMin < tidMax here? or allow reverse ranges ?

func TidFromTime 

func TidFromTime(t time.Time) Tid

TidFromTime converts time to tid.

func (Tid) String 

func (tid Tid) String() string

String converts tid to string.

Default tid string representation is 16-character hex string, e.g.: 

0285cbac258bf266

See also: ParseTid.

func (Tid) Time 

func (tid Tid) Time() TimeStamp

Time converts tid to time.

func (Tid) Valid 

func (tid Tid) Valid() bool

Valid returns whether tid is in valid transaction identifiers range.

func (Tid) XFmtString 

func (tid Tid) XFmtString(b []byte) []byte

type TimeStamp 

type TimeStamp struct {
	time.Time
}

TimeStamp is the same as time.Time only .String() is adjusted to be the same as in ZODB/py.

XXX get rid eventually of this and just use time.Time.

func (TimeStamp) String 

func (t TimeStamp) String() string

func (TimeStamp) XFmtString 

func (t TimeStamp) XFmtString(b []byte) []byte

type TxnInfo 

type TxnInfo struct {
	Tid         Tid
	Status      TxnStatus
	User        []byte
	Description []byte

	// additional information about transaction. ZODB/py usually puts py
	// dict here but it can be arbitrary raw bytes.
	Extension []byte
}

TxnInfo is metadata information about one transaction.

type TxnStatus 

type TxnStatus byte

TxnStatus represents status of a transaction. 

const (
	TxnComplete   TxnStatus = ' ' // completed transaction that hasn't been packed
	TxnPacked     TxnStatus = 'p' // completed transaction that has been packed
	TxnInprogress TxnStatus = 'c' // checkpoint -- a transaction in progress; it's been thru vote() but not finish()
)

func (TxnStatus) Valid 

func (ts TxnStatus) Valid() bool

Valid returns true if transaction status value is well-known and valid.

type Watcher 

type Watcher interface {
	// AddWatch registers watchq to be notified of database changes.
	//
	// Whenever a new transaction is committed into the database,
	// corresponding event will be sent to watchq.
	//
	// It is guaranteed that events are coming with ↑ .Tid .
	//
	// It will be only and all events in (at₀, +∞] range, that will be
	// sent, where at₀ is database head that was current when AddWatch call
	// was made.
	//
	// Once registered, watchq must be read until it is closed or until
	// DelWatch call. Not doing so will stuck whole storage.
	//
	// Registered watchq are closed when the database storage is closed.
	//
	// It is safe to add watch to a closed database storage.
	//
	// AddWatch must be used only once for a particular watchq channel.
	AddWatch(watchq chan<- Event) (at0 Tid)

	// DelWatch unregisters watchq from being notified of database changes.
	//
	// After DelWatch call completes, no new events will be sent to watchq.
	// It is safe to call DelWatch without simultaneously reading watchq.
	// In particular the following example is valid:
	//
	//	at0 := stor.AddWatch(watchq)
	//	defer stor.DelWatch(watchq)
	//
	//	for {
	//		select {
	//		case <-ctx.Done():
	//			return ctx.Err()
	//
	//		case <-watchq:
	//			...
	//		}
	//	}
	//
	// DelWatch is noop if watchq was not registered.
	DelWatch(watchq chan<- Event)
}

Watcher allows to be notified of changes to database.

Watcher is safe to use from multiple goroutines simultaneously.

type Xid 

type Xid struct {
	At  Tid
	Oid Oid
}

Xid is "extended" oid - that fully specifies object and query for its revision.

At specifies whole database state at which object identified with Oid should be looked up. The object revision is taken from latest transaction modifying the object with tid ≤ At.

Note that Xids are not unique - the same object revision can be addressed with several xids.

See also: Tid, Oid.

func ParseXid 

func ParseXid(s string) (Xid, error)

ParseXid parses xid from string.

See also: Xid.String .

func (Xid) String 

func (xid Xid) String() string

String converts xid to string.

Default xid string representation is:

  • string of at
  • ":"
  • string of oid

e.g. 

0285cbac258bf266:0000000000000001	- oid 1 at first newest transaction changing it with tid <= 0285cbac258bf266

See also: ParseXid.

type ΔRevEntry 

type ΔRevEntry struct {
	Rev     Tid
	Changev []Oid
}

ΔRevEntry represents information of what have been changed in one revision.

type ΔTail 

type ΔTail struct {
	// contains filtered or unexported fields
}

ΔTail represents tail of revisional changes.

It semantically consists of 

[](rev↑, []id)		; rev ∈ (tail, head]

and index 

{} id -> max(rev: rev changed id)

where 

rev          - is ZODB revision,
id           - is an identifier of what has been changed(*), and
(tail, head] - is covered revision range

It provides operations to

  • append information to the tail about next revision,
  • forget information in the tail past specified revision,
  • query the tail for slice with rev ∈ (lo, hi],
  • query the tail about what is last revision that changed an id,
  • query the tail for len and (tail, head].

ΔTail is safe to access for multiple-readers / single writer.

(*) examples of id: 

oid  - ZODB object identifier, when ΔTail represents changes to ZODB objects,
#blk - file block number, when ΔTail represents changes to a file.

func NewΔTail 

func NewΔTail(at0 Tid) *ΔTail

NewΔTail creates new ΔTail object.

Initial coverage of created ΔTail is (at₀, at₀].

func (*ΔTail) Append 

func (δtail *ΔTail) Append(rev Tid, changev []Oid)

Append appends to δtail information about what have been changed in next revision.

rev must be ↑.

func (*ΔTail) Data 

func (δtail *ΔTail) Data() []ΔRevEntry

Data returns slice with δtail data.

The caller must not modify returned slice.

func (*ΔTail) ForgetPast 

func (δtail *ΔTail) ForgetPast(revCut Tid)

ForgetPast discards all δtail entries with rev ≤ revCut.

func (*ΔTail) Head 

func (δtail *ΔTail) Head() Tid

Head returns newest database state for which δtail has history coverage.

Head is ↑ on Append, in particular it does not ↓ on Forget even if δtail becomes empty.

func (*ΔTail) LastRevOf 

func (δtail *ΔTail) LastRevOf(id Oid, at Tid) (_ Tid, exact bool)

LastRevOf tries to return what was the last revision that changed id as of at database state.

it must be called with the following condition: 

tail ≤ at ≤ head

Depending on current information in δtail it returns either exact result, or an upper-bound estimate for the last id revision:

1) if δtail has an entry corresponding to id change, it gives exactly the last revision that changed id: 

# ∃ rev ∈ δtail: rev changed id && rev ≤ at
LastRevOf(id, at) = max(rev: rev changed id && rev ≤ at), true

2) if δtail does not contain appropriate record with id - it returns δtail's lower bound as the estimate for the upper bound of the last id revision: 

# ∄ rev ∈ δtail: rev changed id && rev ≤ at
LastRevOf(id, at) = δtail.tail, false

On return exact indicates whether returned revision is exactly the last revision of id, or only an upper-bound estimate of it.

func (*ΔTail) Len 

func (δtail *ΔTail) Len() int

Len returns number of revisions.

func (*ΔTail) SliceByRev 

func (δtail *ΔTail) SliceByRev(low, high Tid) []ΔRevEntry

SliceByRev returns δtail slice of elements with .rev ∈ (low, high].

it must be called with the following condition: 

tail ≤ low ≤ high ≤ head

the caller must not modify returned slice.

Note: contrary to regular go slicing, low is exclusive while high is inclusive.

func (*ΔTail) Tail 

func (δtail *ΔTail) Tail() Tid

Tail returns lower-bound of database state for which δtail has history coverage.

Tail is ↑= on Forget, even if δtail becomes empty.

Source Files

View all Source files

Directories

Path Synopsis
Package btree provides B⁺ Trees for ZODB.
 Package btree provides B⁺ Trees for ZODB.
cmd
zodb
Zodb is a driver program for invoking zodbtools subcommands.
 Zodb is a driver program for invoking zodbtools subcommands.
internal
pickletools
Package pickletools provides utilities related to python pickles.
 Package pickletools provides utilities related to python pickles.
weak
Package weak is stub for weak references.
 Package weak is stub for weak references.
Package storage provides common infrastructure related to ZODB storages.
 Package storage provides common infrastructure related to ZODB storages.
demo
Package demo provides overlayed storage, similar to DemoStorage in ZODB/py.
 Package demo provides overlayed storage, similar to DemoStorage in ZODB/py.
fs1
Package fs1 provides so-called FileStorage version 1 ZODB storage.
 Package fs1 provides so-called FileStorage version 1 ZODB storage.
fs1/cmd/fs1
fs1 is a driver program for running and invoking fs1 subcommands.
 fs1 is a driver program for running and invoking fs1 subcommands.
fs1/fs1tools
Package fs1tools provides tools for managing and maintaining ZODB FileStorage v1 databases.
 Package fs1tools provides tools for managing and maintaining ZODB FileStorage v1 databases.
fs1/fsb
Package fsb specializes cznic/b.Tree for FileStorage index needs.
 Package fsb specializes cznic/b.Tree for FileStorage index needs.
zeo
Package zeo provides simple ZEO client.
 Package zeo provides simple ZEO client.
Package wks links-in well-known ZODB storages.
 Package wks links-in well-known ZODB storages.
Package zodbtools provides tools for managing ZODB databases.
 Package zodbtools provides tools for managing ZODB databases.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.