storage

Documentation

Index

• Constants
• type BoltDB
  • func NewBoltDB(filePath string) (boltDB *BoltDB, err error)
  • func (boltDB *BoltDB) Close()
  • func (boltDB *BoltDB) DumpJson(writer io.Writer) error
  • func (boltDB *BoltDB) FileCount() int
  • func (boltDB *BoltDB) FileIdentifierBatch(offset, limit int) []string
  • func (boltDB *BoltDB) FileIdentifiers() []string
  • func (boltDB *BoltDB) FilePath() string
  • func (boltDB *BoltDB) ForEach(fn func(k, v []byte) error) error
  • func (boltDB *BoltDB) GetGenericFile(key string) (*models.GenericFile, error)
  • func (boltDB *BoltDB) GetIntellectualObject(key string) (*models.IntellectualObject, error)
  • func (boltDB *BoltDB) ObjectIdentifier() string
  • func (boltDB *BoltDB) Save(key string, value interface{}) error

Constants

const FILE_BUCKET = "files"

const OBJ_BUCKET = "objects"

Types

type BoltDB

type BoltDB struct {
	// contains filtered or unexported fields
}

BoltDB represents a bolt database, which is a single-file key-value store. Our validator uses this to track information about the files inside a bag that we're validating. At a minimum, the validator typically needs to track these pieces of information for each file: the absolute path, the manifests' md5 digest, the manifest's sha256 digest, the validator's calculated md5 digest, and the validator's calculated sha256 digest. That can be a few hundred bytes of data per file. APTrust ingest services will track more than that: about 8-9 kilobytes of data per file. Multiply that by 100k or even 1 million files in a bag, and that's too much to keep in memory.

func NewBoltDB

func NewBoltDB(filePath string) (boltDB *BoltDB, err error)

NewBoltDB opens a bolt database, creating the DB file if it doesn't already exist. The DB file is a key-value store that resides in a single file on disk.

func (*BoltDB) Close

func (boltDB *BoltDB) Close()

Close closes the bolt database.

func (*BoltDB) DumpJson

func (boltDB *BoltDB) DumpJson(writer io.Writer) error

DumpJson writes all the records from the db into a single JSON string. The output is the JSON representation of an IntellectualObject with all of its GenericFiles (and Checksums and PremisEvents, if there are any).

func (*BoltDB) FileCount

func (boltDB *BoltDB) FileCount() int

FileCount returns the number of GenericFiles stored in the database.

func (*BoltDB) FileIdentifierBatch

func (boltDB *BoltDB) FileIdentifierBatch(offset, limit int) []string

FileIdentifierBatch returns a list of GenericFile identifiers from offset (zero-based) up to limit, or end of list.

func (*BoltDB) FileIdentifiers

func (boltDB *BoltDB) FileIdentifiers() []string

FileIdentifiers returns a list of all keys in the database.

func (*BoltDB) FilePath

func (boltDB *BoltDB) FilePath() string

FilePath returns the path to the bolt DB file.

func (*BoltDB) ForEach

func (boltDB *BoltDB) ForEach(fn func(k, v []byte) error) error

ForEach calls the specified function for each key in the database's file bucket.

func (*BoltDB) GetGenericFile

func (boltDB *BoltDB) GetGenericFile(key string) (*models.GenericFile, error)

GetGenericFile returns the GenericFile with the specified identifier. The GenericFile will include checksums and events, if they are available. Param key is the GenericFile.Identifier. If key is not found this returns nil and no error.

func (*BoltDB) GetIntellectualObject

func (boltDB *BoltDB) GetIntellectualObject(key string) (*models.IntellectualObject, error)

GetIntellectualObject returns the IntellectualObject that matches the specified key. This object will NOT include GenericFiles. There may be tens of thousands of those, so you have to fetch them individually. Param key is the IntellectualObject.Identifier. If key is not found, this returns nil and no error.

func (*BoltDB) ObjectIdentifier

func (boltDB *BoltDB) ObjectIdentifier() string

ObjectIdentifier returns the IntellectualObject.Identifier for the object stored in this DB file.

func (*BoltDB) Save

func (boltDB *BoltDB) Save(key string, value interface{}) error

Save saves a value to the bolt database.