i3vdir

README

I3vDir

The I3vDir module is responsible for creating and maintaining the directory metadata information stored in the .i3vdir files on disk. This includes all disk interaction and metadata definition. These i3vdirs represent directories on the I3v network.

Structure of the I3vDir

The I3vDir is a dir on the I3v network and the i3vdir metadata is a JSON formatted metadata file that contains aggregate and non-aggregate fields. The aggregate fields are the totals of the i3vdir and any sub i3vdirs, or are calculated based on all the values in the subtree. The non-aggregate fields are information specific to the i3vdir that is not an aggregate of the entire sub directory tree

Subsystems

The following subsystems help the I3vDir module execute its responsibilities:

• Persistence Subsystem
• File Format Subsystem
• I3vDirSet Subsystem
• DirReader Subsystem

Persistence Subsystem

Key Files

• persist.go
• persistwal.go

The Persistence subsystem is responsible for the disk interaction with the .i3vdir files and ensuring safe and performant ACID operations by using the writeaheadlog package. There are two WAL updates that are used, deletion and metadata updates.

The WAL deletion update removes all the contents of the directory including the directory itself.

The WAL metadata update re-writes the entire metadata, which is stored as JSON. This is used whenever the metadata changes and needs to be saved as well as when a new i3vdir is created.

Exports

• ApplyUpdates
• CreateAndApplyTransaction
• IsI3vDirUpdate
• New
• LoadI3vDir
• UpdateMetadata

Inbound Complexities

• callDelete deletes a I3vDir from disk
  • I3vDirSet.Delete uses callDelete
• LoadI3vDir loads a I3vDir from disk
  • I3vDirSet.open uses LoadI3vDir

File Format Subsystem

Key Files

• i3vdir.go

The file format subsystem contains the type definitions for the I3vDir format and methods that return information about the I3vDir.

Exports

• Deleted
• Metatdata
• I3vPath

I3vDirSet Subsystem

Key Files

• i3vdirset.go

A I3vDir object is threadsafe by itself, and to ensure that when a I3vDir is accessed by multiple threads that it is still threadsafe, I3vDirs should always be accessed through the I3vDirSet. The I3vDirSet was created as a pool of I3vDirs which is used by other packages to get access to I3vDirEntries which are wrappers for I3vDirs containing some extra information about how many threads are using it at a certain time. If a I3vDir was already loaded the I3vDirSet will hand out the existing object, otherwise it will try to load it from disk.

Exports

• HealthPercentage
• NewI3vDirSet
• Close
• Delete
• DirInfo
• DirList
• Exists
• InitRootDir
• NewI3vDir
• Open
• Rename

Outbound Complexities

• Delete will use callDelete to delete the I3vDir once it has been acquired in the set
• open calls LoadI3vDir to load the I3vDir from disk

DirReader Subsystem

Key Files

• dirreader.go

The DirReader Subsystem creates the DirReader which is used as a helper to read raw .i3vdir from disk

Exports

• Close
• Read
• Stat
• DirReader

Documentation

Index

• Constants
• func ApplyUpdates(updates ...writeaheadlog.Update) error
• func CreateAndApplyTransaction(wal *writeaheadlog.WAL, updates ...writeaheadlog.Update) error
• func IsI3vDirUpdate(update writeaheadlog.Update) bool
• type DirReader
  • func (sdr *DirReader) Close() error
  • func (sdr *DirReader) Read(b []byte) (int, error)
  • func (sdr *DirReader) Stat() (os.FileInfo, error)
• type I3vDir
  • func LoadI3vDir(path string, deps modules.Dependencies, wal *writeaheadlog.WAL) (sd *I3vDir, err error)
  • func New(path, rootPath string, mode os.FileMode, wal *writeaheadlog.WAL) (*I3vDir, error)
  • func (sd *I3vDir) Delete() error
  • func (sd *I3vDir) Deleted() bool
  • func (sd *I3vDir) DirReader() (*DirReader, error)
  • func (sd *I3vDir) MDPath() string
  • func (sd *I3vDir) Metadata() Metadata
  • func (sd *I3vDir) Path() string
  • func (sd *I3vDir) Rename(targetPath string) error
  • func (sd *I3vDir) SetPath(targetPath string)
  • func (sd *I3vDir) UpdateMetadata(metadata Metadata) error
• type Metadata

Constants

const (
	// I3vDirExtension is the name of the metadata file for the i3v directory
	I3vDirExtension = ".i3vdir"

	// DefaultDirHealth is the default health for the directory and the fall
	// back value when there is an error. This is to protect against falsely
	// trying to repair directories that had a read error
	DefaultDirHealth = float64(0)

	// DefaultDirRedundancy is the default redundancy for the directory and the
	// fall back value when there is an error. This is to protect against
	// falsely trying to repair directories that had a read error
	DefaultDirRedundancy = float64(-1)
)

Functions

func ApplyUpdates

func ApplyUpdates(updates ...writeaheadlog.Update) error

ApplyUpdates applies a number of writeaheadlog updates to the corresponding I3vDir. This method can apply updates from different I3vDirs and should only be run before the I3vDirs are loaded from disk right after the startup of i3vd. Otherwise we might run into concurrency issues.

func CreateAndApplyTransaction

func CreateAndApplyTransaction(wal *writeaheadlog.WAL, updates ...writeaheadlog.Update) error

CreateAndApplyTransaction is a helper method that creates a writeaheadlog transaction and applies it.

func IsI3vDirUpdate

func IsI3vDirUpdate(update writeaheadlog.Update) bool

IsI3vDirUpdate is a helper method that makes sure that a wal update belongs to the I3vDir package.

Types

type DirReader

type DirReader struct {
	// contains filtered or unexported fields
}

DirReader is a helper type that allows reading a raw .i3vdir from disk while keeping the file in memory locked.

func (*DirReader) Close

func (sdr *DirReader) Close() error

Close closes the underlying file.

func (*DirReader) Read

func (sdr *DirReader) Read(b []byte) (int, error)

Read calls Read on the underlying file.

func (*DirReader) Stat

func (sdr *DirReader) Stat() (os.FileInfo, error)

Stat returns the FileInfo of the underlying file.

type I3vDir

type I3vDir struct {
	// contains filtered or unexported fields
}

I3vDir contains the metadata information about a renter directory

func LoadI3vDir

func LoadI3vDir(path string, deps modules.Dependencies, wal *writeaheadlog.WAL) (sd *I3vDir, err error)

LoadI3vDir loads the directory metadata from disk

func New

func New(path, rootPath string, mode os.FileMode, wal *writeaheadlog.WAL) (*I3vDir, error)

New creates a new directory in the renter directory and makes sure there is a metadata file in the directory and creates one as needed. This method will also make sure that all the parent directories are created and have metadata files as well and will return the I3vDir containing the information for the directory that matches the i3vPath provided

func (*I3vDir) Delete

func (sd *I3vDir) Delete() error

Delete removes the directory from disk and marks it as deleted. Once the directory is deleted, attempting to access the directory will return an error.

func (*I3vDir) Deleted

func (sd *I3vDir) Deleted() bool

Deleted returns the deleted field of the i3vDir

func (*I3vDir) DirReader

func (sd *I3vDir) DirReader() (*DirReader, error)

DirReader creates a io.ReadCloser that can be used to read the raw I3vDir from disk.

func (*I3vDir) MDPath

func (sd *I3vDir) MDPath() string

MDPath returns the path of the I3vDir's metadata on disk.

func (*I3vDir) Metadata

func (sd *I3vDir) Metadata() Metadata

Metadata returns the metadata of the I3vDir

func (*I3vDir) Path

func (sd *I3vDir) Path() string

Path returns the path of the I3vDir on disk.

func (*I3vDir) Rename

func (sd *I3vDir) Rename(targetPath string) error

Rename renames the I3vDir to targetPath.

func (*I3vDir) SetPath

func (sd *I3vDir) SetPath(targetPath string)

SetPath sets the path field of the dir.

func (*I3vDir) UpdateMetadata

func (sd *I3vDir) UpdateMetadata(metadata Metadata) error

UpdateMetadata updates the I3vDir metadata on disk

type Metadata

type Metadata struct {

	// The following fields are aggregate values of the i3vdir. These values are
	// the totals of the i3vdir and any sub i3vdirs, or are calculated based on
	// all the values in the subtree
	AggregateHealth              float64   `json:"aggregatehealth"`
	AggregateLastHealthCheckTime time.Time `json:"aggregatelasthealthchecktime"`
	AggregateMinRedundancy       float64   `json:"aggregateminredundancy"`
	AggregateModTime             time.Time `json:"aggregatemodtime"`
	AggregateNumFiles            uint64    `json:"aggregatenumfiles"`
	AggregateNumStuckChunks      uint64    `json:"aggregatenumstuckchunks"`
	AggregateNumSubDirs          uint64    `json:"aggregatenumsubdirs"`
	AggregateSize                uint64    `json:"aggregatesize"`
	AggregateStuckHealth         float64   `json:"aggregatestuckhealth"`

	// The following fields are information specific to the i3vdir that is not
	// an aggregate of the entire sub directory tree
	Health              float64     `json:"health"`
	LastHealthCheckTime time.Time   `json:"lasthealthchecktime"`
	MinRedundancy       float64     `json:"minredundancy"`
	Mode                os.FileMode `json:"mode"`
	ModTime             time.Time   `json:"modtime"`
	NumFiles            uint64      `json:"numfiles"`
	NumStuckChunks      uint64      `json:"numstuckchunks"`
	NumSubDirs          uint64      `json:"numsubdirs"`
	Size                uint64      `json:"size"`
	StuckHealth         float64     `json:"stuckhealth"`

	// Version is the used version of the header file.
	Version string `json:"version"`
}

Metadata is the metadata that is saved to disk as a .i3vdir file