siadir

package
v1.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 22, 2021 License: MIT Imports: 13 Imported by: 0

README

SiaDir

The SiaDir module is responsible for creating and maintaining the directory metadata information stored in the .siadir files on disk. This includes all disk interaction and metadata definition. These siadirs represent directories on the ScPrime network.

Structure of the SiaDir

The SiaDir is a dir on the ScPrime network and the siadir metadata is a JSON formatted metadata file that contains aggregate and non-aggregate fields. The aggregate fields are the totals of the siadir and any sub siadirs, or are calculated based on all the values in the subtree. The non-aggregate fields are information specific to the siadir that is not an aggregate of the entire sub directory tree

Subsystems

The following subsystems help the SiaDir module execute its responsibilities:

Persistence Subsystem

Key Files

The Persistence subsystem is responsible for the disk interaction with the .siadir files and ensuring safe and performant ACID operations by using the writeaheadlog package. There are two WAL updates that are used, deletion and metadata updates.

The WAL deletion update removes all the contents of the directory including the directory itself.

The WAL metadata update re-writes the entire metadata, which is stored as JSON. This is used whenever the metadata changes and needs to be saved as well as when a new siadir is created.

Exports

  • ApplyUpdates
  • CreateAndApplyTransaction
  • IsSiaDirUpdate
  • New
  • LoadSiaDir
  • UpdateMetadata

Inbound Complexities

  • callDelete deletes a SiaDir from disk
    • SiaDirSet.Delete uses callDelete
  • LoadSiaDir loads a SiaDir from disk
    • SiaDirSet.open uses LoadSiaDir
File Format Subsystem

Key Files

The file format subsystem contains the type definitions for the SiaDir format and methods that return information about the SiaDir.

Exports

  • Deleted
  • Metatdata
  • SiaPath
SiaDirSet Subsystem

Key Files

A SiaDir object is threadsafe by itself, and to ensure that when a SiaDir is accessed by multiple threads that it is still threadsafe, SiaDirs should always be accessed through the SiaDirSet. The SiaDirSet was created as a pool of SiaDirs which is used by other packages to get access to SiaDirEntries which are wrappers for SiaDirs containing some extra information about how many threads are using it at a certain time. If a SiaDir was already loaded the SiaDirSet will hand out the existing object, otherwise it will try to load it from disk.

Exports

  • HealthPercentage
  • NewSiaDirSet
  • Close
  • Delete
  • DirInfo
  • DirList
  • Exists
  • InitRootDir
  • NewSiaDir
  • Open
  • Rename

Outbound Complexities

  • Delete will use callDelete to delete the SiaDir once it has been acquired in the set
  • open calls LoadSiaDir to load the SiaDir from disk
DirReader Subsystem

Key Files

The DirReader Subsystem creates the DirReader which is used as a helper to read raw .siadir from disk

Exports

  • Close
  • Read
  • Stat
  • DirReader

Documentation

Index

Constants

View Source 
const (
	// SiaDirExtension is the name of the metadata file for the sia directory
	SiaDirExtension = ".siadir"

	// DefaultDirHealth is the default health for the directory and the fall
	// back value when there is an error. This is to protect against falsely
	// trying to repair directories that had a read error
	DefaultDirHealth = float64(0)

	// DefaultDirRedundancy is the default redundancy for the directory and the
	// fall back value when there is an error. This is to protect against
	// falsely trying to repair directories that had a read error
	DefaultDirRedundancy = float64(-1)
)

Variables

View Source 
var (
	// ErrDeleted is the error returned if the siadir is deleted
	ErrDeleted = errors.New("siadir is deleted")
)

Functions

func ApplyUpdates 

func ApplyUpdates(updates ...writeaheadlog.Update) error

ApplyUpdates applies a number of writeaheadlog updates to the corresponding SiaDir. This method can apply updates from different SiaDirs and should only be run before the SiaDirs are loaded from disk right after the startup of spd. Otherwise we might run into concurrency issues.

func CreateAndApplyTransaction 

func CreateAndApplyTransaction(wal *writeaheadlog.WAL, updates ...writeaheadlog.Update) (err error)

CreateAndApplyTransaction is a helper method that creates a writeaheadlog transaction and applies it.

func IsSiaDirUpdate 

func IsSiaDirUpdate(update writeaheadlog.Update) bool

IsSiaDirUpdate is a helper method that makes sure that a wal update belongs to the SiaDir package.

Types

type DirReader 

type DirReader struct {
	// contains filtered or unexported fields
}

DirReader is a helper type that allows reading a raw .siadir from disk while keeping the file in memory locked.

func (*DirReader) Close 

func (sdr *DirReader) Close() error

Close closes the underlying file.

func (*DirReader) Read 

func (sdr *DirReader) Read(b []byte) (int, error)

Read calls Read on the underlying file.

func (*DirReader) Stat 

func (sdr *DirReader) Stat() (os.FileInfo, error)

Stat returns the FileInfo of the underlying file.

type Metadata 

type Metadata struct {

	// The following fields are aggregate values of the siadir. These values are
	// the totals of the siadir and any sub siadirs, or are calculated based on
	// all the values in the subtree
	AggregateHealth              float64   `json:"aggregatehealth"`
	AggregateLastHealthCheckTime time.Time `json:"aggregatelasthealthchecktime"`
	AggregateMinRedundancy       float64   `json:"aggregateminredundancy"`
	AggregateModTime             time.Time `json:"aggregatemodtime"`
	AggregateNumFiles            uint64    `json:"aggregatenumfiles"`
	AggregateNumStuckChunks      uint64    `json:"aggregatenumstuckchunks"`
	AggregateNumSubDirs          uint64    `json:"aggregatenumsubdirs"`
	AggregateRemoteHealth        float64   `json:"aggregateremotehealth"`
	AggregateSize                uint64    `json:"aggregatesize"`
	AggregateStuckHealth         float64   `json:"aggregatestuckhealth"`

	// The following fields are information specific to the siadir that is not
	// an aggregate of the entire sub directory tree
	Health              float64     `json:"health"`
	LastHealthCheckTime time.Time   `json:"lasthealthchecktime"`
	MinRedundancy       float64     `json:"minredundancy"`
	Mode                os.FileMode `json:"mode"`
	ModTime             time.Time   `json:"modtime"`
	NumFiles            uint64      `json:"numfiles"`
	NumStuckChunks      uint64      `json:"numstuckchunks"`
	NumSubDirs          uint64      `json:"numsubdirs"`
	RemoteHealth        float64     `json:"remotehealth"`
	Size                uint64      `json:"size"`
	StuckHealth         float64     `json:"stuckhealth"`

	// Version is the used version of the header file.
	Version string `json:"version"`
}

Metadata is the metadata that is saved to disk as a .siadir file

type SiaDir 

type SiaDir struct {
	// contains filtered or unexported fields
}

SiaDir contains the metadata information about a renter directory

func LoadSiaDir 

func LoadSiaDir(path string, deps modules.Dependencies, wal *writeaheadlog.WAL) (sd *SiaDir, err error)

LoadSiaDir loads the directory metadata from disk

func New 

func New(path, rootPath string, mode os.FileMode, wal *writeaheadlog.WAL) (*SiaDir, error)

New creates a new directory in the renter directory and makes sure there is a metadata file in the directory and creates one as needed. This method will also make sure that all the parent directories are created and have metadata files as well and will return the SiaDir containing the information for the directory that matches the siaPath provided

func (*SiaDir) Delete 

func (sd *SiaDir) Delete() error

Delete removes the directory from disk and marks it as deleted. Once the directory is deleted, attempting to access the directory will return an error.

func (*SiaDir) Deleted 

func (sd *SiaDir) Deleted() bool

Deleted returns the deleted field of the siaDir

func (*SiaDir) DirReader 

func (sd *SiaDir) DirReader() (*DirReader, error)

DirReader creates a io.ReadCloser that can be used to read the raw SiaDir from disk.

func (*SiaDir) MDPath 

func (sd *SiaDir) MDPath() string

MDPath returns the path of the SiaDir's metadata on disk.

func (*SiaDir) Metadata 

func (sd *SiaDir) Metadata() Metadata

Metadata returns the metadata of the SiaDir

func (*SiaDir) Path 

func (sd *SiaDir) Path() string

Path returns the path of the SiaDir on disk.

func (*SiaDir) Rename 

func (sd *SiaDir) Rename(targetPath string) error

Rename renames the SiaDir to targetPath.

func (*SiaDir) SetPath 

func (sd *SiaDir) SetPath(targetPath string) error

SetPath sets the path field of the dir.

func (*SiaDir) UpdateBubbledMetadata 

func (sd *SiaDir) UpdateBubbledMetadata(metadata Metadata) error

UpdateBubbledMetadata updates the SiaDir Metadata that is bubbled and saves the changes to disk. For fields that are not bubbled, this method sets them to the current values in the SiaDir metadata

func (*SiaDir) UpdateLastHealthCheckTime 

func (sd *SiaDir) UpdateLastHealthCheckTime(aggregateLastHealthCheckTime, lastHealthCheckTime time.Time) error

UpdateLastHealthCheckTime updates the SiaDir LastHealthCheckTime and AggregateLastHealthCheckTime and saves the changes to disk

func (*SiaDir) UpdateMetadata 

func (sd *SiaDir) UpdateMetadata(metadata Metadata) error

UpdateMetadata updates the SiaDir metadata on disk

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.