siadir

README

SiaDir

The SiaDir module is responsible for creating and maintaining the directory metadata information stored in the .siadir files on disk. This includes all disk interaction and metadata definition. These siadirs represent directories on the Sia network.

Structure of the SiaDir

The SiaDir is a dir on the Sia network and the siadir metadata is a JSON formatted metadata file that contains aggregate and non-aggregate fields. The aggregate fields are the totals of the siadir and any sub siadirs, or are calculated based on all the values in the subtree. The non-aggregate fields are information specific to the siadir that is not an aggregate of the entire sub directory tree

Subsystems

The following subsystems help the SiaDir module execute its responsibilities:

• Persistence Subsystem
• File Format Subsystem
• SiaDirSet Subsystem
• DirReader Subsystem

Persistence Subsystem

Key Files

• persist.go

The Persistence subsystem is responsible for the disk interaction with the .siadir files. All the information stored in the .siadir file is metadata that can be recalculated on the fly. Because of this, the persistence is not ACID. The persistence relies on a checksum at the beginning of the file to know whether or not the file is corrupt.

Exports

• New
• LoadSiaDir
• UpdateMetadata

Inbound Complexities

• callDelete deletes a SiaDir from disk
  • SiaDirSet.Delete uses callDelete
• LoadSiaDir loads a SiaDir from disk
  • SiaDirSet.open uses LoadSiaDir

File Format Subsystem

Key Files

• siadir.go

The file format subsystem contains the type definitions for the SiaDir format and methods that return information about the SiaDir.

Exports

• Deleted
• Metatdata
• SiaPath

SiaDirSet Subsystem

Key Files

• siadirset.go

A SiaDir object is threadsafe by itself, and to ensure that when a SiaDir is accessed by multiple threads that it is still threadsafe, SiaDirs should always be accessed through the SiaDirSet. The SiaDirSet was created as a pool of SiaDirs which is used by other packages to get access to SiaDirEntries which are wrappers for SiaDirs containing some extra information about how many threads are using it at a certain time. If a SiaDir was already loaded the SiaDirSet will hand out the existing object, otherwise it will try to load it from disk.

Exports

• HealthPercentage
• NewSiaDirSet
• Close
• Delete
• DirInfo
• DirList
• Exists
• InitRootDir
• NewSiaDir
• Open
• Rename

Outbound Complexities

• Delete will use callDelete to delete the SiaDir once it has been acquired in the set
• open calls LoadSiaDir to load the SiaDir from disk

DirReader Subsystem

Key Files

• dirreader.go

The DirReader Subsystem creates the DirReader which is used as a helper to read raw .siadir from disk

Exports

• Close
• Read
• Stat
• DirReader

Documentation

Index

• Constants
• Variables
• type DirReader
  • func (sdr *DirReader) Close() error
  • func (sdr *DirReader) Read(b []byte) (int, error)
  • func (sdr *DirReader) Stat() (os.FileInfo, error)
• type Metadata
• type SiaDir
  • func LoadSiaDir(path string, deps modules.Dependencies) (sd *SiaDir, err error)
  • func New(fullPath, rootPath string, mode os.FileMode) (*SiaDir, error)
  • func (sd *SiaDir) Delete() error
  • func (sd *SiaDir) Deleted() bool
  • func (sd *SiaDir) DirReader() (*DirReader, error)
  • func (sd *SiaDir) MDPath() string
  • func (sd *SiaDir) Metadata() Metadata
  • func (sd *SiaDir) Path() string
  • func (sd *SiaDir) Rename(targetPath string) error
  • func (sd *SiaDir) SetPath(targetPath string) error
  • func (sd *SiaDir) UpdateBubbledMetadata(metadata Metadata) error
  • func (sd *SiaDir) UpdateLastHealthCheckTime(aggregateLastHealthCheckTime, lastHealthCheckTime time.Time) error
  • func (sd *SiaDir) UpdateMetadata(metadata Metadata) error

Constants

const (
	// SiaDirExtension is the name of the metadata file for the sia directory
	SiaDirExtension = ".siadir"

	// DefaultDirHealth is the default health for the directory and the fall
	// back value when there is an error. This is to protect against falsely
	// trying to repair directories that had a read error
	DefaultDirHealth = float64(0)

	// DefaultDirRedundancy is the default redundancy for the directory and the
	// fall back value when there is an error. This is to protect against
	// falsely trying to repair directories that had a read error
	DefaultDirRedundancy = float64(-1)
)

Variables

var (
	// ErrDeleted is the error returned if the siadir is deleted
	ErrDeleted = errors.New("siadir is deleted")

	// ErrCorruptFile is the error returned if the siadir is believed to be
	// corrupt
	ErrCorruptFile = errors.New(".siadir file is potentially corrupt")

	// ErrInvalidChecksum is the error returned if the siadir checksum is invalid
	ErrInvalidChecksum = errors.New(".siadir has invalid checksum")
)

Types

type DirReader

type DirReader struct {
	// contains filtered or unexported fields
}

DirReader is a helper type that allows reading a raw .siadir from disk while keeping the file in memory locked.

func (*DirReader) Close

func (sdr *DirReader) Close() error

Close closes the underlying file.

func (*DirReader) Read

func (sdr *DirReader) Read(b []byte) (int, error)

Read calls Read on the underlying file.

func (*DirReader) Stat

func (sdr *DirReader) Stat() (os.FileInfo, error)

Stat returns the FileInfo of the underlying file.

type Metadata

type Metadata struct {

	// The following fields are aggregate values of the siadir. These values are
	// the totals of the siadir and any sub siadirs, or are calculated based on
	// all the values in the subtree
	AggregateHealth              float64   `json:"aggregatehealth"`
	AggregateLastHealthCheckTime time.Time `json:"aggregatelasthealthchecktime"`
	AggregateMinRedundancy       float64   `json:"aggregateminredundancy"`
	AggregateModTime             time.Time `json:"aggregatemodtime"`
	AggregateNumFiles            uint64    `json:"aggregatenumfiles"`
	AggregateNumStuckChunks      uint64    `json:"aggregatenumstuckchunks"`
	AggregateNumSubDirs          uint64    `json:"aggregatenumsubdirs"`
	AggregateRemoteHealth        float64   `json:"aggregateremotehealth"`
	AggregateRepairSize          uint64    `json:"aggregaterepairsize"`
	AggregateSize                uint64    `json:"aggregatesize"`
	AggregateStuckHealth         float64   `json:"aggregatestuckhealth"`
	AggregateStuckSize           uint64    `json:"aggregatestucksize"`

	// The following fields are information specific to the siadir that is not
	// an aggregate of the entire sub directory tree
	Health              float64     `json:"health"`
	LastHealthCheckTime time.Time   `json:"lasthealthchecktime"`
	MinRedundancy       float64     `json:"minredundancy"`
	Mode                os.FileMode `json:"mode"`
	ModTime             time.Time   `json:"modtime"`
	NumFiles            uint64      `json:"numfiles"`
	NumStuckChunks      uint64      `json:"numstuckchunks"`
	NumSubDirs          uint64      `json:"numsubdirs"`
	RemoteHealth        float64     `json:"remotehealth"`
	RepairSize          uint64      `json:"repairsize"`
	Size                uint64      `json:"size"`
	StuckHealth         float64     `json:"stuckhealth"`
	StuckSize           uint64      `json:"stucksize"`

	// Version is the used version of the header file.
	Version string `json:"version"`
}

Metadata is the metadata that is saved to disk as a .siadir file

type SiaDir

type SiaDir struct {
	// contains filtered or unexported fields
}

SiaDir contains the metadata information about a renter directory

func LoadSiaDir

func LoadSiaDir(path string, deps modules.Dependencies) (sd *SiaDir, err error)

LoadSiaDir loads the directory metadata from disk

func New

func New(fullPath, rootPath string, mode os.FileMode) (*SiaDir, error)

New creates a new directory in the renter directory and makes sure there is a metadata file in the directory and creates one as needed. This method will also make sure that all the parent directories are created and have metadata files as well and will return the SiaDir containing the information for the directory that matches the siaPath provided

NOTE: the fullPath is expected to include the rootPath. The rootPath is used to determine when to stop recursively creating siadir metadata.

func (*SiaDir) Delete

func (sd *SiaDir) Delete() error

Delete removes the directory from disk and marks it as deleted. Once the directory is deleted, attempting to access the directory will return an error.

func (*SiaDir) Deleted

func (sd *SiaDir) Deleted() bool

Deleted returns the deleted field of the siaDir

func (*SiaDir) DirReader

func (sd *SiaDir) DirReader() (*DirReader, error)

DirReader creates a io.ReadCloser that can be used to read the raw SiaDir from disk.

func (*SiaDir) MDPath

func (sd *SiaDir) MDPath() string

MDPath returns the path of the SiaDir's metadata on disk.

func (*SiaDir) Metadata

func (sd *SiaDir) Metadata() Metadata

Metadata returns the metadata of the SiaDir

func (*SiaDir) Path

func (sd *SiaDir) Path() string

Path returns the path of the SiaDir on disk.

func (*SiaDir) Rename

func (sd *SiaDir) Rename(targetPath string) error

Rename renames the SiaDir to targetPath.

func (*SiaDir) SetPath

func (sd *SiaDir) SetPath(targetPath string) error

SetPath sets the path field of the dir.

func (*SiaDir) UpdateBubbledMetadata

func (sd *SiaDir) UpdateBubbledMetadata(metadata Metadata) error

UpdateBubbledMetadata updates the SiaDir Metadata that is bubbled and saves the changes to disk. For fields that are not bubbled, this method sets them to the current values in the SiaDir metadata

func (*SiaDir) UpdateLastHealthCheckTime

func (sd *SiaDir) UpdateLastHealthCheckTime(aggregateLastHealthCheckTime, lastHealthCheckTime time.Time) error

UpdateLastHealthCheckTime updates the SiaDir LastHealthCheckTime and AggregateLastHealthCheckTime and saves the changes to disk

func (*SiaDir) UpdateMetadata

func (sd *SiaDir) UpdateMetadata(metadata Metadata) error

UpdateMetadata updates the SiaDir metadata on disk