renter

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 16, 2018 License: MIT Imports: 22 Imported by: 8

Documentation

Overview

Package renter provides formats for contracts and files.

Index

Constants

View Source 
const (
	// ContractMagic is the magic string that identifies contract files.
	ContractMagic = "us-contract"

	// ContractHeaderSize is the size in bytes of the contract file header.
	ContractHeaderSize = 11 + 1 + 32 + 64

	// ContractRootOffset is the offset at which the binary-encoded sector Merkle
	// roots of the contract are stored. An offset of 4096 bytes was chosen
	// because it should hold any reasonably-sized transaction and is also the
	// size of a block on most filesystems (and, increasingly, on the underlying
	// hardware).
	ContractRootOffset = 4096

	// ContractArenaSize is the size in bytes of the contract file "arena," which
	// contains the JSON-encoded contract transaction.
	ContractArenaSize = ContractRootOffset - ContractHeaderSize

	// ContractVersion is the current version of the contract file format. It is
	// incremented after each change to the format.
	ContractVersion uint8 = 1
)
View Source 
const (
	// MetaFileVersion is the current version of the meta file format. It is
	// incremented after each change to the format.
	MetaFileVersion = 1
)
View Source 
const SectorSliceSize = 32 + 4 + 4 + 32

SectorSliceSize is the encoded size of a SectorSlice.

Variables

View Source 
var ErrBadChecksum = errors.New("sector data failed checksum validation")

ErrBadChecksum indicates that a piece of sector data failed checksum validation.

Functions

func MetaFileCanDownload 

func MetaFileCanDownload(filename string) (bool, error)

MetaFileCanDownload reads a meta file without extracting it, reporting whether it can be downloaded.

func MetaFileFullyUploaded 

func MetaFileFullyUploaded(filename string) (bool, error)

MetaFileFullyUploaded reads a meta file without extracting it, reporting whether it has been fully uploaded.

func ReadContractTransaction 

func ReadContractTransaction(filename string) (proto.ContractTransaction, error)

ReadContractTransaction reads, decodes, and returns the ContractTransaction of a contract file. The ContractTransaction is not validated.

func SaveContract 

func SaveContract(contract proto.ContractTransaction, filename string) error

SaveContract creates a new contract file using the provided contract. The contract file will not contain any sector Merkle roots.

func SaveRenewedContract 

func SaveRenewedContract(oldContract *Contract, newContract proto.ContractTransaction, filename string) error

SaveRenewedContract creates a new contract file using the provided contract and the sector Merkle roots of the old contract.

Types

type Contract 

type Contract struct {
	proto.ContractTransaction // for convenience
	// contains filtered or unexported fields
}

A Contract represents an open contract file. Contract files contain all the data necessary to revise a file contract.

func LoadContract 

func LoadContract(filename string) (*Contract, error)

LoadContract loads a contract file, including all of its sector Merkle roots, into memory.

func (*Contract) AppendRoot 

func (c *Contract) AppendRoot(root crypto.Hash) (crypto.Hash, error)

AppendRoot appends a sector root to the contract, returning the new top-level Merkle root. The root should be written to durable storage.

func (*Contract) Close 

func (c *Contract) Close() error

Close closes the contract file.

func (*Contract) NumSectors 

func (c *Contract) NumSectors() int

NumSectors returns the number of sector roots in the contract. It does not reflect any pending changes to the roots.

func (*Contract) Revise 

func (c *Contract) Revise(rev types.FileContractRevision) error

Revise sets the latest revision of the file contract.

func (*Contract) SectorRoot 

func (c *Contract) SectorRoot(i int) (crypto.Hash, error)

SectorRoot returns the sector root at index i.

func (*Contract) SyncWithHost 

func (c *Contract) SyncWithHost(hostRevision types.FileContractRevision, hostSignatures []types.TransactionSignature) error

SyncWithHost synchronizes the local version of the contract with the host's version. This may involve modifying the sector roots and/or contract revision. SyncWithHost returns an error iff the contract has permanently desynchronized with the host and recovery is impossible.

func (*Contract) Transaction 

func (c *Contract) Transaction() proto.ContractTransaction

Transaction returns the transaction containing the latest revision of the file contract.

type ContractHeader 

type ContractHeader struct {
	// contains filtered or unexported fields
}

ContractHeader contains the data encoded within the first ContractHeaderSize bytes of the contract file.

func (*ContractHeader) Validate 

func (h *ContractHeader) Validate() error

Validate validates a ContractHeader, checking its magic bytes and version.

type ContractSet 

type ContractSet map[hostdb.HostPublicKey]*Contract

A ContractSet is a map of Contracts keyed by their host public key.

func LoadContracts 

func LoadContracts(dir string) (ContractSet, error)

LoadContracts loads a set of contract files stored in a directory and returns a map that keys the contracts by their host key. Files not ending in .contract are ignored. If multiple contracts have the same host key, LoadContracts returns an error.

func (ContractSet) Close 

func (set ContractSet) Close() error

Close closes all the Contracts in the set.

type EncryptionKey 

type EncryptionKey interface {
	EncryptSegments(ciphertext, plaintext []byte, startIndex uint64)
	DecryptSegments(plaintext, ciphertext []byte, startIndex uint64)
}

An EncryptionKey can encrypt and decrypt segments, where each segment is a []byte with len proto.SegmentSize.

type ErasureCoder 

type ErasureCoder interface {
	Encode(data []byte) [][]byte
	Reconstruct(shards [][]byte) error
	Recover(w io.Writer, shards [][]byte, writeLen int) error
}

An ErasureCoder encodes and decodes data to/from a set of shards.

func NewRSCode 

func NewRSCode(m, n int) ErasureCoder

NewRSCode returns an m-of-n ErasureCoder. It panics if m <= 0 or n < m.

type MetaFile 

type MetaFile struct {
	MetaIndex

	Workdir string
	// contains filtered or unexported fields
}

A MetaFile represents an extracted meta file archive.

func ExtractMetaFile 

func ExtractMetaFile(filename string) (_ *MetaFile, err error)

ExtractMetaFile extracts an existing meta file archive. Like NewMetaFile, it creates a temporary directory to hold the extracted files.

func NewMetaFile 

func NewMetaFile(filename string, mode os.FileMode, size int64, contracts ContractSet, minShards int) (*MetaFile, error)

NewMetaFile creates a meta file using the specified contracts and erasure- coding parameters. The meta file is returned in extracted state, meaning a temporary directory will be created to hold the archive contents. This directory will be removed when Archive is called on the meta file.

func (*MetaFile) Archive 

func (m *MetaFile) Archive(filename string) error

Archive concatenates the meta file index with its referenced shard files and writes the resulting gzipped tar archive to filename.

func (*MetaFile) HostIndex 

func (m *MetaFile) HostIndex(hostKey hostdb.HostPublicKey) int

HostIndex returns the index of the shard that references data stored on the specified host. If m does not reference any data on the host, HostIndex returns -1.

func (*MetaFile) ReplaceHost 

func (m *MetaFile) ReplaceHost(oldHostKey, newHostKey hostdb.HostPublicKey) bool

ReplaceHost replaces a host within the meta file. The shard file corresponding to the replaced host is not deleted until m.Archive is called.

func (*MetaFile) ShardPath 

func (m *MetaFile) ShardPath(hostKey hostdb.HostPublicKey) string

ShardPath returns the canonical path on disk of a shard associated with the given hostKey.

type MetaIndex 

type MetaIndex struct {
	Version   int
	Filesize  int64       // original file size
	Mode      os.FileMode // mode bits
	ModTime   time.Time   // set when Archive is called
	MasterKey keySeed     // seed from which shard encryption keys are derived
	MinShards int         // number of shards required to recover file
	Hosts     []hostdb.HostPublicKey
}

A MetaIndex contains the metadata that ties shards together into a single object with file semantics.

func ReadMetaFileContents 

func ReadMetaFileContents(filename string) (MetaIndex, [][]SectorSlice, error)

ReadMetaFileContents returns the meta file's index and shard slice data.

func ReadMetaIndex 

func ReadMetaIndex(filename string) (MetaIndex, error)

ReadMetaIndex returns the index of a meta file without extracting it.

func (*MetaIndex) EncryptionKey 

func (m *MetaIndex) EncryptionKey(shardIndex int) EncryptionKey

EncryptionKey returns the encryption key used to encrypt sectors in a given shard.

func (*MetaIndex) ErasureCode 

func (m *MetaIndex) ErasureCode() ErasureCoder

ErasureCode returns the erasure code used to encode and decode the shards of m.

func (*MetaIndex) MaxChunkSize 

func (m *MetaIndex) MaxChunkSize() int64

MaxChunkSize returns the maximum amount of file data that can fit into a chunk. A chunk is a buffer of file data pre-erasure coding. When the chunk is encoded, it is split into len(m.Hosts) shards of equal size. Thus the MaxChunkSize is the size of such a buffer that results in shards equal to proto.SectorSize. MaxChunkSize is NOT guaranteed to match the actual chunk size used in the shard files of m.

func (*MetaIndex) MinChunks 

func (m *MetaIndex) MinChunks() int64

MinChunks returns the minimum number of chunks required to fully upload the file. It assumes that each SectorSlice will reference a full sector (proto.SectorSize bytes).

type ScanFn 

type ScanFn func(hostdb.HostPublicKey) (hostdb.ScannedHost, error)

A ScanFn can scan hosts.

type SectorBuilder 

type SectorBuilder struct {
	// contains filtered or unexported fields
}

A SectorBuilder facilitates the construction of sectors for later upload. SectorBuilders are particularly useful when packing data from multiple sources into a single sector. The zero value for a SectorBuilder is an empty sector.

func (*SectorBuilder) Append 

func (sb *SectorBuilder) Append(data []byte, key EncryptionKey, chunkIndex int64)

Append appends data to the sector being constructed, encrypting it with the given key and chunkIndex. The data is also padded with random bytes to the nearest multiple of proto.SegmentSize, which is required by the encryption scheme. Each call to Append creates a SectorSlice that is accessible via the Slices method. This SectorSlice reflects the length and checksum of the original (unpadded, unencrypted) data.

Append panics if len(data) > sb.Remaining().

func (*SectorBuilder) Finish 

func (sb *SectorBuilder) Finish() *[proto.SectorSize]byte

Finish fills the remaining capacity of the sector with random bytes and returns it. The MerkleRoot field of each SectorSlice tracked by sb is set to Merkle root of the resulting sector.

After calling Finish, Len returns proto.SectorSize and Remaining returns 0; no more data can be appended until Reset is called.

func (*SectorBuilder) Len 

func (sb *SectorBuilder) Len() int

Len returns the number of bytes appended to the sector. Note that, due to padding, it is not generally true that Len equals the sum of slices passed to Append.

func (*SectorBuilder) Remaining 

func (sb *SectorBuilder) Remaining() int

Remaining returns the number of bytes remaining in the sector. It is equivalent to proto.SectorSize - sb.Len().

func (*SectorBuilder) Reset 

func (sb *SectorBuilder) Reset()

Reset resets the SectorBuilder to its initial state.

Reset does not allocate a new sector buffer; since Finish returns a pointer to the buffer, this pointer should not be retained after Reset is called.

func (*SectorBuilder) Slices 

func (sb *SectorBuilder) Slices() []SectorSlice

Slices returns the SectorSlices present in the sector. One SectorSlice is returned for each call to Append since the last call to Reset. Slices should only be called after calling Finish; otherwise the MerkleRoot field of each SectorSlice will be unset.

type SectorSlice 

type SectorSlice struct {
	MerkleRoot crypto.Hash
	Offset     uint32
	Length     uint32
	Checksum   crypto.Hash
}

A SectorSlice is the unit element of a shard file. Each SectorSlice uniquely identifies a contiguous slice of data stored on a host.

func ReadShard 

func ReadShard(filename string) ([]SectorSlice, error)

ReadShard loads the slices of a shard file into memory.

type Shard 

type Shard struct {
	// contains filtered or unexported fields
}

A Shard is a shard file open for writing.

func OpenShard 

func OpenShard(filename string) (*Shard, error)

OpenShard opens a shard file for writing, creating it if necessary. If the file's size is not a multiple of SectorSliceSize, it is truncated accordingly.

func (*Shard) Close 

func (s *Shard) Close() error

Close closes the shard file.

func (*Shard) WriteSlice 

func (s *Shard) WriteSlice(slice SectorSlice, index int64) error

WriteSlice writes slice at the specified index, growing the underlying file as necessary.

type ShardDownloader 

type ShardDownloader struct {
	Downloader *proto.Downloader
	Slices     []SectorSlice
	Key        EncryptionKey
}

A ShardDownloader wraps a proto.Downloader to provide SectorSlice-based data retrieval, transparently decrypting and validating the received data.

func NewShardDownloader 

func NewShardDownloader(m *MetaFile, contract *Contract, scan ScanFn) (*ShardDownloader, error)

NewShardDownloader connects to a host and returns a ShardDownloader capable of downloading the SectorSlices of m.

func (*ShardDownloader) Close 

func (d *ShardDownloader) Close() error

Close closes the connection to the host.

func (*ShardDownloader) DownloadAndDecrypt 

func (d *ShardDownloader) DownloadAndDecrypt(chunkIndex int64) ([]byte, error)

DownloadAndDecrypt downloads the SectorSlice associated with chunkIndex. The data is decrypted and validated before it is returned.

func (*ShardDownloader) HostKey 

func (d *ShardDownloader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

type ShardUploader 

type ShardUploader struct {
	Uploader *proto.Uploader
	Shard    *Shard
	Key      EncryptionKey
	// contains filtered or unexported fields
}

A ShardUploader wraps a proto.Uploader to provide SectorSlice-based data storage, transparently encrypting and checksumming all data before transferring it to the host.

func NewShardUploader 

func NewShardUploader(m *MetaFile, hostIndex int, contract *Contract, scan ScanFn, currentHeight types.BlockHeight) (*ShardUploader, error)

NewShardUploader connects to a host and returns a ShardUploader capable of uploading m's data and writing to one of m's Shard files.

func (*ShardUploader) Close 

func (u *ShardUploader) Close() error

Close closes the connection to the host and the Shard file.

func (*ShardUploader) EncryptAndUpload 

func (u *ShardUploader) EncryptAndUpload(data []byte, chunkIndex int64) (SectorSlice, error)

EncryptAndUpload uploads the data associated with chunkIndex, creating a SectorSlice. The data is encrypted and padded to proto.SectorSize before it is uploaded. The resulting SectorSlice is written to u.Shard.

func (*ShardUploader) HostKey 

func (u *ShardUploader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

Source Files

View all Source files

Directories

Path Synopsis
Package proto implements the Sia renter-host protocol.
 Package proto implements the Sia renter-host protocol.
Package renterutil provides convenience functions for common renter actions.
 Package renterutil provides convenience functions for common renter actions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.