renter

package
v0.16.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2020 License: MIT Imports: 24 Imported by: 8

Documentation

Overview

Package renter provides utilities for managing Sia file metadata and for uploading and downloading sectors.

Index

Constants

View Source 
const (
	// MetaFileVersion is the current version of the metafile format. It is
	// incremented after each change to the format.
	MetaFileVersion = 2

	// SectorSliceSize is the encoded size of a SectorSlice.
	SectorSliceSize = 64
)

Variables

View Source 
var ErrBadChecksum = errors.New("sector data failed checksum validation")

ErrBadChecksum indicates that a piece of sector data failed checksum validation.

Functions

func MetaFileCanDownload 

func MetaFileCanDownload(filename string) (bool, error)

MetaFileCanDownload reads a metafile archive and reports whether it can be downloaded.

func MetaFileFullyUploaded 

func MetaFileFullyUploaded(filename string) (bool, error)

MetaFileFullyUploaded reads a metafile archive and reports whether it has been fully uploaded.

func WriteMetaFile added in v0.11.0 

func WriteMetaFile(filename string, m *MetaFile) error

WriteMetaFile creates a gzipped tar archive containing m's index and shards, and writes it to filename. The write is atomic.

Types

type Contract 

type Contract struct {
	HostKey   hostdb.HostPublicKey
	ID        types.FileContractID
	RenterKey ed25519.PrivateKey
}

A Contract identifies a unique file contract and possesses the secret key that can revise it.

type ErasureCoder 

type ErasureCoder interface {
	// Encode encodes data into shards. The resulting shards do not constitute
	// a single matrix, but a series of matrices, each with a shard size of
	// merkletree.SegmentSize. The supplied shards must each have a capacity
	// of at least len(data)/m. Encode may alter the len of the shards.
	Encode(data []byte, shards [][]byte)
	// Reconstruct recalculates any missing shards in the input. Missing
	// shards must have the same capacity as a normal shard, but a length of
	// zero.
	Reconstruct(shards [][]byte) error
	// Recover recalculates any missing data shards and writes them to w,
	// skipping the first off bytes and stopping after n bytes.
	Recover(w io.Writer, shards [][]byte, off, n int) error
}

An ErasureCoder encodes and decodes data to/from a set of shards. The encoding is done piecewise, such that every segment can be decoded individually.

func NewRSCode 

func NewRSCode(m, n int) ErasureCoder

NewRSCode returns an m-of-n ErasureCoder. It panics if m <= 0 or n < m.

type HostKeyResolver added in v0.2.0 

type HostKeyResolver interface {
	ResolveHostKey(pubkey hostdb.HostPublicKey) (modules.NetAddress, error)
}

A HostKeyResolver resolves a host's public key to the most recent NetAddress it announced on the blockchain.

type KeySeed added in v0.2.0 

type KeySeed [32]byte

A KeySeed derives subkeys and uses them to encrypt and decrypt messages.

func (KeySeed) MarshalJSON added in v0.2.0 

func (s KeySeed) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (*KeySeed) UnmarshalJSON added in v0.2.0 

func (s *KeySeed) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*KeySeed) XORKeyStream added in v0.2.0 

func (s *KeySeed) XORKeyStream(msg []byte, nonce []byte, startIndex uint64)

XORKeyStream xors msg with the keystream derived from s, using startIndex as the starting offset within the stream. The nonce must be 24 bytes.

type MetaFile 

type MetaFile struct {
	MetaIndex
	Shards [][]SectorSlice
}

A MetaFile is a set of metadata that represents a file stored on Sia hosts.

func NewMetaFile 

func NewMetaFile(mode os.FileMode, size int64, hosts []hostdb.HostPublicKey, minShards int) *MetaFile

NewMetaFile creates a metafile using the specified hosts and erasure- coding parameters.

func ReadMetaFile added in v0.3.0 

func ReadMetaFile(filename string) (*MetaFile, error)

ReadMetaFile reads a metafile archive into memory.

func (*MetaFile) HostIndex 

func (m *MetaFile) HostIndex(hostKey hostdb.HostPublicKey) int

HostIndex returns the index of the shard that references data stored on the specified host. If m does not reference any data on the host, HostIndex returns -1.

func (*MetaFile) ReplaceHost 

func (m *MetaFile) ReplaceHost(oldHostKey, newHostKey hostdb.HostPublicKey) bool

ReplaceHost replaces a host within the metafile. The shards of the replaced host will not be included in the new archive when Close or Archive is called.

type MetaIndex 

type MetaIndex struct {
	Version   int
	Filesize  int64       // original file size
	Mode      os.FileMode // mode bits
	ModTime   time.Time   // set when Archive is called
	MasterKey KeySeed     // seed from which shard encryption keys are derived
	MinShards int         // number of shards required to recover file
	Hosts     []hostdb.HostPublicKey
}

A MetaIndex contains the traditional file metadata for a MetaFile, along with an encryption key, redundancy parameters, and the set of hosts storing the actual file data.

func ReadMetaIndex 

func ReadMetaIndex(filename string) (MetaIndex, error)

ReadMetaIndex reads the index of a metafile without reading any shards.

func (*MetaIndex) ErasureCode 

func (m *MetaIndex) ErasureCode() ErasureCoder

ErasureCode returns the erasure code used to encode and decode the shards of m.

func (*MetaIndex) MaxChunkSize 

func (m *MetaIndex) MaxChunkSize() int64

MaxChunkSize returns the maximum amount of file data that can fit into a chunk. A chunk is a buffer of file data pre-erasure coding. When the chunk is encoded, it is split into len(m.Hosts) shards of equal size. Thus the MaxChunkSize is the size of such a buffer that results in shards equal to renterhost.SectorSize. MaxChunkSize is NOT guaranteed to match the actual chunk size used in the shard files of m.

func (*MetaIndex) MinChunkSize added in v0.2.0 

func (m *MetaIndex) MinChunkSize() int64

MinChunkSize is the size of the smallest possible chunk. When this chunk is erasure-encoded into shards, each shard will have a length of merkle.SegmentSize, which is the smallest unit of data that the host can provide Merkle proofs for.

func (*MetaIndex) Validate added in v0.8.1 

func (m *MetaIndex) Validate() error

Validate performs basic sanity checks on a MetaIndex.

type SectorBuilder 

type SectorBuilder struct {
	// contains filtered or unexported fields
}

A SectorBuilder facilitates the construction of sectors for later upload. SectorBuilders are particularly useful when packing data from multiple sources into a single sector. The zero value for a SectorBuilder is an empty sector.

func (*SectorBuilder) Append 

func (sb *SectorBuilder) Append(data []byte, key KeySeed) int

Append appends data to the sector being constructed, encrypting it with the given key and chunkIndex. The data must be a multiple of merkle.SegmentSize.

Each call to Append creates a SectorSlice that is accessible via the Slices method, using the index returned by Append.

Append panics if len(data) > sb.Remaining().

func (*SectorBuilder) Finish 

func (sb *SectorBuilder) Finish() *[renterhost.SectorSize]byte

Finish fills the remaining capacity of the sector with random bytes and returns it.

After calling Finish, Len returns renterhost.SectorSize and Remaining returns 0; no more data can be appended until Reset is called.

Finish returns a pointer to sb's internal buffer, so the standard warnings regarding such pointers apply. In particular, the pointer should not be retained after Reset is called.

func (*SectorBuilder) Len 

func (sb *SectorBuilder) Len() int

Len returns the number of bytes appended to the sector.

func (*SectorBuilder) Remaining 

func (sb *SectorBuilder) Remaining() int

Remaining returns the number of bytes remaining in the sector. It is equivalent to renterhost.SectorSize - sb.Len().

func (*SectorBuilder) Reset 

func (sb *SectorBuilder) Reset()

Reset resets the SectorBuilder to its initial state.

Reset does not allocate a new sector buffer; since Finish returns a pointer to the buffer, this pointer should not be retained after Reset is called.

func (*SectorBuilder) SetMerkleRoot added in v0.12.0 

func (sb *SectorBuilder) SetMerkleRoot(root crypto.Hash)

SetMerkleRoot sets the MerkleRoot fields of the SectorSlices tracked by sb.

func (*SectorBuilder) SliceForAppend added in v0.12.0 

func (sb *SectorBuilder) SliceForAppend() []byte

SliceForAppend returns a slice into the unused capacity of the sector. This makes it possible to Append to the sector without allocating new memory.

func (*SectorBuilder) Slices 

func (sb *SectorBuilder) Slices() []SectorSlice

Slices returns the SectorSlices present in the sector. One SectorSlice is returned for each call to Append since the last call to Reset. Slices should only be called after calling SetMerkleRoot.

type SectorSlice 

type SectorSlice struct {
	MerkleRoot   crypto.Hash
	SegmentIndex uint32
	NumSegments  uint32
	Nonce        [24]byte
}

A SectorSlice uniquely identifies a contiguous slice of data stored on a host. Each SectorSlice can only address a single host sector, so multiple SectorSlices may be needed to reference the data comprising a file.

type ShardDownloader 

type ShardDownloader struct {
	Downloader *proto.Session
	Slices     []SectorSlice
	Key        KeySeed
	// contains filtered or unexported fields
}

A ShardDownloader wraps a proto.Session to provide SectorSlice-based data retrieval, transparently decrypting and validating the received data.

func NewShardDownloader 

func NewShardDownloader(m *MetaFile, c Contract, hkr HostKeyResolver) (*ShardDownloader, error)

NewShardDownloader connects to a host and returns a ShardDownloader capable of downloading the SectorSlices of m.

func (*ShardDownloader) Close 

func (d *ShardDownloader) Close() error

Close closes the connection to the host.

func (*ShardDownloader) CopySection added in v0.2.0 

func (d *ShardDownloader) CopySection(w io.Writer, offset, length int64) error

CopySection downloads the requested section of the Shard, decrypts it, and writes it to w.

func (*ShardDownloader) DownloadAndDecrypt 

func (d *ShardDownloader) DownloadAndDecrypt(chunkIndex int64) ([]byte, error)

DownloadAndDecrypt downloads the SectorSlice associated with chunkIndex. The data is decrypted and validated before it is returned. The returned slice is only valid until the next call to DownloadAndDecrypt.

func (*ShardDownloader) HostKey 

func (d *ShardDownloader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

type ShardUploader 

type ShardUploader struct {
	Uploader *proto.Session
	Shard    *[]SectorSlice
	Key      KeySeed
	Sector   SectorBuilder
}

A ShardUploader wraps a proto.Session to provide SectorSlice-based data storage, transparently encrypting and checksumming all data before transferring it to the host.

func NewShardUploader 

func NewShardUploader(m *MetaFile, c Contract, hkr HostKeyResolver, currentHeight types.BlockHeight) (*ShardUploader, error)

NewShardUploader connects to a host and returns a ShardUploader capable of uploading m's data and writing to one of m's Shard files.

func (*ShardUploader) Close 

func (u *ShardUploader) Close() error

Close closes the connection to the host and the Shard file.

func (*ShardUploader) EncryptAndUpload 

func (u *ShardUploader) EncryptAndUpload(data []byte, chunkIndex int64) (SectorSlice, error)

EncryptAndUpload uploads the data associated with chunkIndex, creating a SectorSlice. The data is encrypted and padded to renterhost.SectorSize before it is uploaded. The resulting SectorSlice is written to u.Shard.

func (*ShardUploader) HostKey 

func (u *ShardUploader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

func (*ShardUploader) Upload added in v0.2.0 

func (u *ShardUploader) Upload(chunkIndex int64) error

Upload uploads u.Sector, writing the resulting SectorSlice(s) to u.Shard, starting at offset chunkIndex. Upload does not call Reset on u.Sector.

Source Files

View all Source files

Directories

Path Synopsis
Package proto implements the renter side of the Sia renter-host protocol.
 Package proto implements the renter side of the Sia renter-host protocol.
Package renterutil provides convenience functions for common renter actions.
 Package renterutil provides convenience functions for common renter actions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.