files

Documentation

Index

• Constants
• type Chunk
• type ChunkSpec
  • func CreateChunkSpec(progress []bool) ChunkSpec
  • func Deserialize(serialized string) (*ChunkSpec, error)
  • func (cs ChunkSpec) Serialize() string
• type FileSharingSubSystem
  • func (fsss *FileSharingSubSystem) CompileChunkRequests(fileKey, serializedManifest, tempFile, title string) []model.PeerMessage
  • func (fsss *FileSharingSubSystem) FetchManifest(fileKey string, manifestSize uint64) model.PeerMessage
  • func (fsss *FileSharingSubSystem) ProcessChunk(chunkKey string, chunk []byte) (fileKey string, progress uint64, totalChunks uint64, chunkID uint64, ...)
  • func (fsss *FileSharingSubSystem) ProcessChunkRequest(fileKey string, serializedChunkRequest []byte) []model.PeerMessage
  • func (fsss *FileSharingSubSystem) ReceiveManifestPart(manifestKey string, part []byte) (fileKey string, serializedManifest string)
  • func (fsss *FileSharingSubSystem) RequestManifestParts(fileKey string) []model.PeerMessage
  • func (fsss *FileSharingSubSystem) ShareFile(fileKey string, serializedManifest string)
  • func (fsss *FileSharingSubSystem) VerifyFile(fileKey string) (tempFile string, filePath string, downloaded bool)
• type Manifest
  • func CreateManifest(path string) (*Manifest, error)
  • func LoadManifest(filename string) (*Manifest, error)
  • func (m *Manifest) Close()
  • func (m *Manifest) GetChunkBytes(id uint64) ([]byte, error)
  • func (m *Manifest) GetChunkRequest() ChunkSpec
  • func (m *Manifest) PrepareDownload() error
  • func (m *Manifest) Save(path string) error
  • func (m *Manifest) Serialize() []byte
  • func (m *Manifest) StoreChunk(id uint64, contents []byte) (uint64, error)
  • func (m *Manifest) VerifyFile() error

Constants

const DefaultChunkSize = 4096

DefaultChunkSize is the default value of a manifest chunk

const MaxManifestSize = 2622000

MaxManifestSize is the maximum size of a manifest (in DefaultChunkSize) Because we reconstruct the manifest in memory we have to practically limit this size. 2622000 * 4096 ~= 10GB using 4096 byte chunks This makes the actual manifest size ~125Mb which seems reasonable for a 10Gb file. most file transfers are expected to have manifest that are much smaller.

Types

type Chunk

type Chunk []byte

Chunk is a wrapper around a hash

type ChunkSpec

type ChunkSpec []uint64

ChunkSpec is a wrapper around an uncompressed array of chunk identifiers

func CreateChunkSpec

func CreateChunkSpec(progress []bool) ChunkSpec

CreateChunkSpec given a full list of chunks with their downloaded status (true for downloaded, false otherwise) derives a list of identifiers of chunks that have not been downloaded yet

func Deserialize

func Deserialize(serialized string) (*ChunkSpec, error)

Deserialize takes in a compressed chunk spec and returns an uncompressed ChunkSpec or an error if the serialized chunk spec has format errors

func (ChunkSpec) Serialize

func (cs ChunkSpec) Serialize() string

Serialize compresses the ChunkSpec into a list of inclusive ranges e.g. 1,2,3,5,6,7 becomes "1:3,5:7"

type FileSharingSubSystem

type FileSharingSubSystem struct {
	// contains filtered or unexported fields
}

FileSharingSubSystem encapsulates the functionality necessary to share and download files via Cwtch

func (*FileSharingSubSystem) CompileChunkRequests

func (fsss *FileSharingSubSystem) CompileChunkRequests(fileKey, serializedManifest, tempFile, title string) []model.PeerMessage

CompileChunkRequests takes in a complete serializedManifest and returns a set of chunk request messages TODO in the future we will want this to return the handles of contacts to request chunks from

func (*FileSharingSubSystem) FetchManifest

func (fsss *FileSharingSubSystem) FetchManifest(fileKey string, manifestSize uint64) model.PeerMessage

FetchManifest given a file key and knowledge of the manifest size in chunks (obtained via an attribute lookup) construct a request to download the manifest.

func (*FileSharingSubSystem) ProcessChunk

func (fsss *FileSharingSubSystem) ProcessChunk(chunkKey string, chunk []byte) (fileKey string, progress uint64, totalChunks uint64, chunkID uint64, title string)

ProcessChunk given a chunk key and a chunk attempt to store and verify the chunk as part of an active download If this results in the file download being completed return downloaded = true Always return the progress of a matched download if it exists along with the total number of chunks and the given chunk ID If not such active download exists then return an empty file key and ignore all further processing.

func (*FileSharingSubSystem) ProcessChunkRequest

func (fsss *FileSharingSubSystem) ProcessChunkRequest(fileKey string, serializedChunkRequest []byte) []model.PeerMessage

ProcessChunkRequest given a fileKey, and a chunk request, compile a set of responses for each requested Chunk

func (*FileSharingSubSystem) ReceiveManifestPart

func (fsss *FileSharingSubSystem) ReceiveManifestPart(manifestKey string, part []byte) (fileKey string, serializedManifest string)

ReceiveManifestPart given a manifestKey reconstruct part the manifest from the provided part

func (*FileSharingSubSystem) RequestManifestParts

func (fsss *FileSharingSubSystem) RequestManifestParts(fileKey string) []model.PeerMessage

RequestManifestParts given a fileKey construct a set of messages representing requests to download various parts of the Manifest

func (*FileSharingSubSystem) ShareFile

func (fsss *FileSharingSubSystem) ShareFile(fileKey string, serializedManifest string)

ShareFile given a file key and a serialized manifest, allow the serialized manifest to be downloaded by Cwtch profiles in possession of the fileKey

func (*FileSharingSubSystem) VerifyFile

func (fsss *FileSharingSubSystem) VerifyFile(fileKey string) (tempFile string, filePath string, downloaded bool)

VerifyFile returns true if the file has been downloaded, false otherwise as well as the temporary filename, if one was used

type Manifest

type Manifest struct {
	Chunks           []Chunk
	FileName         string
	RootHash         []byte
	FileSizeInBytes  uint64
	ChunkSizeInBytes uint64
	TempFileName     string `json:"-"`
	Title            string `json:"-"`
	// contains filtered or unexported fields
}

Manifest is a collection of hashes and other metadata needed to reconstruct a file and verify contents given a root hash

func CreateManifest

func CreateManifest(path string) (*Manifest, error)

CreateManifest takes in a file path and constructs a file sharing manifest of hashes along with other information necessary to download, reconstruct and verify the file.

func LoadManifest

func LoadManifest(filename string) (*Manifest, error)

LoadManifest reads in a json serialized Manifest from a file

func (*Manifest) Close

func (m *Manifest) Close()

Close closes the underlying file descriptor

func (*Manifest) GetChunkBytes

func (m *Manifest) GetChunkBytes(id uint64) ([]byte, error)

GetChunkBytes takes in a chunk identifier and returns the bytes associated with that chunk it does not attempt to validate the chunk Hash.

func (*Manifest) GetChunkRequest

func (m *Manifest) GetChunkRequest() ChunkSpec

GetChunkRequest returns an uncompressed list of Chunks needed to complete the file described in the manifest

func (*Manifest) PrepareDownload

func (m *Manifest) PrepareDownload() error

PrepareDownload creates an empty file of the expected size of the file described by the manifest If the file already exists it assume it is the correct file and that it is resuming from when it left off.

func (*Manifest) Save

func (m *Manifest) Save(path string) error

Save writes a JSON encoded byte array version of the manifest to path

func (*Manifest) Serialize

func (m *Manifest) Serialize() []byte

Serialize returns the manifest as a JSON encoded byte array

func (*Manifest) StoreChunk

func (m *Manifest) StoreChunk(id uint64, contents []byte) (uint64, error)

StoreChunk takes in a chunk id and contents, verifies the chunk has the expected hash and if so store the contents in the file.

func (*Manifest) VerifyFile

func (m *Manifest) VerifyFile() error

VerifyFile attempts to calculate the rootHash of a file and compare it to the expected rootHash stored in the manifest