ffi

package module
v0.0.0-...-e42303f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 23, 2022 License: Apache-2.0, MIT Imports: 24 Imported by: 0

README

Filecoin Proofs FFI

C and CGO bindings for Filecoin's Rust libraries

Building

To build and install libfilcrypto, its header file and pkg-config manifest, run:

make

To optionally authenticate with GitHub for assets download (to increase API limits) set GITHUB_TOKEN to personal access token.

If no precompiled static library is available for your operating system, the build tooling will attempt to compile a static library from local Rust sources.

Installation notes

By default, building this will download a pre-built binary of the ffi. The advantages for downloading it are faster build times, and not requiring a rust toolchain and build environment.

The disadvantage to downloading the pre-built binary is that it will not be optimized for your specific hardware. This means that if raw performance is of utmost importance to you, it's highly recommended that you build from source.

Building from Source

To opt out of downloading precompiled assets, set FFI_BUILD_FROM_SOURCE=1:

To allow portable building of the blst dependency, set FFI_USE_BLST_PORTABLE=1:

rm .install-filcrypto \
    ; make clean \
    ; FFI_BUILD_FROM_SOURCE=1 FFI_USE_BLST_PORTABLE=1 make

By default, a 'gpu' option is used in the proofs library. This feature is also used in FFI unless explicitly disabled. To disable building with the 'gpu' dependency, set FFI_USE_GPU=0:

rm .install-filcrypto \
    ; make clean \
    ; FFI_BUILD_FROM_SOURCE=1 FFI_USE_GPU=0 make
Experimental CUDA build support

An experimental 'gpu' option using CUDA can be used in the proofs library. This feature is disabled by default (opencl is the default, when FFI_USE_GPU=1 is set.). To enable building with the 'gpu' CUDA dependency, set FFI_USE_CUDA=1 when building from source.

rm .install-filcrypto \
    ; make clean \
    ; FFI_BUILD_FROM_SOURCE=1 FFI_USE_CUDA=1 make

By default, a 'multicore-sdr' option is used in the proofs library. This feature is also used in FFI unless explicitly disabled. To disable building with the 'multicore-sdr' dependency, set FFI_USE_MULTICORE_SDR=0:

rm .install-filcrypto \
    ; make clean \
    ; FFI_BUILD_FROM_SOURCE=1 FFI_USE_MULTICORE_SDR=0 make

Updating rust-fil-proofs (via rust-filecoin-proofs-api)

If rust-fil-proofs has changed from commit X to Y and you wish to get Y into the filecoin-ffi project, you need to do a few things:

  1. Update the cess-proving-system-api Cargo.toml file to point to Y
  2. Run cd rust && cargo update -p "cess-proving-system-api" from the root of the cess-ffi project
  3. After the previous step alters your Cargo.lock file, commit and push

go get

go get needs some additional steps in order to work as expected.

Get the source, add this repo as a submodule to your repo, build it and point to it:

go get github.com/CESSProject/cess-ffi
git submodule add https://github.com/CESSProject/cess-ffi.git internal/ffi
make -C internal/ffi
go mod edit -replace=github.com/CESSProject/cess-ffi=./internal/ffi

Updating CGO Bindings

The CGO bindings are generated using c-for-go and committed to Git. To generate bindings yourself, install the c-for-go binary, ensure that it's on your path, and then run make cgo-gen. CI builds will fail if generated CGO diverges from what's checked into Git.

Updating the Changelog

The mkreleaselog script (in the project root) can be used to generate a good portion of the filecoin-ffi changelog. For historical reasons, the script must be run from the root of a filecoin-ffi checkout which is in your $GOPATH.

Run it like so:

./mkreleaselog v0.25.0 v0.26.0 > /tmp/v0.26.0.notes.txt

License

MIT or Apache 2.0

Documentation

Index

Constants

View Source 
const DigestBytes = 96

DigestBytes is the length of a BLS message hash/digest

View Source 
const PrivateKeyBytes = 32

PrivateKeyBytes is the length of a BLS private key

View Source 
const PublicKeyBytes = 48

PublicKeyBytes is the length of a BLS public key

View Source 
const SignatureBytes = 96

SignatureBytes is the length of a BLS signature

View Source 
const Version int = 3

Version is most similar to semver's minor version. It is here as we cannot use gomod versioning due to local replace directives for native dependencies.

Variables

This section is empty.

Functions

func AggregateSealProofs 

func AggregateSealProofs(aggregateInfo proof5.AggregateSealVerifyProofAndInfos, proofs [][]byte) (out []byte, err error)

TODO AggregateSealProofs it only needs InteractiveRandomness out of the aggregateInfo.Infos

func ClearCache 

func ClearCache(sectorSize uint64, cacheDirPath string) error

ClearCache

func FauxRep 

func FauxRep(proofType abi.RegisteredSealProof, cacheDirPath string, sealedSectorPath string) (cid.Cid, error)

func FauxRep2 

func FauxRep2(proofType abi.RegisteredSealProof, cacheDirPath string, existingPAuxPath string) (cid.Cid, error)

func GeneratePieceCID 

func GeneratePieceCID(proofType abi.RegisteredSealProof, piecePath string, pieceSize abi.UnpaddedPieceSize) (cid.Cid, error)

GeneratePieceCommitment produces a piece commitment for the provided data stored at a given path.

func GeneratePieceCIDFromFile 

func GeneratePieceCIDFromFile(proofType abi.RegisteredSealProof, pieceCesse *os.File, pieceSize abi.UnpaddedPieceSize) (cid.Cid, error)

GeneratePieceCIDFromFile produces a piece CID for the provided data stored in a given file.

func GenerateSingleVanillaProof 

func GenerateSingleVanillaProof(
	replica PrivateSectorInfo,
	challange []uint64,
) ([]byte, error)

func GenerateUnsealedCID 

func GenerateUnsealedCID(proofType abi.RegisteredSealProof, pieces []abi.PieceInfo) (cid.Cid, error)

GenerateDataCommitment produces a commitment for the sector containing the provided pieces.

func GenerateWindowPoSt 

func GenerateWindowPoSt(
	minerID abi.ActorID,
	privateSectorInfo SortedPrivateSectorInfo,
	randomness abi.PoStRandomness,
) ([]proof5.PoStProof, []abi.SectorNumber, error)

GenerateWindowPoSt

func GenerateWindowPoStWithVanilla 

func GenerateWindowPoStWithVanilla(
	proofType abi.RegisteredPoStProof,
	minerID abi.ActorID,
	randomness abi.PoStRandomness,
	proofs [][]byte,
) ([]proof.PoStProof, error)

func GenerateWinningPoSt 

func GenerateWinningPoSt(
	minerID abi.ActorID,
	privateSectorInfo SortedPrivateSectorInfo,
	randomness abi.PoStRandomness,
) ([]proof5.PoStProof, error)

GenerateWinningPoSt

func GenerateWinningPoStSectorChallenge 

func GenerateWinningPoStSectorChallenge(
	proofType abi.RegisteredPoStProof,
	minerID abi.ActorID,
	randomness abi.PoStRandomness,
	eligibleSectorsLen uint64,
) ([]uint64, error)

GenerateWinningPoStSectorChallenge

func GenerateWinningPoStWithVanilla 

func GenerateWinningPoStWithVanilla(
	proofType abi.RegisteredPoStProof,
	minerID abi.ActorID,
	randomness abi.PoStRandomness,
	proofs [][]byte,
) ([]proof.PoStProof, error)

func GetGPUDevices 

func GetGPUDevices() ([]string, error)

GetGPUDevices produces a slice of strings, each representing the name of a detected GPU device.

func GetNumPartitionForFallbackPost 

func GetNumPartitionForFallbackPost(proofType abi.RegisteredPoStProof, numSectors uint) (uint, error)

func GetPoStVersion 

func GetPoStVersion(proofType abi.RegisteredPoStProof) (string, error)

GetPoStVersion

func GetSealVersion 

func GetSealVersion(proofType abi.RegisteredSealProof) (string, error)

GetSealVersion

func HashVerify 

func HashVerify(signature *Signature, messages []Message, publicKeys []PublicKey) bool

HashVerify verifies that a signature is the aggregated signature of hashed messages.

func MergeWindowPoStPartitionProofs 

func MergeWindowPoStPartitionProofs(
	proofType abi.RegisteredPoStProof,
	partitionProofs []PartitionProof,
) (*proof.PoStProof, error)

func SealCommitPhase1 

func SealCommitPhase1(
	proofType abi.RegisteredSealProof,
	sealedCID cid.Cid,
	unsealedCID cid.Cid,
	cacheDirPath string,
	sealedSectorPath string,
	sectorNum abi.SectorNumber,
	minerID abi.ActorID,
	ticket abi.SealRandomness,
	seed abi.InteractiveSealRandomness,
	pieces []abi.PieceInfo,
) (phase1Output []byte, err error)

SealCommitPhase1

func SealCommitPhase2 

func SealCommitPhase2(
	phase1Output []byte,
	sectorNum abi.SectorNumber,
	minerID abi.ActorID,
) ([]byte, error)

SealCommitPhase2

func SealPreCommitPhase1 

func SealPreCommitPhase1(
	proofType abi.RegisteredSealProof,
	cacheDirPath string,
	stagedSectorPath string,
	sealedSectorPath string,
	sectorNum abi.SectorNumber,
	minerID abi.ActorID,
	ticket abi.SealRandomness,
	pieces []abi.PieceInfo,
) (phase1Output []byte, err error)

SealPreCommitPhase1

func SealPreCommitPhase2 

func SealPreCommitPhase2(
	phase1Output []byte,
	cacheDirPath string,
	sealedSectorPath string,
) (sealedCID cid.Cid, unsealedCID cid.Cid, err error)

SealPreCommitPhase2

func Unseal 

func Unseal(
	proofType abi.RegisteredSealProof,
	cacheDirPath string,
	sealedSector *os.File,
	unsealOutput *os.File,
	sectorNum abi.SectorNumber,
	minerID abi.ActorID,
	ticket abi.SealRandomness,
	unsealedCID cid.Cid,
) error

Unseal

func UnsealRange 

func UnsealRange(
	proofType abi.RegisteredSealProof,
	cacheDirPath string,
	sealedSector *os.File,
	unsealOutput *os.File,
	sectorNum abi.SectorNumber,
	minerID abi.ActorID,
	ticket abi.SealRandomness,
	unsealedCID cid.Cid,
	unpaddedByteIndex uint64,
	unpaddedBytesAmount uint64,
) error

UnsealRange

func Verify 

func Verify(signature *Signature, digests []Digest, publicKeys []PublicKey) bool

Verify verifies that a signature is the aggregated signature of digests - pubkeys

func VerifyAggregateSeals 

func VerifyAggregateSeals(aggregate proof5.AggregateSealVerifyProofAndInfos) (bool, error)

func VerifySeal 

func VerifySeal(info proof5.SealVerifyInfo) (bool, error)

VerifySeal returns true if the sealing operation from which its inputs were derived was valid, and false if not.

func VerifyWindowPoSt 

func VerifyWindowPoSt(info proof5.WindowPoStVerifyInfo) (bool, error)

VerifyWindowPoSt returns true if the Winning PoSt-generation operation from which its inputs were derived was valid, and false if not.

func VerifyWinningPoSt 

func VerifyWinningPoSt(info proof5.WinningPoStVerifyInfo) (bool, error)

VerifyWinningPoSt returns true if the Winning PoSt-generation operation from which its inputs were derived was valid, and false if not.

func WorkflowGenerateWinningPoStSectorChallenge 

func WorkflowGenerateWinningPoStSectorChallenge(t TestHelper)

func WorkflowGenerateWinningPoStSectorChallengeEdgeCase 

func WorkflowGenerateWinningPoStSectorChallengeEdgeCase(t TestHelper)

func WorkflowGetGPUDevicesDoesNotProduceAnError 

func WorkflowGetGPUDevicesDoesNotProduceAnError(t TestHelper)

func WorkflowProofsLifecycle 

func WorkflowProofsLifecycle(t TestHelper)

func WorkflowRegisteredPoStProofFunctions 

func WorkflowRegisteredPoStProofFunctions(t TestHelper)

func WorkflowRegisteredSealProofFunctions 

func WorkflowRegisteredSealProofFunctions(t TestHelper)

func WriteWithAlignment 

func WriteWithAlignment(
	proofType abi.RegisteredSealProof,
	pieceCesse *os.File,
	pieceBytes abi.UnpaddedPieceSize,
	stagedSectorCesse *os.File,
	existingPieceSizes []abi.UnpaddedPieceSize,
) (leftAlignment, total abi.UnpaddedPieceSize, pieceCID cid.Cid, retErr error)

WriteWithAlignment

func WriteWithoutAlignment 

func WriteWithoutAlignment(
	proofType abi.RegisteredSealProof,
	pieceCesse *os.File,
	pieceBytes abi.UnpaddedPieceSize,
	stagedSectorCesse *os.File,
) (abi.UnpaddedPieceSize, cid.Cid, error)

WriteWithoutAlignment

Types

type AllocationManager 

type AllocationManager interface {
	Free()
}

AllocationManager is an interface that provides Free() capability.

type Digest 

type Digest [DigestBytes]byte

Digest is a compressed affine

func Hash 

func Hash(message Message) Digest

Hash computes the digest of a message

type FallbackChallenges 

type FallbackChallenges struct {
	Sectors    []abi.SectorNumber
	Challenges map[abi.SectorNumber][]uint64
}

func GeneratePoStFallbackSectorChallenges 

func GeneratePoStFallbackSectorChallenges(
	proofType abi.RegisteredPoStProof,
	minerID abi.ActorID,
	randomness abi.PoStRandomness,
	sectorIds []abi.SectorNumber,
) (*FallbackChallenges, error)

GenerateWinningPoStSectorChallenge

type Message 

type Message []byte

Message is a byte slice

type PartitionProof 

type PartitionProof proof.PoStProof

func GenerateSinglePartitionWindowPoStWithVanilla 

func GenerateSinglePartitionWindowPoStWithVanilla(
	proofType abi.RegisteredPoStProof,
	minerID abi.ActorID,
	randomness abi.PoStRandomness,
	proofs [][]byte,
	partitionIndex uint,
) (*PartitionProof, error)

type PrivateKey 

type PrivateKey [PrivateKeyBytes]byte

PrivateKey is a compressed affine

func PrivateKeyGenerate 

func PrivateKeyGenerate() PrivateKey

PrivateKeyGenerate generates a private key

func PrivateKeyGenerateWithSeed 

func PrivateKeyGenerateWithSeed(seed PrivateKeyGenSeed) PrivateKey

PrivateKeyGenerate generates a private key in a predictable manner

type PrivateKeyGenSeed 

type PrivateKeyGenSeed [32]byte

Used when generating a private key deterministically

type PrivateSectorInfo 

type PrivateSectorInfo struct {
	proof.SectorInfo
	CacheDirPath     string
	PoStProofType    abi.RegisteredPoStProof
	SealedSectorPath string
}

type PublicKey 

type PublicKey [PublicKeyBytes]byte

PublicKey is a compressed affine

func PrivateKeyPublicKey 

func PrivateKeyPublicKey(privateKey PrivateKey) PublicKey

PrivateKeyPublicKey gets the public key for a private key

type Signature 

type Signature [SignatureBytes]byte

Signature is a compressed affine

func Aggregate 

func Aggregate(signatures []Signature) *Signature

Aggregate aggregates signatures together into a new signature. If the provided signatures cannot be aggregated (due to invalid input or an an operational error), Aggregate will return nil.

func CreateZeroSignature 

func CreateZeroSignature() Signature

CreateZeroSignature creates a zero signature, used as placeholder in Cessecoin.

func PrivateKeySign 

func PrivateKeySign(privateKey PrivateKey, message Message) *Signature

PrivateKeySign signs a message

type SortedPrivateSectorInfo 

type SortedPrivateSectorInfo struct {
	// contains filtered or unexported fields
}

SortedPrivateSectorInfo is a slice of PrivateSectorInfo sorted (lexicographically, ascending) by sealed (replica) CID.

func NewSortedPrivateSectorInfo 

func NewSortedPrivateSectorInfo(sectorInfo []PrivateSectorInfo) SortedPrivateSectorInfo

NewSortedPrivateSectorInfo returns a SortedPrivateSectorInfo

func (SortedPrivateSectorInfo) MarshalJSON 

func (s SortedPrivateSectorInfo) MarshalJSON() ([]byte, error)

MarshalJSON JSON-encodes and serializes the SortedPrivateSectorInfo.

func (*SortedPrivateSectorInfo) UnmarshalJSON 

func (s *SortedPrivateSectorInfo) UnmarshalJSON(b []byte) error

func (*SortedPrivateSectorInfo) Values 

func (s *SortedPrivateSectorInfo) Values() []PrivateSectorInfo

Values returns the sorted PrivateSectorInfo as a slice

type SortedPublicSectorInfo 

type SortedPublicSectorInfo struct {
	// contains filtered or unexported fields
}

SortedPublicSectorInfo is a slice of publicSectorInfo sorted (lexicographically, ascending) by sealed (replica) CID.

func (SortedPublicSectorInfo) MarshalJSON 

func (s SortedPublicSectorInfo) MarshalJSON() ([]byte, error)

MarshalJSON JSON-encodes and serializes the SortedPublicSectorInfo.

func (*SortedPublicSectorInfo) UnmarshalJSON 

func (s *SortedPublicSectorInfo) UnmarshalJSON(b []byte) error

UnmarshalJSON parses the JSON-encoded byte slice and stores the result in the value pointed to by s.f. Note that this method allows for construction of a SortedPublicSectorInfo which violates its invariant (that its publicSectorInfo are sorted in some defined way). Callers should take care to never provide a byte slice which would violate this invariant.

func (*SortedPublicSectorInfo) Values 

func (s *SortedPublicSectorInfo) Values() []publicSectorInfo

Values returns the sorted publicSectorInfo as a slice

type TestHelper 

type TestHelper interface {
	AssertEqual(expected, actual interface{}, msgAndArgs ...interface{}) bool
	AssertNoError(err error, msgAndArgs ...interface{}) bool
	AssertTrue(value bool, msgAndArgs ...interface{}) bool
	RequireEqual(expected interface{}, actual interface{}, msgAndArgs ...interface{})
	RequireNoError(err error, msgAndArgs ...interface{})
	RequireTrue(value bool, msgAndArgs ...interface{})
}

type VanillaProof 

type VanillaProof []byte

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.