README
¶
Filecoin Proofs FFI
C and CGO bindings for Filecoin's Rust libraries
Building
To build and install libfilcrypto, its header file and pkg-config manifest, run:
make
To optionally authenticate with GitHub for assets download (to increase API limits)
set GITHUB_TOKEN
to personal access token.
If no precompiled static library is available for your operating system, the build tooling will attempt to compile a static library from local Rust sources.
Installation notes
By default, building this will download a pre-built binary of the ffi. The advantages for downloading it are faster build times, and not requiring a rust toolchain and build environment.
The disadvantage to downloading the pre-built binary is that it will not be optimized for your specific hardware. This means that if raw performance is of utmost importance to you, it's highly recommended that you build from source.
Building from Source
To opt out of downloading precompiled assets, set FFI_BUILD_FROM_SOURCE=1
:
rm .install-filcrypto \
; make clean \
; FFI_BUILD_FROM_SOURCE=1 make
To allow portable building of the blst
dependency, set FFI_USE_BLST_PORTABLE=1
:
rm .install-filcrypto \
; make clean \
; FFI_BUILD_FROM_SOURCE=1 FFI_USE_BLST_PORTABLE=1 make
By default, a 'gpu' option is used in the proofs library. There is now an experimental 'gpu2' feature which has improved performance. To allow building with the 'gpu2' dependency, set FFI_USE_GPU2=1
:
rm .install-filcrypto \
; make clean \
; FFI_BUILD_FROM_SOURCE=1 FFI_USE_BLST=1 FFI_USE_GPU2=1 make
Updating rust-fil-proofs (via rust-filecoin-proofs-api)
If rust-fil-proofs has changed from commit X to Y and you wish to get Y into the filecoin-ffi project, you need to do a few things:
- Update the rust-filecoin-proofs-api Cargo.toml file to point to Y
- Run
cd rust && cargo update -p "filecoin-proofs-api"
from the root of the filecoin-ffi project - After the previous step alters your Cargo.lock file, commit and push
go get
go get
needs some additional steps in order to work as expected.
Get the source, add this repo as a submodule to your repo, build it and point to it:
$ go get github.com/chenjianmei111/filecoin-ffi
$ git submodule add https://github.com/chenjianmei111/filecoin-ffi.git extern/filecoin-ffi
$ make -C extern/filecoin-ffi
$ go mod edit -replace=github.com/chenjianmei111/filecoin-ffi=./extern/filecoin-ffi
Updating CGO Bindings
The CGO bindings are generated using c-for-go
and committed to Git. To generate bindings yourself, install the c-for-go
binary, ensure that it's on your path, and then run make cgo-gen
. CI builds
will fail if generated CGO diverges from what's checked into Git.
Updating the Changelog
The mkreleaselog
script (in the project root) can be used to generate a good
portion of the filecoin-ffi changelog. For historical reasons, the script must
be run from the root of a filecoin-ffi checkout which is in your $GOPATH
.
Run it like so:
./mkreleaselog v0.25.0 v0.26.0 > /tmp/v0.26.0.notes.txt
License
MIT or Apache 2.0
Documentation
¶
Index ¶
- Constants
- func ClearCache(sectorSize uint64, cacheDirPath string) error
- func FauxRep(proofType abi.RegisteredSealProof, cacheDirPath string, ...) (cid.Cid, error)
- func FauxRep2(proofType abi.RegisteredSealProof, cacheDirPath string, ...) (cid.Cid, error)
- func GeneratePieceCID(proofType abi.RegisteredSealProof, piecePath string, ...) (cid.Cid, error)
- func GeneratePieceCIDFromFile(proofType abi.RegisteredSealProof, pieceFile *os.File, ...) (cid.Cid, error)
- func GenerateSingleVanillaProof(replica PrivateSectorInfo, challange []uint64) ([]byte, error)
- func GenerateUnsealedCID(proofType abi.RegisteredSealProof, pieces []abi.PieceInfo) (cid.Cid, error)
- func GenerateWindowPoSt(minerID abi.ActorID, privateSectorInfo SortedPrivateSectorInfo, ...) ([]proof.PoStProof, []abi.SectorNumber, error)
- func GenerateWindowPoStWithVanilla(proofType abi.RegisteredPoStProof, minerID abi.ActorID, ...) ([]proof.PoStProof, error)
- func GenerateWinningPoSt(minerID abi.ActorID, privateSectorInfo SortedPrivateSectorInfo, ...) ([]proof.PoStProof, error)
- func GenerateWinningPoStSectorChallenge(proofType abi.RegisteredPoStProof, minerID abi.ActorID, ...) ([]uint64, error)
- func GenerateWinningPoStWithVanilla(proofType abi.RegisteredPoStProof, minerID abi.ActorID, ...) ([]proof.PoStProof, error)
- func GetGPUDevices() ([]string, error)
- func GetPoStVersion(proofType abi.RegisteredPoStProof) (string, error)
- func GetSealVersion(proofType abi.RegisteredSealProof) (string, error)
- func HashVerify(signature *Signature, messages []Message, publicKeys []PublicKey) bool
- func SealCommitPhase1(proofType abi.RegisteredSealProof, sealedCID cid.Cid, unsealedCID cid.Cid, ...) (phase1Output []byte, err error)
- func SealCommitPhase2(phase1Output []byte, sectorNum abi.SectorNumber, minerID abi.ActorID) ([]byte, error)
- func SealPreCommitPhase1(proofType abi.RegisteredSealProof, cacheDirPath string, ...) (phase1Output []byte, err error)
- func SealPreCommitPhase2(phase1Output []byte, cacheDirPath string, sealedSectorPath string) (sealedCID cid.Cid, unsealedCID cid.Cid, err error)
- func Unseal(proofType abi.RegisteredSealProof, cacheDirPath string, sealedSector *os.File, ...) error
- func UnsealRange(proofType abi.RegisteredSealProof, cacheDirPath string, sealedSector *os.File, ...) error
- func Verify(signature *Signature, digests []Digest, publicKeys []PublicKey) bool
- func VerifySeal(info proof.SealVerifyInfo) (bool, error)
- func VerifyWindowPoSt(info proof.WindowPoStVerifyInfo) (bool, error)
- func VerifyWinningPoSt(info proof.WinningPoStVerifyInfo) (bool, error)
- func WorkflowGenerateWinningPoStSectorChallenge(t TestHelper)
- func WorkflowGenerateWinningPoStSectorChallengeEdgeCase(t TestHelper)
- func WorkflowGetGPUDevicesDoesNotProduceAnError(t TestHelper)
- func WorkflowProofsLifecycle(t TestHelper)
- func WorkflowRegisteredPoStProofFunctions(t TestHelper)
- func WorkflowRegisteredSealProofFunctions(t TestHelper)
- func WriteWithAlignment(proofType abi.RegisteredSealProof, pieceFile *os.File, ...) (leftAlignment, total abi.UnpaddedPieceSize, pieceCID cid.Cid, retErr error)
- func WriteWithoutAlignment(proofType abi.RegisteredSealProof, pieceFile *os.File, ...) (abi.UnpaddedPieceSize, cid.Cid, error)
- type AllocationManager
- type Digest
- type FallbackChallenges
- type Message
- type PrivateKey
- type PrivateKeyGenSeed
- type PrivateSectorInfo
- type PublicKey
- type Signature
- type SortedPrivateSectorInfo
- type SortedPublicSectorInfo
- type TestHelper
- type VanillaProof
Constants ¶
const DigestBytes = 96
DigestBytes is the length of a BLS message hash/digest
const PrivateKeyBytes = 32
PrivateKeyBytes is the length of a BLS private key
const PublicKeyBytes = 48
PublicKeyBytes is the length of a BLS public key
const SignatureBytes = 96
SignatureBytes is the length of a BLS signature
const Version int = 2
Version is most similar to semver's minor version. It is here as we cannot use gomod versioning due to local replace directives for native dependencies.
Variables ¶
This section is empty.
Functions ¶
func FauxRep ¶
func FauxRep(proofType abi.RegisteredSealProof, cacheDirPath string, sealedSectorPath string) (cid.Cid, error)
func FauxRep2 ¶ added in v0.30.10
func FauxRep2(proofType abi.RegisteredSealProof, cacheDirPath string, existingPAuxPath string) (cid.Cid, error)
func GeneratePieceCID ¶
func GeneratePieceCID(proofType abi.RegisteredSealProof, piecePath string, pieceSize abi.UnpaddedPieceSize) (cid.Cid, error)
GeneratePieceCommitment produces a piece commitment for the provided data stored at a given path.
func GeneratePieceCIDFromFile ¶
func GeneratePieceCIDFromFile(proofType abi.RegisteredSealProof, pieceFile *os.File, pieceSize abi.UnpaddedPieceSize) (cid.Cid, error)
GeneratePieceCIDFromFile produces a piece CID for the provided data stored in a given file.
func GenerateSingleVanillaProof ¶ added in v0.30.10
func GenerateSingleVanillaProof( replica PrivateSectorInfo, challange []uint64, ) ([]byte, error)
func GenerateUnsealedCID ¶
func GenerateUnsealedCID(proofType abi.RegisteredSealProof, pieces []abi.PieceInfo) (cid.Cid, error)
GenerateDataCommitment produces a commitment for the sector containing the provided pieces.
func GenerateWindowPoSt ¶
func GenerateWindowPoSt( minerID abi.ActorID, privateSectorInfo SortedPrivateSectorInfo, randomness abi.PoStRandomness, ) ([]proof.PoStProof, []abi.SectorNumber, error)
GenerateWindowPoSt
func GenerateWindowPoStWithVanilla ¶ added in v0.30.10
func GenerateWindowPoStWithVanilla( proofType abi.RegisteredPoStProof, minerID abi.ActorID, randomness abi.PoStRandomness, proofs [][]byte, ) ([]proof.PoStProof, error)
func GenerateWinningPoSt ¶
func GenerateWinningPoSt( minerID abi.ActorID, privateSectorInfo SortedPrivateSectorInfo, randomness abi.PoStRandomness, ) ([]proof.PoStProof, error)
GenerateWinningPoSt
func GenerateWinningPoStSectorChallenge ¶
func GenerateWinningPoStSectorChallenge( proofType abi.RegisteredPoStProof, minerID abi.ActorID, randomness abi.PoStRandomness, eligibleSectorsLen uint64, ) ([]uint64, error)
GenerateWinningPoStSectorChallenge
func GenerateWinningPoStWithVanilla ¶ added in v0.30.10
func GenerateWinningPoStWithVanilla( proofType abi.RegisteredPoStProof, minerID abi.ActorID, randomness abi.PoStRandomness, proofs [][]byte, ) ([]proof.PoStProof, error)
func GetGPUDevices ¶
GetGPUDevices produces a slice of strings, each representing the name of a detected GPU device.
func GetPoStVersion ¶
func GetPoStVersion(proofType abi.RegisteredPoStProof) (string, error)
GetPoStVersion
func GetSealVersion ¶
func GetSealVersion(proofType abi.RegisteredSealProof) (string, error)
GetSealVersion
func HashVerify ¶
HashVerify verifies that a signature is the aggregated signature of hashed messages.
func SealCommitPhase1 ¶
func SealCommitPhase1( proofType abi.RegisteredSealProof, sealedCID cid.Cid, unsealedCID cid.Cid, cacheDirPath string, sealedSectorPath string, sectorNum abi.SectorNumber, minerID abi.ActorID, ticket abi.SealRandomness, seed abi.InteractiveSealRandomness, pieces []abi.PieceInfo, ) (phase1Output []byte, err error)
SealCommitPhase1
func SealCommitPhase2 ¶
func SealCommitPhase2( phase1Output []byte, sectorNum abi.SectorNumber, minerID abi.ActorID, ) ([]byte, error)
SealCommitPhase2
func SealPreCommitPhase1 ¶
func SealPreCommitPhase1( proofType abi.RegisteredSealProof, cacheDirPath string, stagedSectorPath string, sealedSectorPath string, sectorNum abi.SectorNumber, minerID abi.ActorID, ticket abi.SealRandomness, pieces []abi.PieceInfo, ) (phase1Output []byte, err error)
SealPreCommitPhase1
func SealPreCommitPhase2 ¶
func SealPreCommitPhase2( phase1Output []byte, cacheDirPath string, sealedSectorPath string, ) (sealedCID cid.Cid, unsealedCID cid.Cid, err error)
SealPreCommitPhase2
func Unseal ¶
func Unseal( proofType abi.RegisteredSealProof, cacheDirPath string, sealedSector *os.File, unsealOutput *os.File, sectorNum abi.SectorNumber, minerID abi.ActorID, ticket abi.SealRandomness, unsealedCID cid.Cid, ) error
Unseal
func UnsealRange ¶
func UnsealRange( proofType abi.RegisteredSealProof, cacheDirPath string, sealedSector *os.File, unsealOutput *os.File, sectorNum abi.SectorNumber, minerID abi.ActorID, ticket abi.SealRandomness, unsealedCID cid.Cid, unpaddedByteIndex uint64, unpaddedBytesAmount uint64, ) error
UnsealRange
func VerifySeal ¶
func VerifySeal(info proof.SealVerifyInfo) (bool, error)
VerifySeal returns true if the sealing operation from which its inputs were derived was valid, and false if not.
func VerifyWindowPoSt ¶
func VerifyWindowPoSt(info proof.WindowPoStVerifyInfo) (bool, error)
VerifyWindowPoSt returns true if the Winning PoSt-generation operation from which its inputs were derived was valid, and false if not.
func VerifyWinningPoSt ¶
func VerifyWinningPoSt(info proof.WinningPoStVerifyInfo) (bool, error)
VerifyWinningPoSt returns true if the Winning PoSt-generation operation from which its inputs were derived was valid, and false if not.
func WorkflowGenerateWinningPoStSectorChallenge ¶
func WorkflowGenerateWinningPoStSectorChallenge(t TestHelper)
func WorkflowGenerateWinningPoStSectorChallengeEdgeCase ¶
func WorkflowGenerateWinningPoStSectorChallengeEdgeCase(t TestHelper)
func WorkflowGetGPUDevicesDoesNotProduceAnError ¶
func WorkflowGetGPUDevicesDoesNotProduceAnError(t TestHelper)
func WorkflowProofsLifecycle ¶
func WorkflowProofsLifecycle(t TestHelper)
func WorkflowRegisteredPoStProofFunctions ¶
func WorkflowRegisteredPoStProofFunctions(t TestHelper)
func WorkflowRegisteredSealProofFunctions ¶
func WorkflowRegisteredSealProofFunctions(t TestHelper)
func WriteWithAlignment ¶
func WriteWithAlignment( proofType abi.RegisteredSealProof, pieceFile *os.File, pieceBytes abi.UnpaddedPieceSize, stagedSectorFile *os.File, existingPieceSizes []abi.UnpaddedPieceSize, ) (leftAlignment, total abi.UnpaddedPieceSize, pieceCID cid.Cid, retErr error)
WriteWithAlignment
func WriteWithoutAlignment ¶
func WriteWithoutAlignment( proofType abi.RegisteredSealProof, pieceFile *os.File, pieceBytes abi.UnpaddedPieceSize, stagedSectorFile *os.File, ) (abi.UnpaddedPieceSize, cid.Cid, error)
WriteWithoutAlignment
Types ¶
type AllocationManager ¶ added in v0.30.10
type AllocationManager interface {
Free()
}
AllocationManager is an interface that provides Free() capability.
type FallbackChallenges ¶ added in v0.30.10
type FallbackChallenges struct { Sectors []abi.SectorNumber Challenges map[abi.SectorNumber][]uint64 }
func GeneratePoStFallbackSectorChallenges ¶ added in v0.30.10
func GeneratePoStFallbackSectorChallenges( proofType abi.RegisteredPoStProof, minerID abi.ActorID, randomness abi.PoStRandomness, sectorIds []abi.SectorNumber, ) (*FallbackChallenges, error)
GenerateWinningPoStSectorChallenge
type PrivateKey ¶
type PrivateKey [PrivateKeyBytes]byte
PrivateKey is a compressed affine
func PrivateKeyGenerate ¶
func PrivateKeyGenerate() PrivateKey
PrivateKeyGenerate generates a private key
func PrivateKeyGenerateWithSeed ¶
func PrivateKeyGenerateWithSeed(seed PrivateKeyGenSeed) PrivateKey
PrivateKeyGenerate generates a private key in a predictable manner
type PrivateKeyGenSeed ¶
type PrivateKeyGenSeed [32]byte
Used when generating a private key deterministically
type PrivateSectorInfo ¶
type PrivateSectorInfo struct { proof.SectorInfo CacheDirPath string PoStProofType abi.RegisteredPoStProof SealedSectorPath string }
type PublicKey ¶
type PublicKey [PublicKeyBytes]byte
PublicKey is a compressed affine
func PrivateKeyPublicKey ¶
func PrivateKeyPublicKey(privateKey PrivateKey) PublicKey
PrivateKeyPublicKey gets the public key for a private key
type Signature ¶
type Signature [SignatureBytes]byte
Signature is a compressed affine
func Aggregate ¶
Aggregate aggregates signatures together into a new signature. If the provided signatures cannot be aggregated (due to invalid input or an an operational error), Aggregate will return nil.
func CreateZeroSignature ¶ added in v0.30.10
func CreateZeroSignature() Signature
CreateZeroSignature creates a zero signature, used as placeholder in filecoin.
func PrivateKeySign ¶
func PrivateKeySign(privateKey PrivateKey, message Message) *Signature
PrivateKeySign signs a message
type SortedPrivateSectorInfo ¶
type SortedPrivateSectorInfo struct {
// contains filtered or unexported fields
}
SortedPrivateSectorInfo is a slice of PrivateSectorInfo sorted (lexicographically, ascending) by sealed (replica) CID.
func NewSortedPrivateSectorInfo ¶
func NewSortedPrivateSectorInfo(sectorInfo ...PrivateSectorInfo) SortedPrivateSectorInfo
NewSortedPrivateSectorInfo returns a SortedPrivateSectorInfo
func (SortedPrivateSectorInfo) MarshalJSON ¶
func (s SortedPrivateSectorInfo) MarshalJSON() ([]byte, error)
MarshalJSON JSON-encodes and serializes the SortedPrivateSectorInfo.
func (*SortedPrivateSectorInfo) UnmarshalJSON ¶
func (s *SortedPrivateSectorInfo) UnmarshalJSON(b []byte) error
func (*SortedPrivateSectorInfo) Values ¶
func (s *SortedPrivateSectorInfo) Values() []PrivateSectorInfo
Values returns the sorted PrivateSectorInfo as a slice
type SortedPublicSectorInfo ¶
type SortedPublicSectorInfo struct {
// contains filtered or unexported fields
}
SortedPublicSectorInfo is a slice of publicSectorInfo sorted (lexicographically, ascending) by sealed (replica) CID.
func (SortedPublicSectorInfo) MarshalJSON ¶
func (s SortedPublicSectorInfo) MarshalJSON() ([]byte, error)
MarshalJSON JSON-encodes and serializes the SortedPublicSectorInfo.
func (*SortedPublicSectorInfo) UnmarshalJSON ¶
func (s *SortedPublicSectorInfo) UnmarshalJSON(b []byte) error
UnmarshalJSON parses the JSON-encoded byte slice and stores the result in the value pointed to by s.f. Note that this method allows for construction of a SortedPublicSectorInfo which violates its invariant (that its publicSectorInfo are sorted in some defined way). Callers should take care to never provide a byte slice which would violate this invariant.
func (*SortedPublicSectorInfo) Values ¶
func (s *SortedPublicSectorInfo) Values() []publicSectorInfo
Values returns the sorted publicSectorInfo as a slice
type TestHelper ¶
type TestHelper interface { AssertEqual(expected, actual interface{}, msgAndArgs ...interface{}) bool AssertNoError(err error, msgAndArgs ...interface{}) bool AssertTrue(value bool, msgAndArgs ...interface{}) bool RequireEqual(expected interface{}, actual interface{}, msgAndArgs ...interface{}) RequireNoError(err error, msgAndArgs ...interface{}) RequireTrue(value bool, msgAndArgs ...interface{}) }
type VanillaProof ¶ added in v0.30.10
type VanillaProof []byte