go-metadata

README

Welcome to @dataswap/go-metadata 👋

🏠 Homepage

go-metadata

Implement mapping information collection between source data and target car files during the Dag construction process

Features

go-metadata is a publicly available library that includes functions for source data sampling, CAR generation, dataset proofs, dataset proof challenges, and validation tools.

Development

dependencies

go

To build go-metadata, you need a working installation of Go 1.20.1 or higher:

wget -c https://golang.org/dl/go1.20.1.linux-amd64.tar.gz -O - | sudo tar -xz -C /usr/local

TIP: You'll need to add /usr/local/go/bin to your path. For most Linux distributions you can run something like:

echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.bashrc && source ~/.bashrc

See the official Golang installation instructions if you get stuck.

Clone the repository

git clone https://github.com/dataswap/go-metadata.git
cd go-metadata/

Build

go build -v ./cmd

Test

go test -v ./service

Installation

To be added

Usage

• Data set original file scanning, car file generation, Mapping File Generation
• DatasetProof
  • The DP needs to submit the DatasetProof to the Dataswap contract
  • DA compute Merkle-Tree for challenge proof
• DatasetVerification
  • DA uses data proof verification tools to generate dataset challenge proof verification information.
• Other tools
  • compute commp CID(PieceCID)
  • dump commp info

$ meta 
NAME:
   meta - Utility for working with car files

USAGE:
   meta [global options] command [command options] [arguments...]

COMMANDS:
   create       Create a car file
   list, l, ls  List the CIDs in a car
   proof        compute proof of merkle-tree
   verify       verify challenge proofs of merkle-tree
   tools        
   help, h      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h  show help

Create car file

$ meta create -h
NAME:
   meta create - Create a car file

USAGE:
   meta create command [command options] [arguments...]

COMMANDS:
   car      Create a car file
   chunks   Create car chunks
   help, h  Shows a list of commands or help for one command

OPTIONS:
   --help, -h  show help

DatasetProof

• The DA challenges specific DatasetLeafHashes (CarRootHashes) and CarLeafHashes through random challenges.
• The DP needs to submit the DatasetProof to the business contract, where the DatasetMerkleTree is stored on-chain, and the CarProofs are stored on the Filecoin network (to save on-chain resources).

$ meta proof -h
NAME:
   meta proof - compute proof of merkle-tree

USAGE:
   meta proof command [command options] [arguments...]

COMMANDS:
   chanllenge-proof  compute proof of merkle-tree
   dataset-proof     compute dataset proof of commPs
   help, h           Shows a list of commands or help for one command

OPTIONS:
   --help, -h  show help

DatasetVerification

• The DA submits the challenged DatasetHash Merkle Proof and CarRootHash Merkle Proof to the blockchain as challenge proof information for verification.

$ meta verify -h
NAME:
   meta verify - verify challenge proofs of merkle-tree

USAGE:
   meta verify [command options] <randomness> <cachePath>

OPTIONS:
   --help, -h  show help

Author

👤 dataswap

• GitHub: @dataswap

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page. You can also take a look at the contributing guide.

Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2023 dataswap.
This project is MIT and APACHE licensed.