README
¶
BOXO: IPFS SDK for GO
A set of libraries for building IPFS applications and implementations in GO.
About
Boxo is a component library for building IPFS applications and implementations in Go.
Some scenarios in which you may find Boxo helpful:
- You are building an application that interacts with the IPFS network
- You are building an IPFS implementation
- You want to reuse some components of IPFS such as its Kademlia DHT, Bitswap, data encoding, etc.
- You want to experiment with IPFS
Boxo powers Kubo, which is the most popular IPFS implementation, so its code has been battle-tested on the IPFS network for years, and is well-understood by the community.
Motivation
TL;DR The goal of this repo is to help people build things. Previously users struggled to find existing useful code or to figure out how to use what they did find. We observed many running Kubo and using its HTTP RPC API. This repo aims to do better. We're taking the libraries that many were already effectively relying on in production and making them more easily discoverable and usable.
The maintainers primarily aim to help people trying to build with IPFS in Go that were previously either giving up or relying on the Kubo HTTP RPC API. Some of these people will end up being better served by IPFS tooling in other languages (e.g., Javascript, Rust, Java, Python), but for those who are either looking to write in Go or to leverage the set of IPFS tooling we already have in Go we’d like to make their lives easier.
We’d also like to make life easier on ourselves as the maintainers by reducing the maintenance burden that comes from being the owners on many repos and then use that time to contribute more to the community in the form of easier to use libraries, better implementations, improved protocols, new protocols, etc.
Boxo is not exhaustive nor comprehensive--there are plenty of useful IPFS protocols, specs, libraries, etc. that are not in Boxo. The goal of Boxo is to provide cohesive and well-maintained components for common IPFS use cases.
More details can also be found in the Rationale FAQ.
Scope
What kind of components does Boxo have?
Boxo includes high-quality components useful for interacting with IPFS protocols, public and private IPFS networks, and content-addressed data, such as:
- Content routing (DHT, delegated content routing, providing)
- Data transfer (gateways, Bitswap, incremental verification)
- Naming and mutability (name resolution, IPNS)
- Interacting with public and private IPFS networks
- Working with content-addressed data
Boxo aims to provide a cohesive interface into these components. Note that not all of the underlying components necessarily reside in this respository.
Does Boxo == IPFS?
No. This repo houses some IPFS functionality written in Go that has been useful in practice, and is maintained by a group that has long term commitments to the IPFS project
Is everything related to IPFS in the Go ecosystem in this repo?
No. Not everything related to IPFS is intended to be in Boxo. View it as a starter toolbox (potentially among multiple). If you’d like to build an IPFS implementation with Go, here are some tools you might want that are maintained by a group that has long term commitments to the IPFS project. There are certainly repos that others maintain that aren't included here (e.g., ipfs/go-car) which are still useful to IPFS implementations. It's expected and fine for new IPFS functionality to be developed that won't be part of Boxo.
Consuming
Getting started
See examples.
If you are migrating to Boxo, see Migrating to Boxo.
Migrating to Boxo
Many Go modules under github.com/ipfs have moved here. Boxo provides a tool to ease this migration, which does most of the work for you:
cdinto the root directory of your module (where thego.modfile is)- Run:
go run github.com/gocnpan/boxo/cmd/boxo-migrate@latest update-imports- This will upgrade your module to Boxo v0.8.0 and rewrite your import paths
- Run:
go run github.com/gocnpan/boxo/cmd/boxo-migrate@latest check-dependencies- This will print unmaintained dependencies you still have
- These aren't necessarily an immediate problem, but you should eventually get them out of your dependency graph
This tool only upgrades your module to Boxo v0.8.0, to minimize backwards-incompatible changes. Depending on the versions of IPFS modules before the upgrade, your code may require additional changes to build.
We recommend upgrading to v0.8.0 first, and then upgrading to the latest Boxo release.
If you encounter any challenges, please open an issue and Boxo maintainers will help you.
Deprecations & Breaking Changes
See RELEASE.md.
Development
Should I add my IPFS component to Boxo?
We happily accept external contributions! However, Boxo maintains a high quality bar, so code accepted into Boxo must meet some minimum maintenance criteria:
- Actively maintained
- Must be actively used by, or will be included in software that is actively used by, a significant number of users or production systems. Code that is not actively used cannot be properly maintained.
- Must have multiple engineers who are willing and able to maintain the relevant code in Boxo for a long period of time.
- If either of these changes, Boxo maintainers will consider removing the component from Boxo.
- Adequately tested
- At least with unit tests
- Ideally also including integration tests with other components
- Adequately documented
- Godocs at minimum
- Complex components should have their own doc.go or README.md describing the component, its use cases, tradeoffs, design rationale, etc.
- If the maintainers are not Boxo maintainers, then the component must include a CODEOWNERS file with at least two code owners who can commit to reviewing PRs
If you have some experimental component that you think would benefit the IPFS community, we suggest you build the component in your own repository until it's clear that there's community demand for it, and then open an issue/PR in this repository to discuss including it in Boxo.
Release Process
See RELEASE.md.
Why is the code coverage so bad?
The code coverage of this repo is not currently representative of the actual test coverage of this code. Much of the code in this repo is currently covered by integration tests in Kubo. We are in the process of moving those tests here, and as that continues the code coverage will significantly increase.
General
Help
If you have questions, feel free to open an issue. You can also find the Boxo maintainers in Filecoin Slack at #Boxo-maintainers. (If you would like to engage via IPFS Discord or ipfs.io Matrix, please drop into the #ipfs-implementers channel/room or file an issue, and we'll get bridging from #Boxo-maintainers to these other chat platforms.)
What is the response time for issues or PRs filed?
New issues and PRs to this repo are usually looked at on a weekly basis as part of Kubo triage. However, the response time may vary.
What are some projects that depend on this project?
The exhaustive list is https://github.com/gocnpan/boxo/network/dependents. Some notable projects include:
- Kubo, an IPFS implementation in Go
- Lotus, a Filecoin implementation in Go
- rainbow, a specialized IPFS gateway
- ipfs-check, checks IPFS data availability
- someguy, a dedicated Delegated Routing V1 server and client
- Bifrost Gateway, a dedicated IPFS Gateway daemon backed by a remote datastore
Governance and Access
See CODEOWNERS for the current maintainers list. Governance for graduating additional maintainers hasn't been established. Repo permissions are all managed through ipfs/github-mgmt.
Why is this named "Boxo"?
See https://github.com/gocnpan/boxo/issues/215.
Additional Docs & FAQs
See the wiki.
License
Directories
¶
| Path | Synopsis |
|---|---|
|
client
Package bitswap implements the IPFS exchange interface with the BitSwap bilateral exchange protocol.
|
Package bitswap implements the IPFS exchange interface with the BitSwap bilateral exchange protocol. |
|
client/wantlist
Package wantlist implements an object for bitswap that contains the keys that a given peer wants.
|
Package wantlist implements an object for bitswap that contains the keys that a given peer wants. |
|
server/internal/decision
Package decision implements the decision engine for the bitswap service.
|
Package decision implements the decision engine for the bitswap service. |
|
Package blockservice implements a BlockService interface that provides a single GetBlock/AddBlock interface that seamlessly retrieves data either locally or from a remote peer through the exchange.
|
Package blockservice implements a BlockService interface that provides a single GetBlock/AddBlock interface that seamlessly retrieves data either locally or from a remote peer through the exchange. |
|
Package blockstore implements a thin wrapper over a datastore, giving a clean interface for Getting and Putting block objects.
|
Package blockstore implements a thin wrapper over a datastore, giving a clean interface for Getting and Putting block objects. |
|
Package chunk implements streaming block splitters.
|
Package chunk implements streaming block splitters. |
|
gen
This file generates bytehash LUT
|
This file generates bytehash LUT |
|
datastore
|
|
|
dshelp
Package dshelp provides utilities for parsing and creating datastore keys used by go-ipfs
|
Package dshelp provides utilities for parsing and creating datastore keys used by go-ipfs |
|
Package exchange defines the IPFS exchange interface
|
Package exchange defines the IPFS exchange interface |
|
offline
package offline implements an object that implements the exchange interface but returns nil values to every request.
|
package offline implements an object that implements the exchange interface but returns nil values to every request. |
|
Package filestore implements a Blockstore which is able to read certain blocks of data directly from its original location in the filesystem.
|
Package filestore implements a Blockstore which is able to read certain blocks of data directly from its original location in the filesystem. |
|
posinfo
Package posinfo wraps offset information used by ipfs filestore nodes
|
Package posinfo wraps offset information used by ipfs filestore nodes |
|
internal
|
|
|
ipld
|
|
|
merkledag
Package merkledag implements the IPFS Merkle DAG data structures.
|
Package merkledag implements the IPFS Merkle DAG data structures. |
|
merkledag/traverse
Package traverse provides merkledag traversal functions
|
Package traverse provides merkledag traversal functions |
|
unixfs
Package unixfs implements a data format for files in the IPFS filesystem It is not the only format in ipfs, but it is the one that the filesystem assumes
|
Package unixfs implements a data format for files in the IPFS filesystem It is not the only format in ipfs, but it is the one that the filesystem assumes |
|
unixfs/hamt
Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes.
|
Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes. |
|
unixfs/importer
Package importer implements utilities used to create IPFS DAGs from files and readers.
|
Package importer implements utilities used to create IPFS DAGs from files and readers. |
|
unixfs/importer/balanced
Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root.
|
Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root. |
|
unixfs/importer/trickle
Package trickle allows to build trickle DAGs.
|
Package trickle allows to build trickle DAGs. |
|
unixfs/io
Package io implements convenience objects for working with the ipfs unixfs data format.
|
Package io implements convenience objects for working with the ipfs unixfs data format. |
|
unixfs/mod
Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them.
|
Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them. |
|
Package namesys defines Resolver and Publisher interfaces for IPNS paths, that is, IPFS paths in the form of /ipns/<name_to_be_resolved>.
|
Package namesys defines Resolver and Publisher interfaces for IPNS paths, that is, IPFS paths in the form of /ipns/<name_to_be_resolved>. |
|
republisher
Package republisher provides a utility to automatically re-publish IPNS records related to the keys in a Keystore.
|
Package republisher provides a utility to automatically re-publish IPNS records related to the keys in a Keystore. |
|
Package path contains utilities to work with ipfs paths.
|
Package path contains utilities to work with ipfs paths. |
|
resolver
Package resolver implements utilities for resolving paths within ipfs.
|
Package resolver implements utilities for resolving paths within ipfs. |
|
pinning
|
|
|
pinner
Package pin implements structures and methods to keep track of which objects a user wants to keep stored locally.
|
Package pin implements structures and methods to keep track of which objects a user wants to keep stored locally. |
|
pinner/dsindex
Package dsindex provides secondary indexing functionality for a datastore.
|
Package dsindex provides secondary indexing functionality for a datastore. |
|
pinner/dspinner
Package dspinner implements structures and methods to keep track of which objects a user wants to keep stored locally.
|
Package dspinner implements structures and methods to keep track of which objects a user wants to keep stored locally. |
|
routing
|
|
|
mock
Package mockrouting provides a virtual routing server.
|
Package mockrouting provides a virtual routing server. |
|
none
Package nilrouting implements a routing client that does nothing.
|
Package nilrouting implements a routing client that does nothing. |
|
offline
Package offline implements Routing with a client which is only able to perform offline operations.
|
Package offline implements Routing with a client which is only able to perform offline operations. |
|
Package util implements various utility functions used within ipfs that do not currently have a better place to live.
|
Package util implements various utility functions used within ipfs that do not currently have a better place to live. |