treerequest

README

Treerequest, a libp2p protocol implementation for requesting trees directly from other peers

A block request protocol for ipfs

Each peer can provide a set of persistent root nodes and their children.

Other peers can ask for the CIDs of the trees and also any of the blocks.

Access can be restricted to a set of peers.

Known tree CIDs are stored in the data store you provide.

For convenience, there are http services to access the trees given a prefix you provide that handle GET and (for your own peer) PUT requests with optional encryption for a provided URL prefix:

  unencrypted access: /PREFIX/PEERID/PATH
  encrypted access (only for your peer's files): /PREFIX/PATH

Before using the package, you must call InitTreeRequest().

IPFS-lite works well with TreeRequest and provides all of the required values for InitTreeRequest():

  ipfslite.BadgerDatastore(PATH) returns a ds.Datastore
  (*ipfslite.Peer).BlockStore() returns a blockstore.Blockstore
  *ipfslite.Peer implements ipld.DAGService
  (*ipfslite.Peer).Session(CTX) returns an ipld.NodeGetter

Documentation

Overview

Package treerequest block request protocol for ipfs

Each peer can provide a set of persistent root nodes and their children.

Other peers can ask for the CIDs of the trees and also any of the blocks.

Access can be restricted to a set of peers.

Known tree CIDs are stored in the data store you provide.

For convenience, there are http services to access the trees given a prefix you provide that handle GET and (for your own peer) PUT requests with optional encryption for a provided URL prefix:

unencrypted access: /PREFIX/PEERID/PATH
encrypted access (only for your peer's files): /PREFIX/PATH

Before using the package, you must call InitTreeRequest().

IPFS-lite works well with TreeRequest and provides all of the required values for InitTreeRequest():

ipfslite.BadgerDatastore(PATH) returns a ds.Datastore
(*ipfslite.Peer).BlockStore() returns a blockstore.Blockstore
*ipfslite.Peer implements ipld.DAGService
(*ipfslite.Peer).Session(CTX) returns an ipld.NodeGetter

Index

• func ChangePeers(treeName string, add []peer.ID, remove []peer.ID)
• func Checkpoint() error
• func Decrypt(input io.ReadSeeker) ([]byte, peer.ID, error)
• func Encrypt(content []byte, certs ...*x509.Certificate) ([]byte, error)
• func Fetch(peerID peer.ID, name string) (done chan RequestResult)
• func GetFile(name string, aCid cid.Cid, encrypted bool) (io.ReadSeeker, time.Time, error)
• func HandlePeerFileRequests(prefix string, encrypted bool, published func(path string, treeCid cid.Cid))
• func InitTreeRequest(protocolName string, dstor ds.Datastore, inputHost host.Host, ...) error
• func PeerFileRequestHandler(prefix string, encrypted bool, published func(path string, treeCid cid.Cid)) func(http.ResponseWriter, *http.Request)
• func PeerTreeFromURL(urlPath string) (peer.ID, string, string, error)
• func SetTree(name string, root cid.Cid) error
• func TimeForBlock(blockCid cid.Cid) (time.Time, error)
• type CidTree
  • func FetchSync(peerID peer.ID, name string) (*CidTree, error)
  • func GetTree(peerID peer.ID, name string) (*CidTree, error)
  • func NewCidTree(fileBlocks int) *CidTree
  • func (tree CidTree) AllCids() (allCids map[cid.Cid]bool)
  • func (tree CidTree) AllNodes() (allNodes map[cid.Cid]ipld.Node, err error)
  • func (tree CidTree) Root() cid.Cid
• type MsgBase
• type RequestResult

Functions

func ChangePeers

func ChangePeers(treeName string, add []peer.ID, remove []peer.ID)

ChangePeers add and/or remove peers

func Checkpoint

func Checkpoint() error

Checkpoint store the current trees

func Decrypt

func Decrypt(input io.ReadSeeker) ([]byte, peer.ID, error)

Decrypt decrypt a file (eventually switch to CMS standard: http://pike.lysator.liu.se/docs/ietf/rfc/60/rfc6032.xml) input: [keylen][key][iv][ciphertext] output: plaintext, signer, error

func Encrypt

func Encrypt(content []byte, certs ...*x509.Certificate) ([]byte, error)

Encrypt encrypts content with multiple public keys output: [keylen][key][iv][ciphertext]

func Fetch

func Fetch(peerID peer.ID, name string) (done chan RequestResult)

Fetch get the latest tree from a peer

func GetFile

func GetFile(name string, aCid cid.Cid, encrypted bool) (io.ReadSeeker, time.Time, error)

GetFile get a stream for a CID

func HandlePeerFileRequests

func HandlePeerFileRequests(prefix string, encrypted bool, published func(path string, treeCid cid.Cid))

HandlePeerFileRequests install a PeerFileRequestHandler in the standard http server

func InitTreeRequest

func InitTreeRequest(protocolName string, dstor ds.Datastore, inputHost host.Host, peerKey *rsa.PrivateKey, peerCert *x509.Certificate, bstor blockstore.Blockstore, getter ipld.NodeGetter, dagService ipld.DAGService, pin pinner.Pinner, cacheSize int, trees map[string]cid.Cid, ensureTrees map[string]bool, newConnection func(peer.ID)) error

InitTreeRequest ipfslite provides route (dht is a routing.ValueStore) and ds

func PeerFileRequestHandler

func PeerFileRequestHandler(prefix string, encrypted bool, published func(path string, treeCid cid.Cid)) func(http.ResponseWriter, *http.Request)

PeerFileRequestHandler returns a handler for tree requests

GET paths are of the form TREE/PEER-ID/PATH

PUT paths are of the form TREE/PATH and are written to the peer's tree

Encrypted GET requests decrypt the file using the peer's private key

Encrypted PUT requests encrypt the file using the peer's public key in addition to the peerids in the "PEERIDS" header.

Encrypted format uses a random 32-byte AES key encrypted by multiple public peer keys (including the encrypting peer):

(PLAINTEXT:)
[# keys: int32] -- do we really need to allow for more than 65535 peer keys?
[keylen: int16][key encrypted for peer 1]
[keylen: int16][key encrypted for peer 2]
...
[initialization vector: AES BLOCK SIZE]
(CIPHERTEXT:)
[keylen: int16][peer id 1]
[keylen: int16][peer id 2]
...
[content]

func PeerTreeFromURL

func PeerTreeFromURL(urlPath string) (peer.ID, string, string, error)

PeerTreeFromURL get a peerID, treeName, and the remainder from a URL

func SetTree

func SetTree(name string, root cid.Cid) error

SetTree set my tree to root, root must name a unixfs node

func TimeForBlock

func TimeForBlock(blockCid cid.Cid) (time.Time, error)

TimeForBlock get the time a block was stored

Types

type CidTree

type CidTree struct {
	Nodes     map[string]cid.Cid
	FileNodes []cid.Cid
}

CidTree a CID tree for a peer

func FetchSync

func FetchSync(peerID peer.ID, name string) (*CidTree, error)

FetchSync synchronously get the latest tree from a peer

func GetTree

func GetTree(peerID peer.ID, name string) (*CidTree, error)

GetTree gets a tree from storage or returns an error if it's not there

func NewCidTree

func NewCidTree(fileBlocks int) *CidTree

NewCidTree make a new, initialized cidtree

func (CidTree) AllCids

func (tree CidTree) AllCids() (allCids map[cid.Cid]bool)

AllCids get a map of Cid->true for all Cids in the tree

func (CidTree) AllNodes

func (tree CidTree) AllNodes() (allNodes map[cid.Cid]ipld.Node, err error)

AllNodes get a map of Node->true for all Nodes in the tree

func (CidTree) Root

func (tree CidTree) Root() cid.Cid

Root get the CidTree root

type MsgBase

type MsgBase struct {
	ID uint64
}

type RequestResult

type RequestResult struct {
	Tree *CidTree
	Err  error
}

RequestResult result for an asynchronous request