Documentation ¶
Overview ¶
Package client provides a Go Client for the IPFS Cluster API provided by the "api/rest" component. It supports both the HTTP(s) endpoint and the libp2p-http endpoint.
Package api holds declarations for types used in ipfs-cluster APIs to make them re-usable across differen tools. This include RPC API "Serial[izable]" versions for types. The Go API uses natives types, while RPC API, REST APIs etc use serializable types (i.e. json format). Conversion methods exists between types.
Note that all conversion methods ignore any parsing errors. All values must be validated first before initializing any of the types defined here.
Index ¶
- Constants
- Variables
- func IsPeerAddress(addr multiaddr.Multiaddr) bool
- func PeersToStrings(peers []peer.ID) []string
- func StringsToPeers(strs []string) []peer.ID
- type AddParams
- type AddedOutput
- type Alert
- type Client
- type Config
- type ConnectGraph
- type Error
- type Failover
- type GlobalPinInfo
- type ID
- type IPFSID
- type IPFSPinStatus
- type IPFSRepoStat
- type LBStrategy
- type Metric
- type Multiaddr
- type NodeWithMeta
- type Pin
- type PinInfo
- type PinOptions
- type PinPath
- type PinType
- type RoundRobin
- type StatusFilterParams
- type TrackerStatus
- type Version
Constants ¶
const ( TrackerStatusError = TrackerStatusClusterError | TrackerStatusPinError | TrackerStatusUnpinError TrackerStatusQueued = TrackerStatusPinQueued | TrackerStatusUnpinQueued )
Composite TrackerStatus.
const AllType = DataType | MetaType | ClusterDAGType | ShardType
AllType is a PinType used for filtering all pin types
Variables ¶
var ( DefaultTimeout = 0 DefaultAPIAddr = "/ip4/127.0.0.1/tcp/9094" DefaultLogLevel = "info" DefaultProxyPort = 9095 ResolveTimeout = 30 * time.Second DefaultPort = 9094 )
Configuration defaults
var DefaultShardSize = uint64(100 * 1024 * 1024) // 100 MB
DefaultShardSize is the shard size for params objects created with DefaultParams().
Functions ¶
func IsPeerAddress ¶
func IsPeerAddress(addr multiaddr.Multiaddr) bool
IsPeerAddress detects if the given multiaddress identifies a libp2p peer, either because it has the /p2p/ protocol or because it uses /dnsaddr/
func PeersToStrings ¶
PeersToStrings IDB58Encodes a list of peers.
func StringsToPeers ¶
StringsToPeers decodes peer.IDs from strings.
Types ¶
type AddParams ¶
type AddParams struct { PinOptions Local bool Recursive bool Layout string Chunker string RawLeaves bool Hidden bool Wrap bool Shard bool Progress bool CidVersion int HashFun string StreamChannels bool NoCopy bool }
AddParams contains all of the configurable parameters needed to specify the importing process of a file being added to an ipfs-cluster
func AddParamsFromQuery ¶
AddParamsFromQuery parses the AddParams object from a URL.Query().
func DefaultAddParams ¶
func DefaultAddParams() *AddParams
DefaultAddParams returns a AddParams object with standard defaults
func (*AddParams) ToQueryString ¶
ToQueryString returns a url query string (key=value&key2=value2&...)
type AddedOutput ¶
type AddedOutput struct { Name string `json:"name" codec:"n,omitempty"` Cid cid.Cid `json:"cid" codec:"c"` Bytes uint64 `json:"bytes,omitempty" codec:"b,omitempty"` Size uint64 `json:"size,omitempty" codec:"s,omitempty"` }
AddedOutput carries information for displaying the standard ipfs output indicating a node of a file has been added.
type Client ¶
type Client interface { // ID returns information about the cluster Peer. ID(context.Context) (*ID, error) // Peers requests ID information for all cluster peers. Peers(context.Context) ([]*ID, error) // PeerJoin add a peer join to the cluster. PeerJoin(ctx context.Context, addr string) (*ID, error) // PeerAdd adds a new peer to the cluster. PeerAdd(ctx context.Context, pid peer.ID) (*ID, error) // PeerRm removes a current peer from the cluster PeerRm(ctx context.Context, pid peer.ID) error // Add imports files to the cluster from the given paths. Add(ctx context.Context, paths []string, params *AddParams, out chan<- *AddedOutput) error // AddMultiFile imports new files from a MultiFileReader. AddMultiFile(ctx context.Context, multiFileR *files.MultiFileReader, params *AddParams, out chan<- *AddedOutput) error // Pin tracks a Cid with the given replication factor and a name for // human-friendliness. Pin(ctx context.Context, ci cid.Cid, opts PinOptions) (*Pin, error) // Unpin untracks a Cid from cluster. Unpin(ctx context.Context, ci cid.Cid) (*Pin, error) // PinPath resolves given path into a cid and performs the pin operation. PinPath(ctx context.Context, path string, opts PinOptions) (*Pin, error) // UnpinPath resolves given path into a cid and performs the unpin operation. // It returns Pin of the given cid before it is unpinned. UnpinPath(ctx context.Context, path string) (*Pin, error) // Allocations returns the consensus state listing all tracked items // and the peers that should be pinning them. Allocations(ctx context.Context, filter PinType) ([]*Pin, error) // Allocation returns the current allocations for a given Cid. Allocation(ctx context.Context, ci cid.Cid) (*Pin, error) // Status returns the current ipfs state for a given Cid. If local is true, // the information affects only the current peer, otherwise the information // is fetched from all cluster peers. Status(ctx context.Context, ci cid.Cid, local bool) (*GlobalPinInfo, error) // StatusAll gathers Status() for all tracked items. StatusAll(ctx context.Context, filter TrackerStatus, local bool) ([]*GlobalPinInfo, error) // Sync makes sure the state of a Cid corresponds to the state reported // by the ipfs daemon, and returns it. If local is true, this operation // only happens on the current peer, otherwise it happens on every // cluster peer. Sync(ctx context.Context, ci cid.Cid, local bool) (*GlobalPinInfo, error) // SyncAll triggers Sync() operations for all tracked items. It only // returns informations for items that were de-synced or have an error // state. If local is true, the operation is limited to the current // peer. Otherwise it happens on every cluster peer. SyncAll(ctx context.Context, local bool) ([]*GlobalPinInfo, error) // Recover retriggers pin or unpin ipfs operations for a Cid in error // state. If local is true, the operation is limited to the current // peer, otherwise it happens on every cluster peer. Recover(ctx context.Context, ci cid.Cid, local bool) (*GlobalPinInfo, error) // RecoverAll triggers Recover() operations on all tracked items. If // local is true, the operation is limited to the current peer. // Otherwise, it happens everywhere. RecoverAll(ctx context.Context, local bool) ([]*GlobalPinInfo, error) // Version returns the ipfs-cluster peer's version. Version(context.Context) (*Version, error) // IPFS returns an instance of go-ipfs-api's Shell, pointing to a // Cluster's IPFS proxy endpoint. IPFS(context.Context) *httpapi.HttpApi // GetConnectGraph returns an ipfs-cluster connection graph. GetConnectGraph(context.Context) (*ConnectGraph, error) // Metrics returns a map with the latest metrics of matching name // for the current cluster peers. Metrics(ctx context.Context, name string) ([]*Metric, error) }
Client interface defines the interface to be used by API clients to interact with the ipfs-cluster-service. All methods take a context.Context as their first parameter, this allows for timing out and cancelling of requests as well as recording metrics and tracing of requests through the API.
func DefaultCluster ¶
DefaultCluster initializes a client given a Config.
func NewDefaultClient ¶
NewDefaultClient initializes a client given a Config.
func NewLBClient ¶
func NewLBClient(strategy LBStrategy, cfgs []*Config, retries int) (Client, error)
NewLBClient returns a new client that would load balance requests among clients.
type Config ¶
type Config struct { // Enable SSL support. Only valid without APIAddr. SSL bool // Skip certificate verification (insecure) NoVerifyCert bool // Username and password for basic authentication Username string Password string // The ipfs-cluster REST API endpoint in multiaddress form // (takes precedence over host:port). It this address contains // an /ipfs/, /p2p/ or /dnsaddr, the API will be contacted // through a libp2p tunnel, thus getting encryption for // free. Using the libp2p tunnel will ignore any configurations. APIAddr multiaddr.Multiaddr // REST API endpoint host and port. Only valid without // APIAddr. Host string Port string // If APIAddr is provided, and the peer uses private networks (pnet), // then we need to provide the key. If the peer is the cluster peer, // this corresponds to the cluster secret. ProtectorKey []byte // ProxyAddr is used to obtain a go-ipfs-api Shell instance pointing // to the ipfs proxy endpoint of ipfs-cluster. If empty, the location // will be guessed from one of APIAddr/Host, // and the port used will be ipfs-cluster's proxy default port (9095) ProxyAddr multiaddr.Multiaddr // Define timeout for network operations Timeout time.Duration // Specifies if we attempt to re-use connections to the same // hosts. DisableKeepAlives bool // LogLevel defines the verbosity of the logging facility LogLevel string }
Config allows to configure the parameters to connect to the ipfs-cluster REST API.
type ConnectGraph ¶
type ConnectGraph struct { ClusterID peer.ID // ipfs to ipfs links IPFSLinks map[string][]peer.ID `json:"ipfs_links" codec:"il,omitempty"` // cluster to cluster links ClusterLinks map[string][]peer.ID `json:"cluster_links" codec:"cl,omitempty"` // cluster to ipfs links ClustertoIPFS map[string]peer.ID `json:"cluster_to_ipfs" codec:"ci,omitempty"` }
ConnectGraph holds information about the connectivity of the cluster To read, traverse the keys of ClusterLinks. Each such id is one of the peers of the "ClusterID" peer running the query. ClusterLinks[id] in turn lists the ids that peer "id" sees itself connected to. It is possible that id is a peer of ClusterID, but ClusterID can not reach id over rpc, in which case ClusterLinks[id] == [], as id's view of its connectivity can not be retrieved.
Iff there was an error reading the IPFSID of the peer then id will not be a key of ClustertoIPFS or IPFSLinks. Finally iff id is a key of ClustertoIPFS then id will be a key of IPFSLinks. In the event of a SwarmPeers error IPFSLinks[id] == [].
type Error ¶
type Error struct { Code int `json:"code" codec:"o,omitempty"` Message string `json:"message" codec:"m,omitempty"` }
Error can be used by APIs to return errors.
type Failover ¶
type Failover struct {
// contains filtered or unexported fields
}
Failover is a load balancing strategy that would try the local cluster peer first. If the local call fail it would try other client in a round robin fashion.
func (*Failover) SetClients ¶
SetClients sets a list of clients for this strategy.
type GlobalPinInfo ¶
type GlobalPinInfo struct { Cid cid.Cid `json:"cid" codec:"c"` // https://github.com/golang/go/issues/28827 // Peer IDs are of string Kind(). We can't use peer IDs here // as Go ignores TextMarshaler. PeerMap map[string]*PinInfo `json:"peer_map" codec:"pm,omitempty"` }
GlobalPinInfo contains cluster-wide status information about a tracked Cid, indexed by cluster peer.
func WaitFor ¶
func WaitFor(ctx context.Context, c Client, fp StatusFilterParams) (*GlobalPinInfo, error)
WaitFor is a utility function that allows for a caller to wait for a paticular status for a CID (as defined by StatusFilterParams). It returns the final status for that CID and an error, if there was.
WaitFor works by calling Status() repeatedly and checking that all peers have transitioned to the target TrackerStatus or are Remote. If an error of some type happens, WaitFor returns immediately with an empty GlobalPinInfo.
func (*GlobalPinInfo) String ¶
func (gpi *GlobalPinInfo) String() string
String returns the string representation of a GlobalPinInfo.
type ID ¶
type ID struct { ID peer.ID `json:"id" codec:"i,omitempty"` Addresses []Multiaddr `json:"addresses" codec:"a,omitempty"` ClusterPeers []peer.ID `json:"cluster_peers" codec:"cp,omitempty"` ClusterPeersAddresses []Multiaddr `json:"cluster_peers_addresses" codec:"cpa,omitempty"` Version string `json:"version" codec:"v,omitempty"` Commit string `json:"commit" codec:"c,omitempty"` RPCProtocolVersion protocol.ID `json:"rpc_protocol_version" codec:"rv,omitempty"` Error string `json:"error" codec:"e,omitempty"` IPFS *IPFSID `json:"ipfs,omitempty" codec:"ip,omitempty"` Peername string `json:"peername" codec:"pn,omitempty"` }
ID holds information about the Cluster peer
type IPFSID ¶
type IPFSID struct { ID peer.ID `json:"id,omitempty" codec:"i,omitempty"` Addresses []Multiaddr `json:"addresses" codec:"a,omitempty"` AgentVersion string `json:"AgentVersion"` Error string `json:"error" codec:"e,omitempty"` }
IPFSID is used to store information about the underlying IPFS daemon
type IPFSPinStatus ¶
type IPFSPinStatus int
IPFSPinStatus represents the status of a pin in IPFS (direct, recursive etc.)
const ( IPFSPinStatusBug IPFSPinStatus = iota IPFSPinStatusError IPFSPinStatusDirect IPFSPinStatusRecursive IPFSPinStatusIndirect IPFSPinStatusUnpinned )
IPFSPinStatus values FIXME include maxdepth
func IPFSPinStatusFromString ¶
func IPFSPinStatusFromString(t string) IPFSPinStatus
IPFSPinStatusFromString parses a string and returns the matching IPFSPinStatus.
func (IPFSPinStatus) IsPinned ¶
func (ips IPFSPinStatus) IsPinned(maxDepth int) bool
IsPinned returns true if the item is pinned as expected by the maxDepth parameter.
func (IPFSPinStatus) ToTrackerStatus ¶
func (ips IPFSPinStatus) ToTrackerStatus() TrackerStatus
ToTrackerStatus converts the IPFSPinStatus value to the appropriate TrackerStatus value.
type IPFSRepoStat ¶
type IPFSRepoStat struct { RepoSize uint64 `codec:"r,omitempty"` StorageMax uint64 `codec:"s, omitempty"` }
IPFSRepoStat wraps information about the IPFS repository.
type LBStrategy ¶
LBStrategy is a strategy to load balance requests among clients.
type Metric ¶
type Metric struct { Name string `json:"name" codec:"n,omitempty"` Peer peer.ID `json:"peer" codec:"p,omitempty"` Value string `json:"value" codec:"v,omitempty"` Expire int64 `json:"expire" codec:"e,omitempty"` Valid bool `json:"valid" codec:"d,omitempty"` ReceivedAt int64 `json:"received_at" codec:"t,omitempty"` // ReceivedAt contains a UnixNano timestamp }
Metric transports information about a peer.ID. It is used to decide pin allocations by a PinAllocator. IPFS cluster is agnostic to the Value, which should be interpreted by the PinAllocator. The ReceivedAt value is a timestamp representing when a peer has received the metric value.
type Multiaddr ¶
type Multiaddr struct {
multiaddr.Multiaddr
}
Multiaddr is a concrete type to wrap a Multiaddress so that it knows how to serialize and deserialize itself.
func NewMultiaddr ¶
NewMultiaddr returns a cluster Multiaddr wrapper creating the multiaddr.Multiaddr with the given string.
func NewMultiaddrWithValue ¶
func NewMultiaddrWithValue(ma multiaddr.Multiaddr) Multiaddr
NewMultiaddrWithValue returns a new cluster Multiaddr wrapper using the given multiaddr.Multiaddr.
func (Multiaddr) MarshalBinary ¶
MarshalBinary returs the bytes of the wrapped multiaddress.
func (Multiaddr) MarshalJSON ¶
MarshalJSON returns a JSON-formatted multiaddress.
func (*Multiaddr) UnmarshalBinary ¶
UnmarshalBinary casts some bytes as a multiaddress wraps it with the given cluster Multiaddr.
func (*Multiaddr) UnmarshalJSON ¶
UnmarshalJSON parses a cluster Multiaddr from the JSON representation.
type NodeWithMeta ¶
type NodeWithMeta struct { Data []byte `codec:"d,omitempty"` Cid cid.Cid `codec:"c,omitempty"` CumSize uint64 `codec:"s,omitempty"` // Cumulative size Format string `codec:"f,omitempty"` }
NodeWithMeta specifies a block of data and a set of optional metadata fields carrying information about the encoded ipld node
func (*NodeWithMeta) Size ¶
func (n *NodeWithMeta) Size() uint64
Size returns how big is the block. It is different from CumSize, which records the size of the underlying tree.
type Pin ¶
type Pin struct { PinOptions Cid cid.Cid `json:"cid" codec:"c"` // See PinType comments Type PinType `json:"type" codec:"t,omitempty"` // The peers to which this pin is allocated Allocations []peer.ID `json:"allocations" codec:"a,omitempty"` // MaxDepth associated to this pin. -1 means // recursive. MaxDepth int `json:"max_depth" codec:"d,omitempty"` // We carry a reference CID to this pin. For // ClusterDAGs, it is the MetaPin CID. For the // MetaPin it is the ClusterDAG CID. For Shards, // it is the previous shard CID. // When not needed the pointer is nil Reference *cid.Cid `json:"reference" codec:"r,omitempty"` }
Pin carries all the information associated to a CID that is pinned in IPFS Cluster. It also carries transient information (that may not get protobuffed, like UserAllocations).
func PinCid ¶
func PinCid(c cid.Cid) *Pin
PinCid is a shortcut to create a Pin only with a Cid. Default is for pin to be recursive and the pin to be of DataType.
func PinWithOpts ¶
func PinWithOpts(c cid.Cid, opts PinOptions) *Pin
PinWithOpts creates a new Pin calling PinCid(c) and then sets its PinOptions fields with the given options.
func (*Pin) Equals ¶
Equals checks if two pins are the same (with the same allocations). If allocations are the same but in different order, they are still considered equivalent. pin or pin2 may be nil. If both are nil, Equals returns false.
func (*Pin) IsRemotePin ¶
IsRemotePin determines whether a Pin's ReplicationFactor has been met, so as to either pin or unpin it from the peer.
func (*Pin) ProtoMarshal ¶
ProtoMarshal marshals this Pin using probobuf.
func (*Pin) ProtoUnmarshal ¶
ProtoUnmarshal unmarshals this fields from protobuf-encoded bytes.
type PinInfo ¶
type PinInfo struct { Cid cid.Cid `json:"cid" codec:"c"` Peer peer.ID `json:"peer" codec:"p,omitempty"` PeerName string `json:"peername" codec:"pn,omitempty"` Status TrackerStatus `json:"status" codec:"st,omitempty"` TS time.Time `json:"timestamp" codec:"ts,omitempty"` Error string `json:"error" codec:"e,omitempty"` }
PinInfo holds information about local pins.
type PinOptions ¶
type PinOptions struct { ReplicationFactorMin int `json:"replication_factor_min" codec:"rn,omitempty"` ReplicationFactorMax int `json:"replication_factor_max" codec:"rx,omitempty"` Name string `json:"name" codec:"n,omitempty"` ShardSize uint64 `json:"shard_size" codec:"s,omitempty"` UserAllocations []peer.ID `json:"user_allocations" codec:"ua,omitempty"` Metadata map[string]string `json:"metadata" codec:"m,omitempty"` PinUpdate cid.Cid `json:"pin_update,omitempty" codec:"pu,omitempty"` }
PinOptions wraps user-defined options for Pins
func (*PinOptions) Equals ¶
func (po *PinOptions) Equals(po2 *PinOptions) bool
Equals returns true if two PinOption objects are equivalent. po and po2 may be nil.
func (*PinOptions) FromQuery ¶
func (po *PinOptions) FromQuery(q url.Values) error
FromQuery is the inverse of ToQuery().
func (*PinOptions) ToQuery ¶
func (po *PinOptions) ToQuery() string
ToQuery returns the PinOption as query arguments.
type PinPath ¶
type PinPath struct { PinOptions Path string `json:"path"` }
PinPath is a wrapper for holding pin options and path of the content.
type PinType ¶
type PinType uint64
PinType specifies which sort of Pin object we are dealing with. In practice, the PinType decides how a Pin object is treated by the PinTracker. See descriptions above. A sharded Pin would look like:
[ Meta ] (not pinned on IPFS, only present in cluster state)
| v
[ Cluster DAG ] (pinned everywhere in "direct")
| .. | v v
[Shard1] .. [ShardN] (allocated to peers and pinned with max-depth=1 | | .. | | | .. | v v .. v v v .. v [][]..[] [][]..[] Blocks (indirectly pinned on ipfs, not tracked in cluster)
const ( // BadType type showing up anywhere indicates a bug BadType PinType = 1 << iota // DataType is a regular, non-sharded pin. It is pinned recursively. // It has no associated reference. DataType // MetaType tracks the original CID of a sharded DAG. Its Reference // points to the Cluster DAG CID. MetaType // ClusterDAGType pins carry the CID of the root node that points to // all the shard-root-nodes of the shards in which a DAG has been // divided. Its Reference carries the MetaType CID. // ClusterDAGType pins are pinned directly everywhere. ClusterDAGType // ShardType pins carry the root CID of a shard, which points // to individual blocks on the original DAG that the user is adding, // which has been sharded. // They carry a Reference to the previous shard. // ShardTypes are pinned with MaxDepth=1 (root and // direct children only). ShardType )
PinType values. See PinType documentation for further explanation.
func PinTypeFromString ¶
PinTypeFromString is the inverse of String. It returns the PinType value corresponding to the input string
type RoundRobin ¶
type RoundRobin struct {
// contains filtered or unexported fields
}
RoundRobin is a load balancing strategy that would use clients in a sequence.
func (*RoundRobin) Next ¶
func (r *RoundRobin) Next(count int) Client
Next return the next client to be used.
func (*RoundRobin) SetClients ¶
func (r *RoundRobin) SetClients(cl []Client)
SetClients sets a list of clients for this strategy.
type StatusFilterParams ¶
type StatusFilterParams struct { Cid cid.Cid Local bool Target TrackerStatus CheckFreq time.Duration }
StatusFilterParams contains the parameters required to filter a stream of status results.
type TrackerStatus ¶
type TrackerStatus int
TrackerStatus represents the status of a tracked Cid in the PinTracker
const ( // IPFSStatus should never take this value. // When used as a filter. It means "all". TrackerStatusUndefined TrackerStatus = 0 // The cluster node is offline or not responding TrackerStatusClusterError TrackerStatus = 1 << iota // An error occurred pinning TrackerStatusPinError // An error occurred unpinning TrackerStatusUnpinError // The IPFS daemon has pinned the item TrackerStatusPinned // The IPFS daemon is currently pinning the item TrackerStatusPinning // The IPFS daemon is currently unpinning the item TrackerStatusUnpinning // The IPFS daemon is not pinning the item TrackerStatusUnpinned // The IPFS daemon is not pinning the item but it is being tracked TrackerStatusRemote // The item has been queued for pinning on the IPFS daemon TrackerStatusPinQueued // The item has been queued for unpinning on the IPFS daemon TrackerStatusUnpinQueued // The IPFS daemon is not pinning the item through this cid but it is // tracked in a cluster dag TrackerStatusSharded )
TrackerStatus values
func TrackerStatusAll ¶
func TrackerStatusAll() []TrackerStatus
TrackerStatusAll all known TrackerStatus values.
func TrackerStatusFromString ¶
func TrackerStatusFromString(str string) TrackerStatus
TrackerStatusFromString parses a string and returns the matching TrackerStatus value. The string can be a comma-separated list representing a TrackerStatus filter. Unknown status names are ignored.
func (TrackerStatus) MarshalJSON ¶
func (st TrackerStatus) MarshalJSON() ([]byte, error)
MarshalJSON uses the string representation of TrackerStatus for JSON encoding.
func (TrackerStatus) Match ¶
func (st TrackerStatus) Match(filter TrackerStatus) bool
Match returns true if the tracker status matches the given filter. For example TrackerStatusPinError will match TrackerStatusPinError and TrackerStatusError
func (TrackerStatus) String ¶
func (st TrackerStatus) String() string
String converts a TrackerStatus into a readable string. If the given TrackerStatus is a filter (with several bits set), it will return a comma-separated list.
func (*TrackerStatus) UnmarshalJSON ¶
func (st *TrackerStatus) UnmarshalJSON(data []byte) error
UnmarshalJSON sets a tracker status from its JSON representation.