ipfs

package module
v0.0.0-...-8ac4b46 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 29, 2024 License: Apache-2.0 Imports: 34 Imported by: 1

README

TL;DR

Experimental code to extract some ideas from IPFS/Libp2p - but no plan to use it, too many non-interoperable changes to protocols and complexity that is not needed.

Important ideas

  1. Pair a set of IP addresses with the SHA(public key)
  2. Use SHA(public key) of each host (workload) as the identity to lookup IPs.
  3. Support multiple protocols
  4. built-in NAT/ICE

Not viable but interesting for corner use cases:

  1. Global DHT for discovery (DNS replacement). In rare cases it may be useful, but a federated model is still decentralized but far more efficient/fast.
  2. Replacing DNS names with paths.
  3. Using unfriendly names (effectively UUIDs) for any human interaction.

Major flaws:

  1. No access control ( there is some complicated way to have a separated mesh)
  2. No separation between infra nodes (well connected, maintained) and small user nodes.
  3. DHT treats everything as a reachable peer

Intro

IPFS and libp2p are based on some (over-)complicated NIH protocols, but at the core some good ideas. While much simpler implementations would be desirable, IPFS also has a community and infrastructure that is useful

Similar (over complex, NIH) infra exists for Syncthing, Onion/Tor, etc - but libp2p is more oriented to standalone use as a library and easier to run as a private network.

Example infra:

  • cid.contact - route CID to host
    • alternative to using the DHT - too much bandwidth
    • based on gossip pubsub instead of DHT

Local DHT

The main problem with DHT is traffic and speed. This is in part due to the long-distance calls.

Libp2p DHT is tied to the protocols - and expects the peers to be connected.

It looks like bittorent DHT is ligher and simpler. Not clear which has more nodes.

HTTP routing

https://specs.ipfs.tech/routing/http-routing-v1/ - URL /routing/v1/providers/{cid}, returns a json with Addrs. Also /routing/v1/peers/{peer-id} - and ipns - returning signed record.

https://docs.ipfs.tech/how-to/address-ipfs-on-web/#path-gateway - various URL and DNS schemas

Concepts

Discovery

  • DHT/kadelmia. Avoid 'dual' - it gets confused. LAN/local should be kept separated, use auth.
  • mDNS local
  • also called 'signaling' in WebRTC
  • key is a SHA32 (of something), value is the address[]
  • example key: sha("/libp2p/relay") - address are nodes providing relay service

Circuits / Relay

  • IPFS network provides a number of well-known and discoverable open relays
  • useful for 'behind the NAT' - but should be used primarily for signaling
  • IMO the use or relay for data is a major flaw in IPFS/LIBP2P.

A better alternative is to use them only for push, with a larger set of nodes ( ideally all ), to initiate quic or webrtc connections. For fallback - standard TURN servers could be used.

This allows many home servers to get high speed ( no relay ).

Record store

  • Key is a public key - ideally ED25519
  • Value is a signed proto, typically a /ipfs address but it's a byte[]

Transports

Unfortunately IPFS has major NIH problems around transports. While the concepts are great and similar to modern standards (H2, H3), they made a choice to reject standards under the claim of 'decentralization'.

For example mTLS arbitrarily invents an new extension to pass public key, which makes absolutely no sense. Instead of 'decentralized' all protocols are locked in to IPFS.

However it is possible to plug in different transports, and to use standard based ones instead. For communication with the current IPFS infra - mainly DHT and circuit - we need to support at least TCP and their incompatible QUIC.

For browsers - there is relatively little value in having browsers act as full DHT nodes. Browsers with WSS require servers with real certs - in which case it is better to use standard protocols to the server, and have the server handle IPFS.

webrtc

  • compatible with browsers, not well maintained (3/21 - doesn't compile)
  • 'star' and 'direct'
  • /dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star/p2p/
  • star not supported in go, protocol not documented/clear
  • incomplete, security not based on webrtc certs.

ws + mplex + noise|secio

  • primarily for use with browser-based nodes, without RTC
  • requires DNS certs (apparently for wss)

quic

  • not compatible with browsers
  • flow control on streams
  • mostly standards based
  • using modified TLS - so not interoperable with non-libp2p clients

Interfaces

API

https://docs.ipfs.io/reference/http/api/

http://localhost:5001/api/v0/swarm/peers query: arg (repeated), rest are flags.

Response: JSON

Gateway

/p2p/${PEER_ID}/http/${PATH}

/p2p/${PEER_ID}/x/${PROTO}/http/${PATH}

Documentation

Index

Constants

View Source 
const Protocol = "/ugate/0.0.1"

Variables

This section is empty.

Functions

func Init 

func Init(ug *ugate.UGate)

func InitEvent 

func InitEvent(h host.Host)

Track events in the P2P implementation

func P2PAddrFromString 

func P2PAddrFromString(c string) (*peer.AddrInfo, error)

Types

type IPFS 

type IPFS struct {
	Host host.Host
}

ConnectionGater, Server

func InitIPFS 

func InitIPFS(auth *meshauth.MeshAuth, p2pport int32) *IPFS

InitIPFS creates LibP2P compatible transport. Identity is based on the EC256 workload identity in auth.

Routing is based on HTTP.

Main purpose of this integration is to take advantage of public auto-relay code and infra, for control/signaling channels.

func (*IPFS) DialContext 

func (ipfs *IPFS) DialContext(ctx context.Context, net string, addr string) (net.Conn, error)

func (*IPFS) FindPeer 

func (ipfs *IPFS) FindPeer(ctx context.Context, id peer.ID) (peer.AddrInfo, error)

func (*IPFS) InterceptAccept 

func (p2p *IPFS) InterceptAccept(multiaddrs network.ConnMultiaddrs) (allow bool)

func (*IPFS) InterceptAddrDial 

func (p2p *IPFS) InterceptAddrDial(id peer.ID, m multiaddr.Multiaddr) (allow bool)

func (*IPFS) InterceptPeerDial 

func (p2p *IPFS) InterceptPeerDial(p peer.ID) (allow bool)

func (*IPFS) InterceptSecured 

func (p2p *IPFS) InterceptSecured(direction network.Direction, id peer.ID, multiaddrs network.ConnMultiaddrs) (allow bool)

func (*IPFS) InterceptUpgraded 

func (p2p *IPFS) InterceptUpgraded(conn network.Conn) (allow bool, reason control.DisconnectReason)

type IPFSDiscovery 

type IPFSDiscovery struct {
	DHT  *dht.IpfsDHT
	Host host.Host

	Port int
	Auth *meshauth.MeshAuth

	Bootstrap []multiaddr.Multiaddr
	Key       crypto.PrivKey

	// If empty, this is used with the public infra
	Domain string
	// contains filtered or unexported fields
}

IPFSDiscovery registers in the libp2p DHT infra ( by default public - but can also be a private DHT mesh) and acts as a DHT node, implementing the http routing server API.

func NewDHT 

func NewDHT(ctx context.Context, auth *meshauth.MeshAuth, p2pport int) (*IPFSDiscovery, error)

NewDHT creates a DHT client for the public infra.

func (*IPFSDiscovery) FindPeers 

func (r *IPFSDiscovery) FindPeers(ctx context.Context, pid peer.ID, limit int) (iter.ResultIter[*types.PeerRecord], error)

func (*IPFSDiscovery) FindProviders 

func (r *IPFSDiscovery) FindProviders(ctx context.Context, key cid.Cid, limit int) (iter.ResultIter[types.Record], error)

func (*IPFSDiscovery) GetIPNS 

func (r *IPFSDiscovery) GetIPNS(ctx context.Context, name ipns.Name) (*ipns.Record, error)

func (*IPFSDiscovery) Init 

func (ipfsd *IPFSDiscovery) Init(ctx context.Context) error

NewDHT creates a host using DHT and router server.

func (*IPFSDiscovery) InterceptAccept 

func (p2p *IPFSDiscovery) InterceptAccept(multiaddrs network.ConnMultiaddrs) (allow bool)

func (*IPFSDiscovery) InterceptAddrDial 

func (p2p *IPFSDiscovery) InterceptAddrDial(id peer.ID, m multiaddr.Multiaddr) (allow bool)

func (*IPFSDiscovery) InterceptPeerDial 

func (p2p *IPFSDiscovery) InterceptPeerDial(p peer.ID) (allow bool)

func (*IPFSDiscovery) InterceptSecured 

func (p2p *IPFSDiscovery) InterceptSecured(direction network.Direction, id peer.ID, multiaddrs network.ConnMultiaddrs) (allow bool)

func (*IPFSDiscovery) InterceptUpgraded 

func (p2p *IPFSDiscovery) InterceptUpgraded(conn network.Conn) (allow bool, reason control.DisconnectReason)

func (*IPFSDiscovery) LogAddr 

func (ipfsd *IPFSDiscovery) LogAddr()

func (*IPFSDiscovery) Peering 

func (d *IPFSDiscovery) Peering(h host.Host, rt string)

func (*IPFSDiscovery) ProvideBitswap 

func (r *IPFSDiscovery) ProvideBitswap(ctx context.Context, req *server.BitswapWriteProvideRequest) (time.Duration, error)

nolint deprecated

func (*IPFSDiscovery) PutIPNS 

func (r *IPFSDiscovery) PutIPNS(ctx context.Context, name ipns.Name, record *ipns.Record) error

func (*IPFSDiscovery) ServeHTTP 

func (d *IPFSDiscovery) ServeHTTP(w http.ResponseWriter, r *http.Request)

IPFS autoregistration and debug function.

Normally should only be exposed to mesh workloads - i.e. the request should be over ambient or authenticated with JWT or mTLS.

func (*IPFSDiscovery) StartDHT 

func (d *IPFSDiscovery) StartDHT()

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.