ipfscliwrapper

README

IPFS CLI Wrapper

ipfs-cli-wrapper is a Go package that provides a convenient way to manage IPFS nodes by running the IPFS binary as a separate process alongside your Go application. This package allows you to easily start, control, and interact with an IPFS node using the HTTP Kubo RPC API.

Key Features

• Automatic IPFS Binary Download: Automatically downloads the appropriate IPFS binary for your machine's architecture on first use.
• Run IPFS as a Background Process: Start the IPFS daemon in the background, allowing your application to continue running seamlessly.
• HTTP API Interaction: Use the HTTP Kubo RPC API to interact with your IPFS node.

Getting Started

Follow these steps to start using ipfs-cli-wrapper in your Go project:

Installation

1. Install the package:

   go get github.com/bartmika/ipfs-cli-wrapper
   
   

2. Update your .gitignore to exclude the IPFS binary:

   Add the following to your .gitignore to prevent the binary from being tracked in your version control:

   bin
   ./bin
   ./bin/*
   

3. Open port 4001 on your server:

   Make sure port 4001 (the Swarm port used by IPFS) is open on your server to allow connections to the IPFS network.

Basic Usage

Here's a simple example to get started with ipfs-cli-wrapper:

package main

import (
    "log"
    ipfswrap "github.com/bartmika/ipfs-cli-wrapper"
)

func main() {
    // Initialize the IPFS CLI wrapper.
    wrapper, err := ipfswrap.NewWrapper()
    if err != nil {
        log.Fatalf("Failed to create IPFS wrapper: %v", err)
    }

    // Start the IPFS daemon in the background.
    if err := wrapper.StartDaemonInBackground(); err != nil {
        log.Fatalf("Failed to start IPFS daemon: %v", err)
    }

    // Ensure the IPFS daemon shuts down gracefully on application exit.
    defer func() {
        if err := wrapper.ShutdownDaemon(); err != nil {
            log.Fatalf("Failed to shut down IPFS daemon: %v", err)
        }
    }()

    // Continue with your application logic...
}

Advanced Example: Interacting with the IPFS Node

Once the IPFS daemon is running, you can interact with it using the HTTP Kubo RPC API:

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"
    "strings"

    ipfsFiles "github.com/ipfs/go-ipfs-files"
    "github.com/ipfs/kubo/client/rpc"
    ipfscliwrapper "github.com/bartmika/ipfs-cli-wrapper"
)

func main() {
    wrapper, err := ipfscliwrapper.NewWrapper()
    if err != nil {
        log.Fatalf("Failed to create IPFS wrapper: %v", err)
    }

    if err := wrapper.StartDaemonInBackground(); err != nil {
        log.Fatalf("Failed to start IPFS daemon: %v", err)
    }
    defer func() {
        if err := wrapper.ShutdownDaemon(); err != nil {
            log.Fatalf("Failed to shut down IPFS daemon: %v", err)
        }
    }()

    // Create an IPFS HTTP client.
    httpClient := &http.Client{}
    httpApi, err := rpc.NewURLApiWithClient("http://127.0.0.1:5001", httpClient)
    if err != nil {
        log.Fatalf("Failed to create IPFS HTTP API client: %v", err)
    }

    // Add data to IPFS.
    content := strings.NewReader("Hello world from IPFS CLI Wrapper!")
    p, err := httpApi.Unixfs().Add(context.Background(), ipfsFiles.NewReaderFile(content))
    if err != nil {
        log.Fatalf("Failed to add data to IPFS: %v", err)
    }

    fmt.Printf("Data stored in IPFS with CID: %v\n", p)
}

For the full example, see the RPC example code.

Documentation

Detailed documentation can be found on pkg.go.dev.

Examples

See the examples folder for more code samples and use cases.

Contributing

Found a bug or have a feature request? Please open an issue. Contributions are welcome!

License

Made with ❤️ by Bartlomiej Mika.
The project is licensed under the ISC License.

Documentation

Overview

ipfscliwrapper is a package that manages running an IPFS node in the background and offers a user-friendly interface, enabling you to build IPFS-embedded Golang applications more easily.

Index

• Constants
• type IpfsCliWrapper
  • func NewWrapper(options ...Option) (IpfsCliWrapper, error)
• type IpfsNodeInfo
• type Option
  • func WithContinousOperation() Option
  • func WithCustomOsOperator(osOperator oskit.OSOperater) Option
  • func WithCustomRandomGenerator(gen randomkit.RandomGenerator) Option
  • func WithCustomUrlDownloader(urlDownloader urlkit.URLDownloader) Option
  • func WithDenylist(denylistFilename string, denylistURL string) Option
  • func WithForcedShutdownDaemonOnStartup() Option
  • func WithOverrideBinaryOsAndArch(overrideOS, overrideArch string) Option
  • func WithOverrideDaemonInitialWarmupDuration(seconds int) Option

Constants

const (
	// IPFSBinaryFilePath defines the path to the IPFS binary executable
	// (commonly known as 'kubo'). This path is used when executing IPFS
	// commands via the command line interface in the application.
	IPFSBinaryFilePath = "./bin/kubo/ipfs"

	// IPFSDataDirPath defines the path to the directory where IPFS stores
	// its data, including the repository and configuration files. This path
	// is crucial for ensuring the IPFS node has access to its necessary
	// data files during operation.
	IPFSDataDirPath = "./bin/kubo/data"

	// IPFSDenylistDirPath defines the path to the denylist directory within
	// the IPFS data directory. Denylists are used to block or restrict
	// access to certain content on the IPFS network by specifying content
	// that should not be accessed or shared.
	IPFSDenylistDirPath = IPFSDataDirPath + "/denylists/"
)

Constants related to the IPFS binary and data directory paths.

const (
	// AllPinType represents the option to list all types of pinned objects in IPFS.
	// This type includes recursive, direct, and indirect pins.
	AllPinType = "all"

	// RecursivePinType represents recursively pinned objects in IPFS.
	// Recursively pinned objects ensure that all their linked content (descendants)
	// is also pinned, keeping the entire data tree available on the node.
	RecursivePinType = "recursive"

	// IndirectPinType represents indirectly pinned objects in IPFS.
	// Indirect pins occur when an object is pinned due to being linked by
	// a recursively pinned object, rather than being pinned directly.
	IndirectPinType = "indirect"

	// DirectPinType represents directly pinned objects in IPFS.
	// Direct pins ensure that only the specified object is kept available,
	// without recursively pinning any linked content.
	DirectPinType = "direct"
)

Constants representing various types of pins in IPFS.

Types

type IpfsCliWrapper

type IpfsCliWrapper interface {
	// StartDaemonInBackground starts the IPFS daemon process in the background,
	// making it ready to accept API requests. It should ensure that the daemon
	// runs independently of the calling application.
	//
	// Returns an error if the daemon fails to start.
	StartDaemonInBackground() error

	// ShutdownDaemon gracefully shuts down the running IPFS daemon.
	// It sends a termination signal to the daemon process, allowing it
	// to perform cleanup tasks before shutting down.
	//
	// Returns an error if the daemon could not be shut down.
	ShutdownDaemon() error

	// ForceShutdownDaemon immediately terminates the IPFS daemon process,
	// without allowing it to perform any cleanup. This is a forceful operation
	// that should be used when the daemon does not respond to a graceful shutdown.
	//
	// Returns an error if the daemon could not be forcefully terminated.
	ForceShutdownDaemon() error

	// AddFile adds a file to the IPFS network using its file path. The function
	// executes the `ipfs add` command to store the file in the IPFS node.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   filepath - The path to the file to be added to IPFS.
	//
	// Returns:
	//   The CID (Content Identifier) of the added file on success.
	//   An error if the file could not be added.
	AddFile(ctx context.Context, filepath string) (string, error)

	// AddFileContent adds a file to the IPFS network from a byte slice containing
	// the file content, rather than a file path. The function handles the creation
	// and storage of the file directly in the IPFS node.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   fileContent - The byte slice containing the content of the file.
	//
	// Returns:
	//   The CID (Content Identifier) of the added file on success.
	//   An error if the file could not be added.
	AddFileContent(ctx context.Context, fileContent []byte) (string, error)

	// GetFile retrieves a file from the IPFS network using its CID (Content Identifier).
	// The function executes the `ipfs get` command, which downloads the file from the
	// IPFS network to the local machine.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   cid - The CID of the file to be retrieved from IPFS.
	//
	// Returns an error if the file could not be retrieved.
	GetFile(ctx context.Context, cid string) error

	// Cat retrieves the content of a file from the IPFS network using its CID and returns it as a byte slice.
	// The function executes the `ipfs cat` command, which outputs the file content directly.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   cid - The CID of the file whose content is to be retrieved from IPFS.
	//
	// Returns:
	//   A byte slice containing the file content on success.
	//   An error if the file content could not be retrieved.
	Cat(ctx context.Context, cid string) ([]byte, error)

	// ListPins retrieves a list of all pinned objects' CIDs from the IPFS node.
	// The function executes the `ipfs pin ls` command to fetch the list of pins.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//
	// Returns:
	//   A slice of strings, each representing a CID of a pinned object.
	//   An error if the pins could not be listed.
	ListPins(ctx context.Context) ([]string, error)

	// ListPinsByType retrieves a list of pinned objects' CIDs from the IPFS node
	// filtered by a specific type (e.g., recursive, direct).
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   typeID - The type of pins to list (e.g., "all", "recursive", "direct", "indirect").
	//
	// Returns:
	//   A slice of strings, each representing a CID of a pinned object of the specified type.
	//   An error if the pins could not be listed.
	ListPinsByType(ctx context.Context, typeID string) ([]string, error)

	// Pin pins an object in the IPFS node using its CID, ensuring the object
	// remains available locally on the IPFS node and is not removed during
	// garbage collection.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   cid - The CID of the object to pin in IPFS.
	//
	// Returns an error if the object could not be pinned.
	Pin(ctx context.Context, cid string) error

	// Unpin removes a pinned object from the IPFS node, making it eligible
	// for removal during garbage collection if it is no longer needed.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//   cid - The CID of the object to unpin from IPFS.
	//
	// Returns an error if the object could not be unpinned.
	Unpin(ctx context.Context, cid string) error

	// GarbageCollection runs the garbage collection process on the IPFS node,
	// removing any unpinned objects that are no longer needed, freeing up space.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//
	// Returns an error if the garbage collection process failed.
	GarbageCollection(ctx context.Context) error

	// Id returns the IPFS node connection details of the running daemon.
	//
	// Parameters:
	//   ctx - Context for controlling cancellation and deadlines.
	//
	// Returns an error if the failed getting connection details from IPFS.
	Id(ctx context.Context) (*IpfsNodeInfo, error)
}

IpfsCliWrapper interface represents a wrapper around the `ipfs` executable binary in the operating system, providing methods to control the IPFS daemon and perform various operations such as adding files, retrieving content, pinning, and garbage collection.

func NewWrapper

func NewWrapper(options ...Option) (IpfsCliWrapper, error)

NewWrapper creates a new instance of IpfsCliWrapper with the specified options. This function provides a flexible way to initialize the wrapper, allowing customization through a set of functional options that modify the default configuration.

Parameters:

• options: A variadic list of Option functions that customize the behavior of the wrapper. Each Option function applies a specific modification to the configuration.

Returns:

• (IpfsCliWrapper, error): Returns an initialized IpfsCliWrapper instance or an error if the configuration is invalid or if initialization fails.

Example usage:

wrapper, err := NewWrapper(
    WithLogger(myLogger),
    WithDaemonWarmupDuration(5 * time.Second),
    WithContinuousDaemonRunning(true),
)
if err != nil {
    log.Fatalf("Failed to initialize IPFS CLI wrapper: %v", err)
}

Notes:

• The wrapper is designed to abstract the complexities of managing the IPFS daemon, providing a simple interface for starting, stopping, and interacting with the daemon.
• It is crucial to configure the daemon warmup duration appropriately based on the expected startup time of the IPFS daemon on the host machine. Insufficient warmup time can lead to unexpected errors or failures in subsequent operations.
• For long-running IPFS nodes that should not be interrupted, set `isDaemonRunningContinously` to true to ensure the daemon persists until explicitly shut down using `ForceShutdown()`.

type IpfsNodeInfo

type IpfsNodeInfo struct {
	ID              string   `json:"ID"`
	PublicKey       string   `json:"PublicKey"`
	Addresses       []string `json:"Addresses"`
	AgentVersion    string   `json:"AgentVersion"`
	ProtocolVersion string   `json:"ProtocolVersion"`
}

IpfsNodeInfo represents the structured data of the `id` command results.

type Option

type Option func(*ipfsCliWrapper)

Option is a functional option type that allows us to configure the IpfsCliWrapper.

func WithContinousOperation

func WithContinousOperation() Option

WithContinousOperation is a functional option to configure our wrapper to not terminate the operation of the `ipfs` binary when running in the background; in addition when we run the `Start()` function, no errors will occure pertaining to previously active running `ipfs` binary instance. This is a useful option if you developing an app in which you restart often and you don't want to restart the `ipfs` binary often then use this option.

func WithCustomOsOperator

func WithCustomOsOperator(osOperator oskit.OSOperater) Option

func WithCustomRandomGenerator

func WithCustomRandomGenerator(gen randomkit.RandomGenerator) Option

func WithCustomUrlDownloader

func WithCustomUrlDownloader(urlDownloader urlkit.URLDownloader) Option

func WithDenylist

func WithDenylist(denylistFilename string, denylistURL string) Option

WithDenylist is a functional option which downloads a `denylist` [0] from the URL you provided and applies it to the `ipfs` binary running instance. [0] https://github.com/ipfs/kubo/blob/master/docs/content-blocking.md

func WithForcedShutdownDaemonOnStartup

func WithForcedShutdownDaemonOnStartup() Option

WithForcedShutdownDaemonOnStartup is a functional option to add if you want this package to look for any previously running `ipfs` binary in the system background and shut it down before our package loads up a new `ipfs` binary instance.

func WithOverrideBinaryOsAndArch

func WithOverrideBinaryOsAndArch(overrideOS, overrideArch string) Option

WithOverrideBinaryOsAndArch is a functional option to configure our wrapper to use a specific binary. The available `os` options are: darwin, linux, freebsd, openbsd and windows. The available `arch` choices are: arm, arm64, 386, and amd64.

func WithOverrideDaemonInitialWarmupDuration

func WithOverrideDaemonInitialWarmupDuration(seconds int) Option

WithOverrideDaemonInitialWarmupDuration is a functional option to configure our wrapper to set a custom warmup delay for our app to give a custom delay to allow the `ipfs` to loadup before giving your app execution control.