cli

package module
v1.6.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 10, 2024 License: LGPL-3.0 Imports: 0 Imported by: 0

README

The Grid CLI

Submit directly jobs to the DeepSquare Grid!

tui-demo

submit-demo

Usage

NAME:
   dps - Overwatch the job scheduling and register the compute to the Deepsquare Grid.

USAGE:
   dps [global options] command [command options] [arguments...]

COMMANDS:
   allowance  Manage allowance.
   credit     Manage credits.
   init       Bootstrap a job workflow file.
   job        Manage jobs.
   provider   Manage providers (need to use an admin smart-contract).
   submit     Quickly submit a job.
   help, h    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --metascheduler.rpc value             Metascheduler Avalanche C-Chain JSON-RPC endpoint. (default: "https://testnet.deepsquare.run/rpc") [$METASCHEDULER_RPC]
   --metascheduler.ws value              Metascheduler Avalanche C-Chain WS endpoint. (default: "wss://testnet.deepsquare.run/ws") [$METASCHEDULER_WS]
   --metascheduler.smart-contract value  Metascheduler smart-contract address. (default: "0x3707aB457CF457275b7ec32e203c54df80C299d5") [$METASCHEDULER_SMART_CONTRACT]
   --sbatch.endpoint value               SBatch Service GraphQL endpoint. (default: "https://sbatch.deepsquare.run/graphql") [$SBATCH_ENDPOINT]
   --logger.endpoint value               Grid Logger endpoint. (default: "https://grid-logger.deepsquare.run") [$LOGGER_ENDPOINT]
   --private-key value                   An hexadecimal private key for ethereum transactions. [$ETH_PRIVATE_KEY]
   --debug                               Debug logging (default: false) [$DEBUG]
   --help, -h                            show help
   --version, -v                         print the version

submit:

NAME:
   dps submit - Quickly submit a job.

USAGE:
   dps submit [command options] <job.yaml>

OPTIONS:
   DeepSquare Settings:

   --logger.endpoint value               Grid Logger endpoint. (default: "https://grid-logger.deepsquare.run") [$LOGGER_ENDPOINT]
   --metascheduler.rpc value             Metascheduler Avalanche C-Chain JSON-RPC endpoint. (default: "https://testnet.deepsquare.run/rpc") [$METASCHEDULER_RPC]
   --metascheduler.smart-contract value  Metascheduler smart-contract address. (default: "0x3707aB457CF457275b7ec32e203c54df80C299d5") [$METASCHEDULER_SMART_CONTRACT]
   --metascheduler.ws value              Metascheduler Avalanche C-Chain WS endpoint. (default: "wss://testnet.deepsquare.run/ws") [$METASCHEDULER_WS]
   --private-key value                   An hexadecimal private key for ethereum transactions. [$ETH_PRIVATE_KEY]
   --sbatch.endpoint value               SBatch Service GraphQL endpoint. (default: "https://sbatch.deepsquare.run/graphql") [$SBATCH_ENDPOINT]

   Submit Settings:

   --affinities key<value [ --affinities key<value ]  Affinities flag. Used to filter the clusters. Format: key<value, `key<=value`, `key=value`, `key>=value`, `key>value`, `key!=value`
   --credits value                                    Allocated a number of credits. Unit is 1e18. Is a float and is not precise. (default: 0)
   --credits-wei value                                Allocated a number of credits. Unit is wei. Is a big int.
   --exit-on-job-exit, -e                             Exit the job after the job has finished and throw on error. (default: false)
   --job-name value                                   The job name.
   --no-timestamp, --no-ts                            Hide timestamp. (default: false)
   --uses key=value [ --uses key=value ]              Uses flag. Used to filter the clusters. Format: key=value
   --watch, -w                                        Watch logs after submitting the job (default: false)

Install binaries

You can download static binaries in the Releases tab.

Install from source

Install Go and run:

go install github.com/deepsquare-io/grid/cli/cmd/dps@latest

Build

The main is stored in the cmd/dps package. You can build the CLI using the following:

make
# Defaults to: make bin/dps

Available make commands:

# Build
make bin/dps
make bin/dps-darwin-amd64
make bin/dps-darwin-arm64
make bin/dps-freebsd-amd64
make bin/dps-freebsd-arm64
make bin/dps-linux-amd64
make bin/dps-linux-arm64
make bin/dps-linux-mips64
make bin/dps-linux-mips64le
make bin/dps-linux-ppc64
make bin/dps-linux-ppc64le
make bin/dps-linux-riscv64
make bin/dps-linux-s390x
make bin/dps-windows-amd64.exe
make build-all

# Checksums
make bin/checksums.md
make bin/checksums.txt

# Clean builds
make clean

# Generate code
make generate

# Generate license
make license

# Lint code
make lint

# Run unit tests
make unit

# Print version that will be used for the next build.
make version

# Preview doc
make doc

Use as a library

The deepsquare, sbatch, logger and metascheduler packages can be used as a library. For example, to submit a job:

// Parse private key
pk, err := crypto.HexToECDSA(ethHexPK)
if err != nil {
   // ...
}

// Initialize client for simple RPCs
client, err := deepsquare.NewClient(ctx, &deepsquare.ClientConfig{
   MetaschedulerAddress: common.HexToAddress("0x..."),
   RPCEndpoint:          "https://testnet.deepsquare.run/rpc",  // Optional
   SBatchEndpoint:       "https://sbatch.deepsquare.run/graphql",  // Optional
   LoggerEndpoint:       "https://grid-logger.deepsquare.run",  // Optional
   UserPrivateKey:       pk,  // Optional, but needed for authenticated requests
})

// Example of job submit
curr, err := client.GetAllowance(ctx)
if err != nil {
   // ...
}
err = client.SetAllowance(ctx, curr.Add(curr, lockedAmount))
if err != nil {
   // ...
}
jobID, err = client.SubmitJob(
   ctx,
   &sbatch.Job{
      Resources: &sbatch.JobResources{
         Tasks:       1,
         CpusPerTask: 1,
         MemPerCPU:   100,
         Gpus: 0,
      },
      Steps: []*sbatch.Step{
         {
            Run: &sbatch.StepRun{
               Command: "echo test",
            },
         },
      },
   },
   big.NewInt(100),
   jobName,
)

For more information, check out:

Licence

The DeepSquare CLI library (i.e. all code outside the cmd and tui directories) is licensed under the GNU Lesser General Public License v3.0, also included in our repository in the COPYING file.

The DeepSquare CLI binaries (i.e. all code inside the cmd and tui directories) are licensed under the GNU General Public License v3.0, also included in our repository in the COPYING.GPL3 file.

Documentation

Overview

Package cli is the root of the Go client for interacting with the DeepSquare Grid.

The Terminal User Interface

The TUI can be run by simply running the github.com/deepsquare-io/grid/cli/cmd/dps package. Its implementation is in the github.com/deepsquare-io/grid/cli/tui package and uses the bubbletea framework to build the terminal application.

The Command Line Interface

The CLI can be run by running subcommands defined in the github.com/deepsquare-io/grid/cli/cmd package. Compared to the TUI, the CLI contains all the commands of the DeepSquare Client.

The DeepSquare Client

A DeepSquare client is available in the github.com/deepsquare-io/grid/cli/deepsquare package. It implements all of the features needed to manage jobs, providers, credits and job logs.

Example: 

// Parse private key
pk, err := crypto.HexToECDSA(ethHexPK)
if err != nil {
	// ...
}

// Initialize client for simple RPCs
client, err := deepsquare.NewClient(ctx, &deepsquare.ClientConfig{
	MetaschedulerAddress: common.HexToAddress("0x..."),
	RPCEndpoint:          "https://testnet.deepsquare.run/rpc",  // Optional
	SBatchEndpoint:       "https://sbatch.deepsquare.run/graphql",  // Optional
	LoggerEndpoint:       "https://grid-logger.deepsquare.run",  // Optional
	UserPrivateKey:       pk,  // Optional, but needed for authenticated requests
})

// Example of job submit
curr, err := client.AllowanceManager.GetAllowance(ctx)
if err != nil {
	// ...
}
err = client.AllowanceManager.SetAllowance(ctx, curr.Add(curr, lockedAmount))
if err != nil {
	// ...
}
jobID, err := client.SubmitJob(ctx, job, lockedAmount, jobName, types.WithAffinity(affinities))

// Initialize client for streaming RPCs
watcher, err := deepsquare.NewWatcher(ctx, &deepsquare.WatcherConfig{
	MetaschedulerAddress: common.HexToAddress("0x..."),
	RPCEndpoint:          "https://testnet.deepsquare.run/rpc",  // Optional
	WSEndpoint:           "https://testnet.deepsquare.run/ws",  // Optional
	UserPrivateKey:       pk,  // Optional, but needed for authenticated requests
})
defer watcher.Close()

// Example of watching job transition
transitions := make(chan types.JobTransition, 1)
sub, err := watcher.SubscribeEvents(ctx, event.FilterJobTransition(transitions))
if err != nil {
	return err
}
defer sub.Unsubscribe()

See the github.com/deepsquare-io/grid/cli/deepsquare package documentation for more information.

Fine-grained RPC or WebSocket client

If the DeepSquare client is too much "coupled", a more fine-grained client is available in the github.com/deepsquare-io/grid/cli/metascheduler package.

Example: 

// Initialize RPC client
ethClientRPC := ethclient.Dial("https://testnet.deepsquare.run/rpc")
chainID, err := ethClientRPC.ChainID(ctx)
if err != nil {
	return nil, err
}
rpcBackend := metascheduler.Backend{
	EthereumBackend:      ethClientRPC,
	MetaschedulerAddress: common.HexToAddress("0x..."),
	ChainID:              chainID,
	UserPrivateKey:       pk,
}
clientset := metascheduler.NewRPCClientSet(rpcBackend)

// Example of job submit
curr, err := clientset.AllowanceManager().GetAllowance(ctx)
if err != nil {
	// ...
}
err = clientset.AllowanceManager().SetAllowance(ctx, curr.Add(curr, lockedAmount))
if err != nil {
	// ...
}

sbatch := sbatch.NewService(http.DefaultClient, "https://sbatch.deepsquare.run/graphql")
jobID, err := clientset.JobScheduler(sbatch).SubmitJob(ctx, job, lockedAmount, jobName, types.WithAffinity(affinities))

// Initialize WebSocket client
ethClientWS := ethclient.Dial("https://testnet.deepsquare.run/ws")
defer ethClientWS.Close()
wsBackend := metascheduler.Backend{
	MetaschedulerAddress: common.HexToAddress("0x..."),
	EthereumBackend:      ethClientWS,
	ChainID:              chainID,
	UserPrivateKey:       pk,
}
es := metascheduler.NewEventSubscriber(rpcBackend, wsBackend)

// Example of watching job transition
transitions := make(chan types.JobTransition, 1)
sub, err := es.SubscribeEvents(ctx, event.FilterJobTransition(transitions))
if err != nil {
	return err
}
defer sub.Unsubscribe()

See the github.com/deepsquare-io/grid/cli/metascheduler package documentation for more information.

Source Files

View all Source files

Directories

Path Synopsis
_examples
init
jobs
submit
cmd
allowance
Package allowance provides subcommands to manage the allowance of an user.
 Package allowance provides subcommands to manage the allowance of an user.
credit
Package credit provides subcommands to manage the credits of an user.
 Package credit provides subcommands to manage the credits of an user.
dps
DeepSquare TUI is a client to launch jobs on the DeepSquare Grid.
 DeepSquare TUI is a client to launch jobs on the DeepSquare Grid.
initc
Package initc provides subcommands to initialize templates to get started with DeepSquare.
 Package initc provides subcommands to initialize templates to get started with DeepSquare.
job
Package job provides subcommands to manage the jobs of an user.
 Package job provides subcommands to manage the jobs of an user.
provider
Package provider provides subcommands to manage the providers of the Grid.
 Package provider provides subcommands to manage the providers of the Grid.
submit
Package submit permits the submission of a job to the DeepSquare Grid.
 Package submit permits the submission of a job to the DeepSquare Grid.
validate
Package submit permits the submission of a job to the DeepSquare Grid.
 Package submit permits the submission of a job to the DeepSquare Grid.
Package deepsquare provides a all-in-one Client that implements all of the DeepSquare services.
 Package deepsquare provides a all-in-one Client that implements all of the DeepSquare services.
internal
ether
Package ether provides utilities for ether to wei conversions.
 Package ether provides utilities for ether to wei conversions.
graphql
Package graphql provides wrappers for graphQL API objects.
 Package graphql provides wrappers for graphQL API objects.
log
Package log initialize the logger for the TUI.
 Package log initialize the logger for the TUI.
logger/v1alpha1
utils
Package utils provides utilities functions.
 Package utils provides utilities functions.
validator
Package validator providers utilities for user input validation.
 Package validator providers utilities for user input validation.
wordlists
Package wordlists providers utilities to generate random words.
 Package wordlists providers utilities to generate random words.
Package logger providers an implementation of the Grid Logger reader.
 Package logger providers an implementation of the Grid Logger reader.
Package metascheduler defines APIs for interacting with the DeepSquare Grid.
 Package metascheduler defines APIs for interacting with the DeepSquare Grid.
mocks
mockjob
Package sbatch provides implementations of the SBatchService client.
 Package sbatch provides implementations of the SBatchService client.
tui
Package tui implements all the fron-end logic for a Terminal User Interface of the DeepSquare client.
 Package tui implements all the fron-end logic for a Terminal User Interface of the DeepSquare client.
channel
Package channel provides utilities to create an Bubbletea model compatible with Go channels.
 Package channel provides utilities to create an Bubbletea model compatible with Go channels.
components/table
Package table is a forked bubbletea component with support single cell styling.
 Package table is a forked bubbletea component with support single cell styling.
components/ticker
Package ticker contains the Model and Update function to work with a Go Ticker.
 Package ticker contains the Model and Update function to work with a Go Ticker.
editor
Package editor contains the Model, View and Update functions for the workflow editor.
 Package editor contains the Model, View and Update functions for the workflow editor.
nav
Package nav contains the Model, View and Update functions for the main navigator.
 Package nav contains the Model, View and Update functions for the main navigator.
provider
Package provider contains the Model, View and Update functions to manage providers.
 Package provider contains the Model, View and Update functions to manage providers.
provider/details
Package details contains the Model, View and Update functions for viewing the provider details.
 Package details contains the Model, View and Update functions for viewing the provider details.
status
Package status contains the Model, View and Update functions for the job statuses dashboard.
 Package status contains the Model, View and Update functions for the job statuses dashboard.
status/log
Package log contains the Model, View and Update functions for viewing the logs of a job.
 Package log contains the Model, View and Update functions for viewing the logs of a job.
status/topup
Package topup contains the Model, View and Update functions for submitting a top up request.
 Package topup contains the Model, View and Update functions for submitting a top up request.
style
Package style provides object for styling the TUI.
 Package style provides object for styling the TUI.
transfer
Package transfer contains the Model, View and Update functions for transferring credits between users.
 Package transfer contains the Model, View and Update functions for transferring credits between users.
Package types provides the main types of the library.
 Package types provides the main types of the library.
abi/errors
abi/metascheduler
allowance
Package allowance describes the types and methods to interact with the
 Package allowance describes the types and methods to interact with the
credit
Package credit describes the types and methods to interact with the credits.
 Package credit describes the types and methods to interact with the credits.
event
Package event describes the types and methods to interact with the events.
 Package event describes the types and methods to interact with the events.
job
Package job describes the types and methods to interact with the jobs.
 Package job describes the types and methods to interact with the jobs.
provider
Package provider defines the interface for managing providers.
 Package provider defines the interface for managing providers.
Package version contains functions to track the module version.
 Package version contains functions to track the module version.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.