dms

README

dms

• Project README
• Release/Build Status
• Changelog
• License
• Contribution Guidelines
• Code of Conduct
• Secure Coding Guidelines

Table of Contents

• dms
  • Table of Contents
  • Specification
    • 1. Description
    • 2. Structure and organisation
    • 3. Class Diagram
      • Source file
      • Rendered from source file
    • 4. Functionality
      • Supervision
    • 5. Data Types
    • 6. Testing
    • 7. Proposed Functionality / Requirements
      • List of issues
      • Interfaces & Methods
        • proposed Capability_interface
      • Data types
        • proposed Capability
        • proposed Connectivity
        • proposed PriceInformation
        • proposed TimeInformation
    • 8. References

Specification

Description

This package is responsible for starting the whole application. It also contains various core functionality of DMS:

• Onboarding compute provider devices
• Job orchestration and management
• Resource management
• Actor implementation for each node

Structure and Organisation

Here is quick overview of the contents of this pacakge:

• README: Current file which is aimed towards developers who wish to use and modify the dms functionality.

• dms: This file contains code to initialize the DMS by loading configuration, starting REST API server etc

• init: This file creates a new logger instance.

• sanity_check: This file defines a method for performing consistency check before starting the DMS. proposed Note that the functionality of this method needs to be developed as per refactored DMS design.

Subpackages

• jobs: Deals with the management of local jobs on the machine.

• node: Contains implementation of Node as an actor.

• onboarding: Code related to onboarding of compute provider machines to the network.

• orchestrator: Contains job orchestration logic.

• resources: Deals with the management of resources on the machine.

proposed: All files with *_test.go naming convention contain unit tests with respect to the specific implementation.

Class Diagram

The class diagram for the dms package is shown below.

Source file

dms Class diagram

Rendered from source file

!$rootUrlGitlab = "https://gitlab.com/nunet/device-management-service/-/raw/main"
!$packageRelativePath = "/dms"
!$packageUrlGitlab = $rootUrlGitlab + $packageRelativePath
 
!include $packageUrlGitlab/specs/class_diagram.puml

Functionality

TBD

Note: the functionality of DMS is being currently developed. See the proposed section for the suggested design of interfaces and methods.

Supervision

TBD as per proposed implementation

• Supervisor
• SupervisorStrategy
• Statistics

Data Types

TBD

Note: the functionality of DMS is being currently developed. See the proposed section for the suggested data types.

Testing

proposed Refer to *_test.go files for unit tests of different functionalities.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of dms package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

• dms package implementation

Interfaces & Methods

proposed Capability_interface

type Capability_interface interface {
	add()
	subtract()
}

add method will combine capabilities of two nodes. Example usage - When two jobs have to be run on a single machine, the capability requirements of each will need to be combined.

subtract method will subtract two capabilities. Example usage - When resources are locked for a job, the available capability of a machine will need to be reduced.

Data types

proposed dms.Capability

The Capability struct will capture all the relevant data that defines the capability of a node to perform the job. At the same time this will be used to define capability requirements that a job requires from a node.

An initial data model for Capability is defined below.

type Capability struct {
	// Executor is the type of executor available on the machine or the executor 
    // required for the job (example - Docker, VM, WASM etc)
    Executor    string           
	
    // Type specifies the details of type of job (One time, batch, recurring,
    // long running)
    Type        dms.jobs.JobType        
	
    // Resources specifies the description of the resources required
    Resources   dms.resources.Resource

    // Libraries specifies the libraries needed for the job        
	Libraries   []string         
	
    // Locality contains preferred localities of the machine for execution
    Locality    []string         
	
    // Storage specifies the preferred storage options that the machine should have
    Storage         []string         
	
    // Connectivity specifies the network configuration required
    Connectivity    dms.Connectivity          
	
    // Price specifies the price information of the job / machine
    Price           dms.PriceInformation 
	
    // Time specifies the time information of the job / machine
	Time            dms.TimeInformation 
	
    // KYC specifies the KYC requirements or KYC status of the machine 
    KYC  []string        
}

proposed dms.Connectivity

type Connectivity struct {

// Ports contains the ports that need to be open for the job to run
Ports []int

// VPN specifies whether VPN is required
VPN       bool  

}

proposed dms.PriceInformation

type PriceInformation struct {
	// Currency holds which currency is used for pricing ex - NTX
	Currency   string 
	
    // CurrencyPerHour is the price of the machine per hour
    CurrencyPerHour int   
	
    // TotalPerJob is the maximum total price or budget of the job
	TotalPerJob   int 
	
    // Preference is Pricing preference as compared to time
    Preference int 
}

proposed dms.TimeInformation

type TimeInformation struct { // Units holds the units of time ex - hours, days, weeks Units string

// MaxTime holds the maximum time that the job should run
MaxTime    int

// Preference holds the time preference as compared to price
Preference int    

}

References

Documentation

Index

• Variables
• func GenerateAndStorePrivKey(ks keystore.KeyStore, passphrase string, keyID string) (crypto.PrivKey, error)
• func NewDMSDB(path string) (*clover.DB, error)
• func SanityCheck(_ *gorm.DB)
• type DMS
  • func NewDMS(gcfg *config.Config, ksPassphrase, contextName string) (*DMS, error)
  • func (d *DMS) Run() error
  • func (d *DMS) Stop()

Variables

var ErrNotFound = errors.New("not found")

Functions

func GenerateAndStorePrivKey

func GenerateAndStorePrivKey(ks keystore.KeyStore, passphrase string, keyID string) (crypto.PrivKey, error)

GenerateAndStorePrivKey generates a new key pair using Secp256k1, storing the private key into user's keystore.

func NewDMSDB

func NewDMSDB(path string) (*clover.DB, error)

NewDMSDB creates a clover database with all known dms collections

func SanityCheck

func SanityCheck(_ *gorm.DB)

SanityCheck before being deleted performed basic consistency checks before starting the DMS in the following sequence: It checks for services that are marked running from the database and stops then removes them. Update their status to 'finshed with errors'. Recalculates free resources and update the database.

Deleted now because dependencies such as the docker package have been replaced with executor/docker

Types

type DMS

type DMS struct {
	P2P        *libp2p.Libp2p
	Node       *node.Node
	RestServer *api.Server
}

func NewDMS

func NewDMS(gcfg *config.Config, ksPassphrase, contextName string) (*DMS, error)

func (*DMS) Run

func (d *DMS) Run() error

func (*DMS) Stop

func (d *DMS) Stop()