slurm

Documentation

Index

• Variables
• func SLURMBatchSubmit(Ctx context.Context, config SlurmConfig, path string) (string, error)
• type CreateStruct
• type JidStruct
• type ResourceLimits
• type SidecarHandler
  • func (h *SidecarHandler) CreateDirectories() error
  • func (h *SidecarHandler) GetLogsHandler(w http.ResponseWriter, r *http.Request)
  • func (h *SidecarHandler) LoadJIDs() error
  • func (h *SidecarHandler) StatusHandler(w http.ResponseWriter, r *http.Request)
  • func (h *SidecarHandler) StopHandler(w http.ResponseWriter, r *http.Request)
  • func (h *SidecarHandler) SubmitHandler(w http.ResponseWriter, r *http.Request)
• type SingularityCommand
• type SlurmConfig
  • func NewSlurmConfig() (SlurmConfig, error)

Variables

var Clientset *kubernetes.Clientset

Functions

func SLURMBatchSubmit

func SLURMBatchSubmit(Ctx context.Context, config SlurmConfig, path string) (string, error)

SLURMBatchSubmit submits the job provided in the path argument to the SLURM queue. At this point, it's up to the SLURM scheduler to manage the job. Returns the output of the sbatch command and the first encoundered error.

Types

type CreateStruct

type CreateStruct struct {
	PodUID string `json:"PodUID"`
	PodJID string `json:"PodJID"`
}

type JidStruct

type JidStruct struct {
	PodUID       string    `json:"PodUID"`
	PodNamespace string    `json:"PodNamespace"`
	JID          string    `json:"JID"`
	StartTime    time.Time `json:"StartTime"`
	EndTime      time.Time `json:"EndTime"`
}

type ResourceLimits

type ResourceLimits struct {
	CPU    int64
	Memory int64
}

type SidecarHandler

type SidecarHandler struct {
	Config SlurmConfig
	JIDs   *map[string]*JidStruct
	Ctx    context.Context
}

func (*SidecarHandler) CreateDirectories

func (h *SidecarHandler) CreateDirectories() error

CreateDirectories is just a function to be sure directories exists at runtime

func (*SidecarHandler) GetLogsHandler

func (h *SidecarHandler) GetLogsHandler(w http.ResponseWriter, r *http.Request)

GetLogsHandler reads Jobs' output file to return what's logged inside. What's returned is based on the provided parameters (Tail/LimitBytes/Timestamps/etc)

func (*SidecarHandler) LoadJIDs

func (h *SidecarHandler) LoadJIDs() error

LoadJIDs loads Job IDs into the main JIDs struct from files in the root folder. It's useful went down and needed to be restarded, but there were jobs running, for example. Return only error in case of failure

func (*SidecarHandler) StatusHandler

func (h *SidecarHandler) StatusHandler(w http.ResponseWriter, r *http.Request)

StatusHandler performs a squeue --me and uses regular expressions to get the running Jobs' status

func (*SidecarHandler) StopHandler

func (h *SidecarHandler) StopHandler(w http.ResponseWriter, r *http.Request)

StopHandler runs a scancel command, updating JIDs and cached statuses

func (*SidecarHandler) SubmitHandler

func (h *SidecarHandler) SubmitHandler(w http.ResponseWriter, r *http.Request)

SubmitHandler generates and submits a SLURM batch script according to provided data. 1 Pod = 1 Job. If a Pod has multiple containers, every container is a line with it's parameters in the SLURM script.

type SingularityCommand

type SingularityCommand struct {
	// contains filtered or unexported fields
}

type SlurmConfig

type SlurmConfig struct {
	VKConfigPath      string `yaml:"VKConfigPath"`
	Sbatchpath        string `yaml:"SbatchPath"`
	Scancelpath       string `yaml:"ScancelPath"`
	Squeuepath        string `yaml:"SqueuePath"`
	Sidecarport       string `yaml:"SidecarPort"`
	Socket            string `yaml:"Socket"`
	ExportPodData     bool   `yaml:"ExportPodData"`
	Commandprefix     string `yaml:"CommandPrefix"`
	DataRootFolder    string `yaml:"DataRootFolder"`
	Namespace         string `yaml:"Namespace"`
	Tsocks            bool   `yaml:"Tsocks"`
	Tsockspath        string `yaml:"TsocksPath"`
	Tsockslogin       string `yaml:"TsocksLoginNode"`
	BashPath          string `yaml:"BashPath"`
	VerboseLogging    bool   `yaml:"VerboseLogging"`
	ErrorsOnlyLogging bool   `yaml:"ErrorsOnlyLogging"`
	SingularityPrefix string `yaml:"SingularityPrefix"`
	// contains filtered or unexported fields
}

InterLinkConfig holds the whole configuration

var SlurmConfigInst SlurmConfig

func NewSlurmConfig

func NewSlurmConfig() (SlurmConfig, error)

NewSlurmConfig returns a variable of type SlurmConfig, used in many other functions and the first encountered error.