quantRunner

Documentation

Overview

Exposes interfaces and structures required to run PIQUANT in the Kubernetes cluster along with functions to access quantification files, logs, results and summaries of quant jobs.

Example (EstimateNodeCount)

// Based on experimental runs in: https://github.com/pixlise/core/-/issues/113

// Can only use the ones where we had allcoation of 7.5 set in kubernetes, because the others weren't maxing out cores

// 5x11, 4035 PMCs, 3 elements, 8 cores (7.5 allocation in kubernetes), 10 nodes. Runtime: 3:22
fmt.Println(EstimateNodeCount(4035*2, 3, 3*60+22, 8, 50))
// 5x11, 4035 PMCs, 3 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 1:52 (with params)
fmt.Println(EstimateNodeCount(4035*2, 3, 60+52, 8, 50))
// 5x11, 4035 PMCs, 4 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:11 (with params)
fmt.Println(EstimateNodeCount(4035*2, 4, 2*60+11, 8, 50))
// 5x11, 4035 PMCs, 5 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:26 (with params)
fmt.Println(EstimateNodeCount(4035*2, 5, 2*60+26, 8, 50))
// 5x11, 4035 PMCs, 6 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:47 (with params)
fmt.Println(EstimateNodeCount(4035*2, 6, 2*60+47, 8, 50))
// 5x11, 4035 PMCs, 7 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:55 (no params though)
fmt.Println(EstimateNodeCount(4035*2, 7, 2*60+55, 8, 50))
// 5x11, 4035 PMCs, 8 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:12 (with params)
fmt.Println(EstimateNodeCount(4035*2, 8, 3*60+12, 8, 50))
// 5x11, 4035 PMCs, 9 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:23 (with params)
fmt.Println(EstimateNodeCount(4035*2, 9, 3*60+23, 8, 50))
// 5x11, 4035 PMCs, 10 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:35 (with params)
fmt.Println(EstimateNodeCount(4035*2, 10, 3*60+35, 8, 50))
// 5x11, 4035 PMCs, 11 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:46 (with params)
fmt.Println(EstimateNodeCount(4035*2, 11, 3*60+46, 8, 50))

// 5x5, 1769 PMCs, 11 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 1:47 (with params)
fmt.Println(EstimateNodeCount(1769*2, 11, 60+47, 8, 50))

// 5x5, 1769 PMCs, 4 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 0:59 (with params)
fmt.Println(EstimateNodeCount(1769*2, 4, 59, 8, 50))

// Ensure the max cores have an effect
fmt.Println(EstimateNodeCount(1769*2, 4, 59, 8, 6))

// It's a bit unfortunate we ran all but 1 tests on the same number of cores, but
// the above data varies the spectra count, element count and expected runtime

// Below we'd expect 20 for all answers except the first one, but there's a slight (and
// allowable) drift because we're not exactly spot on with our estimate, and there's
// fixed overhead time we aren't even calculating properly

Output:

10
18
18
18
18
19
19
20
20
21
19
17
6

Example (FilesPerNode)

fmt.Println(FilesPerNode(8088, 5))
fmt.Println(FilesPerNode(8068, 3))

Output:

1619
2690

Index

• func EstimateNodeCount(spectraCount int32, elementCount int32, desiredRunTimeSec int32, ...) int32
• func FilesPerNode(spectraCount int32, nodeCount int32) int32
• func MakePMCFileName(nodeNumber int) string
• type PiquantParams
• type QuantRunner
  • func GetQuantRunner(name string) (QuantRunner, error)

Examples

• Package (EstimateNodeCount)
• Package (FilesPerNode)

Functions

func EstimateNodeCount

func EstimateNodeCount(spectraCount int32, elementCount int32, desiredRunTimeSec int32, coresPerNode int32, maxNodes int32) int32

func FilesPerNode

func FilesPerNode(spectraCount int32, nodeCount int32) int32

func MakePMCFileName

func MakePMCFileName(nodeNumber int) string

MakePMCFileName - generates a name for the PMC list file based on a given node-numer

NOTE: nodes are 0-based whereas pmc filenames are 1-based,
	  so the first node is 0 and will work on pmcs in node00001.pmcs

Types

type PiquantParams

type PiquantParams struct {
	RunTimeEnv        string   `json:"runtimeEnv"`
	JobID             string   `json:"jobId"`
	JobsPath          string   `json:"jobsPath"`
	DatasetPath       string   `json:"datasetPath"`
	DetectorConfig    string   `json:"detectorConfig"`
	Elements          []string `json:"elements"`
	Parameters        string   `json:"parameters"`
	DatasetsBucket    string   `json:"datasetsBucket"`
	ConfigBucket      string   `json:"configBucket"`
	PiquantJobsBucket string   `json:"jobBucket"`
	QuantName         string   `json:"name"`
	PMCListName       string   `json:"pmcListName"`
	Command           string   `json:"command"`
}

PiquantParams - Parameters for running piquant, as generated by PIXLISE API

type QuantRunner

type QuantRunner interface {
	RunPiquant(piquantDockerImage string, params PiquantParams, pmcListNames []string, cfg config.APIConfig, requestorUserId string, log logger.ILogger) error
}

func GetQuantRunner

func GetQuantRunner(name string) (QuantRunner, error)