models

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 8, 2025 License: GPL-3.0 Imports: 10 Imported by: 0

Documentation

Overview

Package models defines different models used in stats

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AdminUsers 

type AdminUsers struct {
	ID            int64  `json:"-"      sql:"id"              sqlitetype:"integer not null primary key"`
	Source        string `json:"source" sql:"source"          sqlitetype:"text"` // Source of admin users
	Users         List   `json:"users"  sql:"users"           sqlitetype:"text"` // List of users. In DB, users will be stored with | delimiter
	LastUpdatedAt string `json:"-"      sql:"last_updated_at" sqlitetype:"text"` // Last Updated time
}

AdminUsers from different sources.

func (AdminUsers) TableName 

func (AdminUsers) TableName() string

TableName returns the table which admin users list is stored into.

func (AdminUsers) TagMap 

func (a AdminUsers) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (AdminUsers) TagNames 

func (a AdminUsers) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type Allocation 

type Allocation = Generic

Allocation is a type alias to Generic that stores allocation data of compute units.

type CLIConfig 

type CLIConfig struct {
	Path    string            `yaml:"path"`
	EnvVars map[string]string `yaml:"environment_variables"`
}

CLIConfig contains the configuration of CLI client.

type Cluster 

type Cluster struct {
	ID       string    `json:"id"      sql:"cluster_id"       yaml:"id"`
	Manager  string    `json:"manager" sql:"resource_manager" yaml:"manager"`
	Web      WebConfig `json:"-"       yaml:"web"`
	CLI      CLIConfig `json:"-"       yaml:"cli"`
	Updaters []string  `json:"-"       yaml:"updaters"`
	Extra    yaml.Node `json:"-"       yaml:"extra_config"`
}

Cluster contains the configuration of the given resource manager.

type ClusterProjects 

type ClusterProjects struct {
	Cluster  Cluster
	Projects []Project
}

ClusterProjects is the container for the projects for a given cluster.

type ClusterUnits 

type ClusterUnits struct {
	Cluster Cluster
	Units   []Unit
}

ClusterUnits is the container for the units and config of a given cluster.

type ClusterUsers 

type ClusterUsers struct {
	Cluster Cluster
	Users   []User
}

ClusterUsers is the container for the users for a given cluster.

type DailyUsage added in v0.2.0 

type DailyUsage struct {
	Usage
}

DailyUsage statistics of each project/tenant/namespace.

func (DailyUsage) TableName added in v0.2.0 

func (DailyUsage) TableName() string

TableName returns the table which usage stats are stored into.

type Generic 

type Generic map[string]interface{}

Generic is map to store any mixed data types. Only string and int are supported. Any number will be converted into int64. Ref: https://go.dev/play/p/89ra6QgcZba, https://husobee.github.io/golang/database/2015/06/12/scanner-valuer.html, https://gist.github.com/jmoiron/6979540

func (*Generic) Scan 

func (g *Generic) Scan(v interface{}) error

Scan implements Scanner interface.

func (Generic) Value 

func (g Generic) Value() (driver.Value, error)

Value implements Valuer interface.

type JSONFloat 

type JSONFloat float64

JSONFloat is a custom float64 that can handle Inf and NaN during JSON (un)marshalling.

func (JSONFloat) MarshalJSON 

func (j JSONFloat) MarshalJSON() ([]byte, error)

MarshalJSON marshals JSONFloat into byte array The custom marshal interface will truncate the float64 to 8 decimals as storing all decimals will bring a very low added value and high DB storage.

func (*JSONFloat) Scan 

func (j *JSONFloat) Scan(v interface{}) error

Scan implements Scanner interface.

func (*JSONFloat) UnmarshalJSON 

func (j *JSONFloat) UnmarshalJSON(v []byte) error

UnmarshalJSON unmarshals byte array into JSONFloat.

func (JSONFloat) Value 

func (j JSONFloat) Value() (driver.Value, error)

Value implements Valuer interface.

type Key added in v0.2.0 

type Key struct {
	Name string `json:"name" sql:"name" sqlitetype:"text"` // Name of the metric key
}

Key represents arbritrary keys used in metric maps.

func (Key) TagMap added in v0.2.0 

func (k Key) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (Key) TagNames added in v0.2.0 

func (k Key) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type List 

type List []interface{}

List is a generic type to store slices. Only string and int slices are supported. Any number will be converted into int64.

func (*List) Scan 

func (l *List) Scan(v interface{}) error

Scan implements Scanner interface.

func (List) Value 

func (l List) Value() (driver.Value, error)

Value implements Valuer interface.

type MetricMap 

type MetricMap map[string]JSONFloat

MetricMap is a type alias to Generic that stores arbritrary metrics as a map.

func (*MetricMap) Scan 

func (m *MetricMap) Scan(v interface{}) error

Scan implements Scanner interface.

func (MetricMap) Value 

func (m MetricMap) Value() (driver.Value, error)

Value implements Valuer interface.

type Project 

type Project struct {
	ID              int64  `json:"-"                sql:"id"               sqlitetype:"integer not null primary key"`
	UID             string `json:"uid,omitempty"    sql:"uid"              sqlitetype:"text"` // Unique identifier of the project provided by cluster
	ClusterID       string `json:"cluster_id"       sql:"cluster_id"       sqlitetype:"text"` // Identifier of the resource manager that owns project. It is used to differentiate multiple clusters of same resource manager.
	ResourceManager string `json:"resource_manager" sql:"resource_manager" sqlitetype:"text"` // Name of the resource manager that owns project. Eg slurm, openstack, kubernetes, etc
	Name            string `json:"name"             sql:"name"             sqlitetype:"text"` // Name of the project
	Users           List   `json:"users"            sql:"users"            sqlitetype:"text"` // List of users of the project
	Tags            List   `json:"tags,omitempty"   sql:"tags"             sqlitetype:"text"` // List of meta data tags of the project
	LastUpdatedAt   string `json:"-"                sql:"last_updated_at"  sqlitetype:"text"` // Last Updated time
}

Project is the container for a given account/tenant/namespace of cluster.

func (Project) TableName 

func (Project) TableName() string

TableName returns the table which admin users list is stored into.

func (Project) TagMap 

func (p Project) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (Project) TagNames 

func (p Project) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type Stat added in v0.2.0 

type Stat struct {
	ClusterID        string `json:"cluster_id"         sql:"cluster_id"         sqlitetype:"text"`    // Identifier of the resource manager that owns compute unit. It is used to differentiate multiple clusters of same resource manager.
	ResourceManager  string `json:"resource_manager"   sql:"resource_manager"   sqlitetype:"text"`    // Name of the resource manager that owns project. Eg slurm, openstack, kubernetes, etc
	NumUnits         int64  `json:"num_units"          sql:"num_units"          sqlitetype:"integer"` // Number of active and terminated units
	NumInActiveUnits int64  `json:"num_inactive_units" sql:"num_inactive_units" sqlitetype:"integer"` // Number of inactive units that are in terminated/cancelled/error state
	NumActiveUnits   int64  `json:"num_active_units"   sql:"num_active_units"   sqlitetype:"integer"` // Number of active units that are in running state
	NumProjects      int64  `json:"num_projects"       sql:"num_projects"       sqlitetype:"integer"` // Number of projects
	NumUsers         int64  `json:"num_users"          sql:"num_users"          sqlitetype:"integer"` // Number of users
}

Stat represents high level statistics of each cluster.

func (Stat) TagMap added in v0.2.0 

func (s Stat) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (Stat) TagNames added in v0.2.0 

func (s Stat) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type Tag 

type Tag = Generic

Tag is a type alias to Generic that stores metadata of compute units.

type Unit 

type Unit struct {
	ID                  int64      `json:"-"                                    sql:"id"                         sqlitetype:"integer not null primary key"`
	ClusterID           string     `json:"cluster_id,omitempty"                 sql:"cluster_id"                 sqlitetype:"text"`    // Identifier of the resource manager that owns compute unit. It is used to differentiate multiple clusters of same resource manager.
	ResourceManager     string     `json:"resource_manager,omitempty"           sql:"resource_manager"           sqlitetype:"text"`    // Name of the resource manager that owns compute unit. Eg slurm, openstack, kubernetes, etc
	UUID                string     `json:"uuid"                                 sql:"uuid"                       sqlitetype:"text"`    // Unique identifier of unit. It can be Job ID for batch jobs, UUID for pods in k8s or VMs in Openstack
	Name                string     `json:"name,omitempty"                       sql:"name"                       sqlitetype:"text"`    // Name of compute unit
	Project             string     `json:"project,omitempty"                    sql:"project"                    sqlitetype:"text"`    // Account in batch systems, Tenant in Openstack, Namespace in k8s
	Group               string     `json:"groupname,omitempty"                  sql:"groupname"                  sqlitetype:"text"`    // User group
	User                string     `json:"username,omitempty"                   sql:"username"                   sqlitetype:"text"`    // Username
	CreatedAt           string     `json:"created_at,omitempty"                 sql:"created_at"                 sqlitetype:"text"`    // Creation time
	StartedAt           string     `json:"started_at,omitempty"                 sql:"started_at"                 sqlitetype:"text"`    // Start time
	EndedAt             string     `json:"ended_at,omitempty"                   sql:"ended_at"                   sqlitetype:"text"`    // End time
	CreatedAtTS         int64      `json:"created_at_ts,omitempty"              sql:"created_at_ts"              sqlitetype:"integer"` // Creation timestamp
	StartedAtTS         int64      `json:"started_at_ts,omitempty"              sql:"started_at_ts"              sqlitetype:"integer"` // Start timestamp
	EndedAtTS           int64      `json:"ended_at_ts,omitempty"                sql:"ended_at_ts"                sqlitetype:"integer"` // End timestamp
	Elapsed             string     `json:"elapsed,omitempty"                    sql:"elapsed"                    sqlitetype:"text"`    // Human readable total elapsed time string
	State               string     `json:"state,omitempty"                      sql:"state"                      sqlitetype:"text"`    // Current state of unit
	Allocation          Allocation `json:"allocation,omitempty"                 sql:"allocation"                 sqlitetype:"text"`    // Allocation map of unit. Only string and int64 values are supported in map
	TotalTime           MetricMap  `json:"total_time_seconds,omitempty"         sql:"total_time_seconds"         sqlitetype:"text"`    // Different types of times in seconds consumed by the unit. This map contains at minimum `walltime`, `alloc_cputime`, `alloc_cpumemtime`, `alloc_gputime` and `alloc_gpumem_time` keys.
	AveCPUUsage         MetricMap  `json:"avg_cpu_usage,omitempty"              sql:"avg_cpu_usage"              sqlitetype:"text"`    // Average CPU usage(s) during lifetime of unit
	AveCPUMemUsage      MetricMap  `json:"avg_cpu_mem_usage,omitempty"          sql:"avg_cpu_mem_usage"          sqlitetype:"text"`    // Average CPU memory usage(s) during lifetime of unit
	TotalCPUEnergyUsage MetricMap  `json:"total_cpu_energy_usage_kwh,omitempty" sql:"total_cpu_energy_usage_kwh" sqlitetype:"text"`    // Total CPU energy usage(s) in kWh during lifetime of unit
	TotalCPUEmissions   MetricMap  `json:"total_cpu_emissions_gms,omitempty"    sql:"total_cpu_emissions_gms"    sqlitetype:"text"`    // Total CPU emissions from source(s) in grams during lifetime of unit
	AveGPUUsage         MetricMap  `json:"avg_gpu_usage,omitempty"              sql:"avg_gpu_usage"              sqlitetype:"text"`    // Average GPU usage(s) during lifetime of unit
	AveGPUMemUsage      MetricMap  `json:"avg_gpu_mem_usage,omitempty"          sql:"avg_gpu_mem_usage"          sqlitetype:"text"`    // Average GPU memory usage(s) during lifetime of unit
	TotalGPUEnergyUsage MetricMap  `json:"total_gpu_energy_usage_kwh,omitempty" sql:"total_gpu_energy_usage_kwh" sqlitetype:"text"`    // Total GPU energy usage(s) in kWh during lifetime of unit
	TotalGPUEmissions   MetricMap  `json:"total_gpu_emissions_gms,omitempty"    sql:"total_gpu_emissions_gms"    sqlitetype:"text"`    // Total GPU emissions from source(s) in grams during lifetime of unit
	TotalIOWriteStats   MetricMap  `json:"total_io_write_stats,omitempty"       sql:"total_io_write_stats"       sqlitetype:"text"`    // Total IO write statistics during lifetime of unit
	TotalIOReadStats    MetricMap  `json:"total_io_read_stats,omitempty"        sql:"total_io_read_stats"        sqlitetype:"text"`    // Total IO read statistics GB during lifetime of unit
	TotalIngressStats   MetricMap  `json:"total_ingress_stats,omitempty"        sql:"total_ingress_stats"        sqlitetype:"text"`    // Total Ingress statistics of unit
	TotalOutgressStats  MetricMap  `json:"total_outgress_stats,omitempty"       sql:"total_outgress_stats"       sqlitetype:"text"`    // Total Outgress statistics of unit
	Tags                Tag        `json:"tags,omitempty"                       sql:"tags"                       sqlitetype:"text"`    // A map to store generic info. String and int64 are valid value types of map
	Ignore              int        `json:"-"                                    sql:"ignore"                     sqlitetype:"integer"` // Whether to ignore unit
	NumUpdates          int64      `json:"-"                                    sql:"num_updates"                sqlitetype:"integer"` // Number of updates. This is used internally to update aggregate metrics
	LastUpdatedAt       string     `json:"-"                                    sql:"last_updated_at"            sqlitetype:"text"`    // Last updated time. It can be used to clean up DB
}

Unit is an abstract compute unit that can mean Job (batchjobs), VM (cloud) or Pod (k8s).

func (Unit) TableName 

func (Unit) TableName() string

TableName returns the table which units are stored into.

func (Unit) TagMap 

func (u Unit) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (Unit) TagNames 

func (u Unit) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type Usage 

type Usage struct {
	ID                  int64     `json:"-"                                    sql:"id"                         sqlitetype:"integer not null primary key"`
	ClusterID           string    `json:"cluster_id"                           sql:"cluster_id"                 sqlitetype:"text"`    // Identifier of the resource manager that owns compute unit. It is used to differentiate multiple clusters of same resource manager.
	ResourceManager     string    `json:"resource_manager"                     sql:"resource_manager"           sqlitetype:"text"`    // Name of the resource manager that owns project. Eg slurm, openstack, kubernetes, etc
	NumUnits            int64     `json:"num_units"                            sql:"num_units"                  sqlitetype:"integer"` // Number of consumed units
	Project             string    `json:"project"                              sql:"project"                    sqlitetype:"text"`    // Account in batch systems, Tenant in Openstack, Namespace in k8s
	Group               string    `json:"groupname"                            sql:"groupname"                  sqlitetype:"text"`    // User group
	User                string    `json:"username"                             sql:"username"                   sqlitetype:"text"`    // Username
	LastUpdatedAt       string    `json:"-"                                    sql:"last_updated_at"            sqlitetype:"text"`    // Last updated time. It can be used to clean up DB
	TotalTime           MetricMap `json:"total_time_seconds,omitempty"         sql:"total_time_seconds"         sqlitetype:"text"`    // Different times in seconds consumed by the unit. This map must contain `walltime`, `alloc_cputime`, `alloc_cpumemtime`, `alloc_gputime` and `alloc_gpumem_time` keys.
	AveCPUUsage         MetricMap `json:"avg_cpu_usage,omitempty"              sql:"avg_cpu_usage"              sqlitetype:"text"`    // Average CPU usage(s) during lifetime of project
	AveCPUMemUsage      MetricMap `json:"avg_cpu_mem_usage,omitempty"          sql:"avg_cpu_mem_usage"          sqlitetype:"text"`    // Average CPU memory usage(s) during lifetime of project
	TotalCPUEnergyUsage MetricMap `json:"total_cpu_energy_usage_kwh,omitempty" sql:"total_cpu_energy_usage_kwh" sqlitetype:"text"`    // Total CPU energy usage(s) in kWh during lifetime of project
	TotalCPUEmissions   MetricMap `json:"total_cpu_emissions_gms,omitempty"    sql:"total_cpu_emissions_gms"    sqlitetype:"text"`    // Total CPU emissions from source(s) in grams during lifetime of project
	AveGPUUsage         MetricMap `json:"avg_gpu_usage,omitempty"              sql:"avg_gpu_usage"              sqlitetype:"text"`    // Average GPU usage(s) during lifetime of project
	AveGPUMemUsage      MetricMap `json:"avg_gpu_mem_usage,omitempty"          sql:"avg_gpu_mem_usage"          sqlitetype:"text"`    // Average GPU memory usage(s) during lifetime of project
	TotalGPUEnergyUsage MetricMap `json:"total_gpu_energy_usage_kwh,omitempty" sql:"total_gpu_energy_usage_kwh" sqlitetype:"text"`    // Total GPU energy usage(s) in kWh during lifetime of project
	TotalGPUEmissions   MetricMap `json:"total_gpu_emissions_gms,omitempty"    sql:"total_gpu_emissions_gms"    sqlitetype:"text"`    // Total GPU emissions from source(s) in grams during lifetime of project
	TotalIOWriteStats   MetricMap `json:"total_io_write_stats,omitempty"       sql:"total_io_write_stats"       sqlitetype:"text"`    // Total IO write statistics during lifetime of unit
	TotalIOReadStats    MetricMap `json:"total_io_read_stats,omitempty"        sql:"total_io_read_stats"        sqlitetype:"text"`    // Total IO read statistics GB during lifetime of unit
	TotalIngressStats   MetricMap `json:"total_ingress_stats,omitempty"        sql:"total_ingress_stats"        sqlitetype:"text"`    // Total Ingress statistics of unit
	TotalOutgressStats  MetricMap `json:"total_outgress_stats,omitempty"       sql:"total_outgress_stats"       sqlitetype:"text"`    // Total Outgress statistics of unit
	NumUpdates          int64     `json:"-"                                    sql:"num_updates"                sqlitetype:"text"`    // Number of updates. This is used internally to update aggregate metrics
}

Usage statistics of each project/tenant/namespace.

func (Usage) TableName 

func (Usage) TableName() string

TableName returns the table which usage stats are stored into.

func (Usage) TagMap 

func (u Usage) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (Usage) TagNames 

func (u Usage) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type User 

type User struct {
	ID              int64  `json:"-"                sql:"id"               sqlitetype:"integer not null primary key"`
	UID             string `json:"uid,omitempty"    sql:"uid"              sqlitetype:"text"` // Unique identifier of the user provided by cluster
	ClusterID       string `json:"cluster_id"       sql:"cluster_id"       sqlitetype:"text"` // Identifier of the resource manager that owns user. It is used to differentiate multiple clusters of same resource manager.
	ResourceManager string `json:"resource_manager" sql:"resource_manager" sqlitetype:"text"` // Name of the resource manager that owns user. Eg slurm, openstack, kubernetes, etc
	Name            string `json:"name"             sql:"name"             sqlitetype:"text"` // Name of the user
	Projects        List   `json:"projects"         sql:"projects"         sqlitetype:"text"` // List of projects of the user
	Tags            List   `json:"tags,omitempty"   sql:"tags"             sqlitetype:"text"` // List of meta data tags of the user
	LastUpdatedAt   string `json:"-"                sql:"last_updated_at"  sqlitetype:"text"` // Last Updated time
}

User is the container for a given user of cluster.

func (User) TableName 

func (User) TableName() string

TableName returns the table which admin users list is stored into.

func (User) TagMap 

func (u User) TagMap(keyTag string, valueTag string) map[string]string

TagMap returns a map of tags based on keyTag and valueTag. If keyTag is empty, field names are used as map keys.

func (User) TagNames 

func (u User) TagNames(tag string) []string

TagNames returns a slice of all tag names.

type WebConfig 

type WebConfig struct {
	URL              string                  `yaml:"url"`
	HTTPClientConfig config.HTTPClientConfig `yaml:",inline"`
}

WebConfig contains the client related configuration of a REST API server.

func (*WebConfig) SetDirectory 

func (c *WebConfig) SetDirectory(dir string)

SetDirectory joins any relative file paths with dir.

func (*WebConfig) UnmarshalYAML 

func (c *WebConfig) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements the yaml.Unmarshaler interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.