memory

package
v0.0.0-...-8f1fe78 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 12, 2024 License: MIT Imports: 25 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewResource 

func NewResource(node jgf.Node) *types.Resource

Generate a new resource from a JGF node A resource is associated with a dominant subsystem resource

func NewSubsystemResource 

func NewSubsystemResource(node jgf.Node) *types.Resource

New SubsystemResource creates a resource, but also adds arbitrary metadata

Types

type ClusterGraph 

type ClusterGraph struct {
	State map[string]interface{}

	// Courtesy holder for name
	Name string
	// contains filtered or unexported fields
}

A ClusterGraph holds a single graph with one or more subsystems

func NewClusterGraph 

func NewClusterGraph(name string, domSubsystem string) *ClusterGraph

NewClusterGraph creates a new cluster graph with a dominant subsystem We assume the dominant is hard coded to be containment

func (*ClusterGraph) DFSForMatch 

func (g *ClusterGraph) DFSForMatch(
	jobspec *v1.Jobspec,
	matcher algorithm.MatchAlgorithm,
) (bool, error)

DFSForMatch WILL is a depth first search if the cluter matches It starts by looking at total cluster resources on the top level, and then traverses into those that match the first check THIS IS EXPERIMENTAL and likely wrong, or missing details, which is OK as we will only be using it for prototyping.

func (*ClusterGraph) DominantSubsystem 

func (c *ClusterGraph) DominantSubsystem() *Subsystem

Dominant subsystem gets the dominant subsystem

func (*ClusterGraph) GetMetrics 

func (g *ClusterGraph) GetMetrics(subsystem string) Metrics

GetMetrics for a named subsystem, defaulting to dominant

func (*ClusterGraph) GetState 

func (c *ClusterGraph) GetState() types.ClusterState

GetState of the cluster We could expose this as a public variable, but I'm leaving like this in case we want to do additional processing (for example, maybe some attributes are private)

func (*ClusterGraph) LoadClusterNodes 

func (g *ClusterGraph) LoadClusterNodes(
	nodes *jgf.JsonGraph,
	subsystem string,
) error

func (*ClusterGraph) LoadSubsystemNodes 

func (g *ClusterGraph) LoadSubsystemNodes(
	nodes *jgf.JsonGraph,
	subsystem string,
) error

LoadSubsystemNodes into the cluster

type Graph 

type Graph struct {
	Clusters map[string]*ClusterGraph
	// contains filtered or unexported fields
}

A graph holds one or more named clusters

func NewGraph 

func NewGraph() *Graph

NewGraph creates a structure that holds one or more graphs

func (*Graph) Close 

func (g *Graph) Close() error

Close the database and save to backup file

func (*Graph) GetStates 

func (g *Graph) GetStates(names []string) (map[string]types.ClusterState, error)

GetStates for clusters in the graph

func (*Graph) LoadBackup 

func (g *Graph) LoadBackup() error

LoadBackup loads the saved database from a backup

func (*Graph) LoadClusterNodes 

func (g *Graph) LoadClusterNodes(
	clusterName string,
	nodes *jgf.JsonGraph,
	subsystem string,
) error

LoadClusterNodes loads a new cluster into the graph

func (*Graph) LoadSubsystemNodes 

func (g *Graph) LoadSubsystemNodes(
	clusterName string,
	nodes *jgf.JsonGraph,
	subsystem string,
) error

LoadSubsystemNodes into the graph For addition, we can have a two way pointer from the subsystem node TO the dominant node and then back: The pointer TO the dominant subsystem let's us find it to delete the opposing one The other one is used during the search to find the subsystem node

func (*Graph) RegisterCluster 

func (g *Graph) RegisterCluster(
	name string,
	payload string,
	subsystem string,
) (*service.Response, error)

Register cluster should: 1. Load in json graph of nodes from string 2. Add nodes to the graph, also keep top level metrics? 3. Return corresponding response

func (*Graph) Satisfies 

func (g *Graph) Satisfies(
	payload string,
	matcher algorithm.MatchAlgorithm,
) (*service.SatisfyResponse, error)

Satisfy should: 1. Read in and populate the payload into a jobspec 2. Determine by way of a depth first search if we can satisfy 3. Return the names of the cluster

func (*Graph) UpdateState 

func (g *Graph) UpdateState(name string, state *types.ClusterState) error

UpdateState updates the state of a known cluster in the graph

type MemoryGraph 

type MemoryGraph struct{}

func (MemoryGraph) AddCluster 

func (m MemoryGraph) AddCluster(
	name string,
	nodes *jgf.JsonGraph,
	subsystem string,
) error

AddCluster adds a JGF graph of new nodes Note that a client can interact with the database (in read only) but since this is directly in the rainbow cluster, we call the functions directly. The "addCluster" here is referring to the dominant subsystem, while a "subsystem" below is considered supplementary to that.

func (MemoryGraph) AddSubsystem 

func (m MemoryGraph) AddSubsystem(
	name string,
	nodes *jgf.JsonGraph,
	subsystem string,
) error

UpdateState of a cluster in the graph. This is used for selection algorithms

func (MemoryGraph) Description 

func (m MemoryGraph) Description() string

func (MemoryGraph) GetStates 

func (m MemoryGraph) GetStates(names []string) (map[string]types.ClusterState, error)

GetStates for a list of clusters

func (MemoryGraph) Init 

func (g MemoryGraph) Init(
	options map[string]string,
) error

Init provides extra initialization functionality, if needed The in memory database can take a backup file if desired

func (MemoryGraph) Name 

func (m MemoryGraph) Name() string

func (MemoryGraph) RegisterService 

func (m MemoryGraph) RegisterService(s *grpc.Server) error

func (MemoryGraph) Satisfies 

func (g MemoryGraph) Satisfies(
	jobspec *js.Jobspec,
	matcher algorithm.MatchAlgorithm,
) ([]string, error)

Satisfies - determine what clusters satisfy a jobspec request Since this is called from the client function, it's technically running from the client (not from the server)

func (MemoryGraph) UpdateState 

func (m MemoryGraph) UpdateState(
	name string,
	payload string,
) error

Add subsystem adds a new subsystem to the graph!

type MemoryServer 

type MemoryServer struct {
	service.UnimplementedMemoryGraphServer
}

func (MemoryServer) Register 

func (MemoryServer) Register(c context.Context, req *service.RegisterRequest) (*service.Response, error)

Register takes a cluster node payload and adds to the in memory graph

func (MemoryServer) Satisfy 

func (MemoryServer) Satisfy(c context.Context, req *service.SatisfyRequest) (*service.SatisfyResponse, error)

Satisfy determines if the graph can satisfy a request

type Metrics 

type Metrics struct {
	// This is across all subsystems
	Vertices int   `json:"vertices"`
	Writes   int64 `json:"writes"`
	Reads    int64 `json:"reads"`

	// Courtesy to print the subsystem name
	Name string `json:"name"`

	// Resource specific metrics
	ResourceCounts map[string]int64
}

Metrics keeps track of counts of things

func (*Metrics) CountResource 

func (m *Metrics) CountResource(resourceType string)

CountResource under some named top level, usually a root-> cluster (the cluster)

func (*Metrics) IncReadCount 

func (m *Metrics) IncReadCount() *Metrics

func (*Metrics) IncWriteCount 

func (m *Metrics) IncWriteCount() *Metrics

func (*Metrics) NewResource 

func (m *Metrics) NewResource(levelName string)

NewResource resets the resource counters for a cluster

func (*Metrics) Show 

func (m *Metrics) Show() error

Show prints a summary of resources for an entire subsystem

type Subsystem 

type Subsystem struct {

	// Name of the subsystem
	Name string

	// Using a map means O(1) lookup time
	Vertices map[int]*types.Vertex `json:"vertices"`

	// There are a small number of vertices we care to lookup by name
	// Put them here for now until I have a better idea :)
	Lookup map[string]int

	// Subsystem level metrics
	Metrics Metrics
	// contains filtered or unexported fields
}

A subsystem is a graph with a set of vertices that are connected by edges We use "vertex" instead of node to distinguish the graph vs. a compute note

func NewSubsystem 

func NewSubsystem(name string) *Subsystem

NewSubsystem generates a new subsystem graph

func (*Subsystem) AddInternalEdge 

func (s *Subsystem) AddInternalEdge(
	src, dest, weight int,
	relation,
	subsystem string,
) error

Add an edge to the graph with a source and dest identifier This assumes they belong in the same subsystem (src subsystem == dest subsystem) Optionally add a weight. We aren't using this (but I think might)

func (*Subsystem) AddNode 

func (s *Subsystem) AddNode(
	lookupName, typ string,
	size int32,
	unit string,
	meta metadata.Metadata,
	countResource bool,
) int

AddNode (a physical node) as a vertex, return the vertex id

func (*Subsystem) AddSubsystemEdge 

func (s *Subsystem) AddSubsystemEdge(
	src int,
	dest *types.Vertex,
	weight int,
	relation string,
	subsystem string,
) error

Add an subsystem edge, meaning adding the edge AND a link to the dominant subsystem This would be called by the dominant to add an edge to itself

func (*Subsystem) CountVertices 

func (s *Subsystem) CountVertices() int

func (*Subsystem) GetConnections 

func (s *Subsystem) GetConnections(src int) []int

GetConnections to a vertex

func (*Subsystem) GetNode 

func (s *Subsystem) GetNode(name string) (int, bool)

GetNode returns the vertex if of a node, if it exists in the lookup

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.