datapath

Documentation

Index

• type DataInfo
• type Edge
• type Environment
• type Node
• type ResolvedEdge
  • func (re *ResolvedEdge) String() string
• type Solution
  • func (solution *Solution) Reverse()

Types

type DataInfo

type DataInfo struct {
	// Source connection details
	DataDetails *datacatalog.GetAssetResponse
	// Pointer to the relevant data context in the Fybrik application spec
	Context *fappv1.DataContext
	// Evaluated config policies
	Configuration adminconfig.EvaluatorOutput
	// Workload cluster
	WorkloadCluster multicluster.Cluster
	// Required governance actions to perform on this asset
	Actions []taxonomy.Action
	// Potential actions to be taken on storing this asset in a specific location
	StorageRequirements map[taxonomy.ProcessingLocation][]taxonomy.Action
}

DataInfo defines all the information about the given data set that comes from the fybrikapplication spec and from the connectors.

type Edge

type Edge struct {
	Source          *Node
	Sink            *Node
	Module          *fappv1.FybrikModule
	CapabilityIndex int
}

Edge represents a module capability that gets data via source and returns data via sink interface

type Environment

type Environment struct {
	Modules          map[string]*fappv1.FybrikModule
	Clusters         []multicluster.Cluster
	StorageAccounts  []*fappv2.FybrikStorageAccount
	AttributeManager *infrastructure.AttributeManager
}

Environment defines the available resources (clusters, modules, storageAccounts) It also contains the results of queries to policy manager regarding writing data to storage accounts

type Node

type Node struct {
	Connection *taxonomy.Interface
	Virtual    bool
}

Node represents an access point to data (as a physical source/sink, or a virtual endpoint) A virtual endpoint is activated by the workload for read/write actions.

type ResolvedEdge

type ResolvedEdge struct {
	Edge
	Actions        []taxonomy.Action
	Cluster        string
	StorageAccount fappv2.FybrikStorageAccountSpec
}

ResolvedEdge extends an Edge by adding actions that a module should perform, and the cluster where the module will be deployed TODO(shlomitk1): add plugins/transformation capabilities to this structure

func (*ResolvedEdge) String

func (re *ResolvedEdge) String() string

type Solution

type Solution struct {
	DataPath []*ResolvedEdge
}

Solution is a final solution enabling a plotter construction. It represents a full data flow between the data source and the workload.

func (*Solution) Reverse

func (solution *Solution) Reverse()