pipeline

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2023 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ClusterRoleBindingIngestName = "k8s-cluster-role-binding-ingest"
)
View Source 
const (
	ClusterRoleIngestName = "k8s-cluster-role-ingest"
)
View Source 
const (
	EndpointIngestName = "k8s-endpoint-ingest"
)
View Source 
const (
	NodeIngestName = "k8s-node-ingest"
)
View Source 
const (
	PodIngestName = "k8s-pod-ingest"
)
View Source 
const (
	RoleBindingIngestName = "k8s-role-binding-ingest"
)
View Source 
const (
	RoleIngestName = "k8s-role-ingest"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type CleanupFunc 

type CleanupFunc func(ctx context.Context) error

CleanupFunc is a callback to be registered in the cleanup array.

type ClusterRoleBindingIngest 

type ClusterRoleBindingIngest struct {
	// contains filtered or unexported fields
}

func (*ClusterRoleBindingIngest) Close 

func (i *ClusterRoleBindingIngest) Close(ctx context.Context) error

func (*ClusterRoleBindingIngest) Complete 

func (i *ClusterRoleBindingIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all roles have been streamed. The function flushes all writers and waits for completion.

func (*ClusterRoleBindingIngest) IngestClusterRoleBinding 

func (i *ClusterRoleBindingIngest) IngestClusterRoleBinding(ctx context.Context, crb types.ClusterRoleBindingType) error

streamCallback is invoked by the collector for each cluster role binding collected. The function ingests an input cluster role binding object into the store/graph and then ingests all child objects (identites, etc) through their own ingestion pipeline.

func (*ClusterRoleBindingIngest) Initialize 

func (i *ClusterRoleBindingIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*ClusterRoleBindingIngest) Name 

func (i *ClusterRoleBindingIngest) Name() string

func (*ClusterRoleBindingIngest) Run 

func (i *ClusterRoleBindingIngest) Run(ctx context.Context) error

type ClusterRoleIngest 

type ClusterRoleIngest struct {
	// contains filtered or unexported fields
}

func (*ClusterRoleIngest) Close 

func (i *ClusterRoleIngest) Close(ctx context.Context) error

func (*ClusterRoleIngest) Complete 

func (i *ClusterRoleIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all cluster roles have been streamed. The function flushes all writers and waits for completion.

func (*ClusterRoleIngest) IngestClusterRole 

func (i *ClusterRoleIngest) IngestClusterRole(ctx context.Context, role types.ClusterRoleType) error

streamCallback is invoked by the collector for each cluster role collected. The function ingests an input cluster role into the cache/store/graph databases asynchronously.

func (*ClusterRoleIngest) Initialize 

func (i *ClusterRoleIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*ClusterRoleIngest) Name 

func (i *ClusterRoleIngest) Name() string

func (*ClusterRoleIngest) Run 

func (i *ClusterRoleIngest) Run(ctx context.Context) error

type Dependencies 

type Dependencies struct {
	Config    *config.KubehoundConfig
	Collector collector.CollectorClient
	Cache     cache.CacheProvider
	StoreDB   storedb.Provider
	GraphDB   graphdb.Provider
}

Dependencies encapsulates all of the ingest pipeline dependencies (initialized).

type EndpointIngest 

type EndpointIngest struct {
	// contains filtered or unexported fields
}

func (*EndpointIngest) Close 

func (i *EndpointIngest) Close(ctx context.Context) error

func (*EndpointIngest) Complete 

func (i *EndpointIngest) Complete(ctx context.Context) error

Complete is invoked by the collector when all nodes have been streamed. The function flushes all writers and waits for completion.

func (*EndpointIngest) IngestEndpoint 

func (i *EndpointIngest) IngestEndpoint(ctx context.Context, eps types.EndpointType) error

IngestEndpoint is invoked by the collector for each endpoint slice collected. The function ingests an input endpoint slice into the cache/store/graph databases asynchronously.

func (*EndpointIngest) Initialize 

func (i *EndpointIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*EndpointIngest) Name 

func (i *EndpointIngest) Name() string

func (*EndpointIngest) Run 

func (i *EndpointIngest) Run(ctx context.Context) error

type FlushFunc 

type FlushFunc func(ctx context.Context) error

FlushFunc is a callback to be registered in the flush array.

type Group 

type Group struct {
	Name    string         // Name of the ingest group
	Ingests []ObjectIngest // Parallelized object ingest pipelines
}

Group encapsulates a collection of object ingest pipelines that can be run in parallel.s

func (*Group) Run 

func (g *Group) Run(outer context.Context, deps *Dependencies) error

Run executes all the object ingest pipelines in parallel and returns when all complete.

type IngestResourceOption 

type IngestResourceOption func(ctx context.Context, oic *resourceOptions, deps *Dependencies) error

IngestResourceOption enables options to be passed to the pipeline initializer.

func WithCacheReader 

func WithCacheReader() IngestResourceOption

WithCacheReader initializes a cache reader (and registers a cleanup function to close the connection) for the ingest pipeline.

func WithCacheWriter 

func WithCacheWriter(opts ...cache.WriterOption) IngestResourceOption

WithCacheWriter initializes a cache writer (and registers a cleanup function) for the ingest pipeline.

func WithConverterCache 

func WithConverterCache() IngestResourceOption

WithCacheWriter initializes a store converter with cache access for the ingest pipeline.

func WithGraphWriter 

func WithGraphWriter(v vertex.Builder) IngestResourceOption

WithStoreWriter initializes a bulk graph writer (and registers a cleanup function) for the provided vertex. To access the writer use the graphWriter(v vertex.Vertex) function.

func WithStoreWriter 

func WithStoreWriter[T collections.Collection](c T) IngestResourceOption

WithStoreWriter initializes a bulk store writer (and registers a cleanup function) for the provided collection. To access the writer use the storeWriter(c collections.Collection) function.

type IngestResources 

type IngestResources struct {
	// contains filtered or unexported fields
}

IngestResources provides the base functionality (service initialization, flush and cleanup) for any object ingest pipeline.

func CreateResources 

func CreateResources(ctx context.Context, deps *Dependencies, opts ...IngestResourceOption) (*IngestResources, error)

CreateResources handles the base initialization of service dependencies for an object ingest pipeline. This should be called from the ObjectIngest::Initialize function.

type NodeIngest 

type NodeIngest struct {
	// contains filtered or unexported fields
}

func (*NodeIngest) Close 

func (i *NodeIngest) Close(ctx context.Context) error

func (*NodeIngest) Complete 

func (i *NodeIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all nodes have been streamed. The function flushes all writers and waits for completion.

func (*NodeIngest) IngestNode 

func (i *NodeIngest) IngestNode(ctx context.Context, node types.NodeType) error

streamCallback is invoked by the collector for each node collected. The function ingests an input node into the cache/store/graph databases asynchronously.

func (*NodeIngest) Initialize 

func (i *NodeIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*NodeIngest) Name 

func (i *NodeIngest) Name() string

func (*NodeIngest) Run 

func (i *NodeIngest) Run(ctx context.Context) error

type ObjectIngest 

type ObjectIngest interface {
	// Name returns the name of the object ingest pipeline.
	Name() string

	// Initialize intializes an object ingest pipeline with any servic dependencies.
	Initialize(ctx context.Context, deps *Dependencies) error

	// Run executes the ingest pipeline, returning when all are complete.
	Run(ctx context.Context) error

	// Close cleans up any resources held in the ingest pipeline on completion/error.
	Close(ctx context.Context) error
}

ObjectIngest represents an ingestion pipeline that receives an input object from a collector implementation, processes and persists all resulting KubeHound objects (store models, cache entries, graph vertices, etc).

type PodIngest 

type PodIngest struct {
	// contains filtered or unexported fields
}

func (*PodIngest) Close 

func (i *PodIngest) Close(ctx context.Context) error

func (*PodIngest) Complete 

func (i *PodIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all pods have been streamed. The function flushes all writers and waits for completion.

func (*PodIngest) IngestPod 

func (i *PodIngest) IngestPod(ctx context.Context, pod types.PodType) error

streamCallback is invoked by the collector for each pod collected. The function ingests an input pod object into the cache/store/graph and then ingests all child objects (containers, volumes, etc) through their own ingestion pipeline.

func (*PodIngest) Initialize 

func (i *PodIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*PodIngest) Name 

func (i *PodIngest) Name() string

func (*PodIngest) Run 

func (i *PodIngest) Run(ctx context.Context) error

type RoleBindingIngest 

type RoleBindingIngest struct {
	// contains filtered or unexported fields
}

func (*RoleBindingIngest) Close 

func (i *RoleBindingIngest) Close(ctx context.Context) error

func (*RoleBindingIngest) Complete 

func (i *RoleBindingIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all roles have been streamed. The function flushes all writers and waits for completion.

func (*RoleBindingIngest) IngestRoleBinding 

func (i *RoleBindingIngest) IngestRoleBinding(ctx context.Context, rb types.RoleBindingType) error

streamCallback is invoked by the collector for each role binding collected. The function ingests an input role binding object into the store/graph and then ingests all child objects (identites, etc) through their own ingestion pipeline.

func (*RoleBindingIngest) Initialize 

func (i *RoleBindingIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*RoleBindingIngest) Name 

func (i *RoleBindingIngest) Name() string

func (*RoleBindingIngest) Run 

func (i *RoleBindingIngest) Run(ctx context.Context) error

type RoleIngest 

type RoleIngest struct {
	// contains filtered or unexported fields
}

func (*RoleIngest) Close 

func (i *RoleIngest) Close(ctx context.Context) error

func (*RoleIngest) Complete 

func (i *RoleIngest) Complete(ctx context.Context) error

completeCallback is invoked by the collector when all roles have been streamed. The function flushes all writers and waits for completion.

func (*RoleIngest) IngestRole 

func (i *RoleIngest) IngestRole(ctx context.Context, role types.RoleType) error

streamCallback is invoked by the collector for each role collected. The function ingests an input role into the cache/store/graph databases asynchronously.

func (*RoleIngest) Initialize 

func (i *RoleIngest) Initialize(ctx context.Context, deps *Dependencies) error

func (*RoleIngest) Name 

func (i *RoleIngest) Name() string

func (*RoleIngest) Run 

func (i *RoleIngest) Run(ctx context.Context) error

type Sequence 

type Sequence struct {
	Name   string
	Groups []Group
}

Sequence encapsulates a set of ingest pipeline groups that must be executed sequentially.

func (*Sequence) Run 

func (s *Sequence) Run(ctx context.Context, deps *Dependencies) error

Run executes all the pipeline groups in sequence and returns when all complete.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.