cnfreg

README

CNF Registry Plugin

CNF Registry plugin allows to load a CNF module into the StoneWork (all-in-one VPP distribution; SW for short) during the Init phase. CNF can be built as another image and run as a separate container. This allows to enable/disable CNF without having to rebuild StoneWork docker image or the StoneWork agent binary.

Apart from single common VPP it is also possible to share network namespace between all/some CNFs and therefore integrate different network functions inside the Linux network stack (e.g. to use OSPF-learned routes to connect with a BGP peer).

The plugin operates in one of the 3 following modes depending on the value of the CNF_MODE environment variable:

1. STANDALONE (default, i.e. assumed if the variable is not defined):
   • CNF is used on its own, potentially chained with other CNFs using for example NSM (i.e. each VPP-based CNF runs its own VPP instance)
   • The CNF Registry plugin is also used by a Standalone CNF, but merely to keep track of CNF Index ID.
2. STONEWORK_MODULE:
   • CNF used as a SW-module
   • VPP-based CNFs do not run VPP inside their container, instead they connect with the all-in-one VPP of StoneWork
   • in this mode the Registry acts as a client of the Registry running by the StoneWork agent
   • internally the plugin uses gRPC to exchange all the information needed between the Registries of CNF and SW to load the CNF and use with the all-in-one VPP
   • CNF should use only those methods of the plugin which are defined by the CnfAPI interface
3. STONEWORK:
   • CNF Registry plugin is used by StoneWork firstly to discover all the enabled CNFs and then to collect all the information about them to be able to integrate them with the all-in-one VPP
   • for each CNF, StoneWork needs to learn the NB-facing configuration models, traffic Punting to use (some subset of the traffic typically needs to be diverted from VPP into the Linux network stack via TAPs or directly into CNF using memifs for further processing)
   • StoneWork should use only those methods of the plugin which are defined by the StoneWorkAPI interface

Documentation

Index

• Constants
• Variables
• type CnfAPI
• type CnfDeps
• type CnfModelCallbacks
• type CnfRegistryAPI
• type Config
• type Deps
• type Info
• type ItemDepsClb
• type Option
• type PidFile
• type Plugin
  • func NewPlugin(opts ...Option) *Plugin
  • func (p *Plugin) AfterInit() (err error)
  • func (p *Plugin) Close() error
  • func (p *Plugin) DiscoverCnf(ctx context.Context, req *pb.DiscoverCnfReq) (resp *pb.DiscoverCnfResp, err error)
  • func (p *Plugin) GetCnfCfgClient(cnfMsLabel string) (cfgClient client.GenericClient, err error)
  • func (p *Plugin) GetCnfGrpcConn(cnfMsLabel string) (conn grpc.ClientConnInterface, err error)
  • func (p *Plugin) GetCnfMode() pb.CnfMode
  • func (p *Plugin) GetGrpcPort() (port int)
  • func (p *Plugin) GetHttpPort() (port int)
  • func (p *Plugin) GetItemDependencies(ctx context.Context, item *generic.Item) (itemDeps *pb.GetDependenciesResp, err error)
  • func (p *Plugin) GetPuntRequests(ctx context.Context, item *generic.Item) (puntReqs *puntmgr.PuntRequests, err error)
  • func (p *Plugin) GetSWCfgClient() (cfgClient client.GenericClient, err error)
  • func (p *Plugin) GetSWGrpcConn() (conn grpc.ClientConnInterface, err error)
  • func (p *Plugin) Init() (err error)
  • func (p *Plugin) RegisterCnfModel(model models.KnownModel, descriptor *kvs.KVDescriptor, ...) error
• type PuntManagerAPI
• type PuntRequestsClb
• type StoneWorkAPI

Constants

const (
	// PluginName is the name of the CNF Registry Plugin.
	PluginName = "cnfreg"

	// Environment variable that is used to select the CNF mode.
	CnfModeEnvVar = "CNF_MODE"

	// Environment variable that is used to select network interface for CNF management.
	// If not defined then it will be selected automatically.
	CnfMgmtInterfaceEnvVar = "CNF_MGMT_INTERFACE"
	// Environment variable that is used to specify what IP subnet the management
	// IP addresses are allocated from. For example, if CNFs are deployed using docker
	// compose, then this variable should contain the subnet (~ IP pool) used by the bridge.
	// It is an alternative hint to CNF_MGMT_INTERFACE (of lower priority) for
	// correctly selecting the management interface.
	CnfMgmtSubnetEnvVar = "CNF_MGMT_SUBNET"
)

Variables

var DefaultPlugin = *NewPlugin()

DefaultPlugin is a default instance of CNF Registry.

Types

type CnfAPI

type CnfAPI interface {
	// Get mode in which the CNF operates with respect to other CNFs.
	GetCnfMode() pb.CnfMode
	// Returns gRPC port that should be used by this CNF.
	// Not to be used by StoneWork or a standalone CNF (they should respect what is in grpc.conf).
	GetGrpcPort() (port int)
	// Returns HTTP port that should be used by this CNF.
	// Not to be used by StoneWork or a standalone CNF (they should respect what is in http.conf).
	GetHttpPort() (port int)
	// Returns gRPC connection established with StoneWork.
	GetSWGrpcConn() (conn grpc.ClientConnInterface, err error)
	// Returns remote configuration client connected with the StoneWork.
	// Can be used to apply configuration into the VPP
	// (determined by the operation of a CNF, e.g. IP routes received over BGP).
	GetSWCfgClient() (cfgClient client.GenericClient, err error)
	// RegisterCnfModel registers configuration model implemented by the CNF.
	// This method should be used by a SW-Module CNF in the Init phase to convey the definition of a CNF NB API model
	// into StoneWork, which will then act as a proxy for all the operations over that model.
	RegisterCnfModel(model models.KnownModel, descriptor *kvs.KVDescriptor, callbacks *CnfModelCallbacks) error
}

API to be used by CNF (standalone or as SW-Module)

type CnfDeps

type CnfDeps struct {
	// CNF index that should be unique for each CNF.
	// Allocate 1 for the first CNF and give +1 for each new CNF.
	CnfIndex int
}

Dependencies to define/provide for CNF running standalone or as a SW-Module.

type CnfModelCallbacks

type CnfModelCallbacks struct {
	PuntRequests     PuntRequestsClb
	ItemDependencies ItemDepsClb
}

CnfModelCallbacks groups (optional) callbacks that can be assigned to a model registration.

type CnfRegistryAPI

type CnfRegistryAPI interface {
	CnfAPI
	StoneWorkAPI
}

CnfRegistryAPI encapsulates all methods exposed by CNF Registry. Please note, however, that not all the methods may be available in the given CNF mode.

type Config

type Config struct {
	// Start of the gRPC port range for StoneWork modules
	SwModGrpcBasePort int `json:"sw-module-grpc-base-port"`
	// Start of the HTTP port range for StoneWork modules
	SwModHttpBasePort int `json:"sw-module-http-base-port"`
	// Deprecated. Using this option will result in a warning
	CnfDiscoveryTimeout time.Duration `json:"cnf-discovery-timeout"`
}

Config file for CnfRegistry plugin.

type Deps

type Deps struct {
	infra.PluginDeps
	CnfDeps
	MgmtInterface string
	MgmtSubnet    string
	ServiceLabel  servicelabel.ReaderAPI
	KVScheduler   kvs.KVScheduler
	GRPCPlugin    *grpc_plugin.Plugin
	HTTPPlugin    *rest.Plugin // optional
	PuntMgr       PuntManagerAPI
}

Deps is a set of dependencies of the CNF Registry plugin

type Info

type Info struct {
	PID           int
	MsLabel       string
	CnfMode       pb.CnfMode
	IPAddr        string
	GRPCPort      int
	HTTPPort      int
	GRPCConnState connectivity.State
}

type ItemDepsClb

type ItemDepsClb func(configItem proto.Message) []*pb.ConfigItemDependency

ItemDepsClb is used to inform the plugin about all the dependencies of a given configuration item (apart from punt dependencies which are determined from the punt requests).

type Option

type Option func(plugin *Plugin)

Option is a function that can be used in NewPlugin allowing plugin customization

type PidFile

type PidFile struct {
	Pid       int    `json:"pid"`
	MsLabel   string `json:"ms-label"`
	IpAddress string `json:"ip-address"`
	GrpcPort  int    `json:"grpc-port"`
	HttpPort  int    `json:"http-port"`
}

PidFile written by SW-Module CNF.

type Plugin

type Plugin struct {
	Deps
	pb.UnimplementedCnfDiscoveryServer
	// contains filtered or unexported fields
}

CNF Registry plugin allows to load a CNF module into the StoneWork (all-in-one VPP distribution; SW for short) during the Init phase. CNF can be built as another image and run as a separate container. This allows to enable/disable CNF without having to rebuild StoneWork docker image or the agent binary. Apart from single common VPP it is also possible to share network namespace between all/some CNFs and therefore integrate different network functions inside the Linux network stack (e.g. to use OSPF-learned routes to connect with a BGP peer). The plugin operates in one of the 3 following modes depending on the value of the "CNF_MODE" environment variable:

1. STANDALONE (default, i.e. assumed if the variable is not defined): - CNF is used on its own, potentially chained with other CNFs using for example NSM (i.e. each VPP-based CNF runs its own VPP instance) - The CNF Registry plugin is also used by a Standalone CNF, but merely to keep track of CNF Index ID .
2. STONEWORK_MODULE: - CNF used as a SW-module - VPP-based CNFs do not run VPP inside their container, instead they connect with the all-in-one VPP of StoneWork - in this mode the Registry acts as a client of the Registry running by the StoneWork agent - internally the plugin uses gRPC to exchange all the information needed between the Registries of CNF and SW to load the CNF and use with the all-in-one VPP - CNF should use only those methods of the plugin which are defined by the CnfAPI interface
3. STONEWORK: - CNF Registry plugin is used by StoneWork firstly to discover all the enabled CNFs and then to collect all the information about them to be able to integrate them with the all-in-one VPP - for each CNF, StoneWork needs to learn the NB-facing configuration models, traffic Punting to use (some subset of the traffic typically needs to be diverted from VPP into the Linux network stack for the Linux-based CNF to process) - StoneWork should use only those methods of the plugin which are defined by the StoneWorkAPI interface

func NewPlugin

func NewPlugin(opts ...Option) *Plugin

NewPlugin creates a new Plugin with provided options

func (*Plugin) AfterInit

func (p *Plugin) AfterInit() (err error)

AfterInit is used by CNF-Module to write pid file under a known directory for StoneWork to discover it.

func (*Plugin) Close

func (p *Plugin) Close() error

Close is NOOP.

func (*Plugin) DiscoverCnf

func (p *Plugin) DiscoverCnf(ctx context.Context, req *pb.DiscoverCnfReq) (resp *pb.DiscoverCnfResp, err error)

DiscoverCnf is served by the CNFRegistry of each SW-Module CNF. It is called by StoneWork during Init of CNFRegistry.

func (*Plugin) GetCnfCfgClient

func (p *Plugin) GetCnfCfgClient(cnfMsLabel string) (cfgClient client.GenericClient, err error)

Returns remote configuration client connected with the given SW-Module CNF.

func (*Plugin) GetCnfGrpcConn

func (p *Plugin) GetCnfGrpcConn(cnfMsLabel string) (conn grpc.ClientConnInterface, err error)

Returns gRPC connection established with the given SW-Module CNF.

func (*Plugin) GetCnfMode

func (p *Plugin) GetCnfMode() pb.CnfMode

Get mode in which the CNF operates with respect to other CNFs.

func (*Plugin) GetGrpcPort

func (p *Plugin) GetGrpcPort() (port int)

Returns gRPC port that should be used by this CNF. Not to be used by StoneWork or a standalone CNF (they should respect what is in grpc.conf).

func (*Plugin) GetHttpPort

func (p *Plugin) GetHttpPort() (port int)

Returns gRPC port that should be used by this CNF. Not to be used by StoneWork or a standalone CNF (they should respect what is in http.conf).

func (*Plugin) GetItemDependencies

func (p *Plugin) GetItemDependencies(ctx context.Context, item *generic.Item) (itemDeps *pb.GetDependenciesResp, err error)

GetItemDependencies is served by CNFRegistry of a SW-Module CNF and returns the set of dependencies of the given configuration item (apart from punt deps which are determined from punt requests).

func (*Plugin) GetPuntRequests

func (p *Plugin) GetPuntRequests(ctx context.Context, item *generic.Item) (puntReqs *puntmgr.PuntRequests, err error)

GetPuntRequests is served by CNFRegistry of a SW-Module CNF and returns the set of packet punting requests corresponding to the given configuration item.

func (*Plugin) GetSWCfgClient

func (p *Plugin) GetSWCfgClient() (cfgClient client.GenericClient, err error)

Returns remote configuration client connected with the StoneWork. Can be used in combination to apply configuration into the VPP (determined by the operation of a CNF, e.g. IP routes received over BGP).

func (*Plugin) GetSWGrpcConn

func (p *Plugin) GetSWGrpcConn() (conn grpc.ClientConnInterface, err error)

Returns gRPC connection established with StoneWork.

func (*Plugin) Init

func (p *Plugin) Init() (err error)

Init initializes internal attributes and depending on the mode does the following: case STONEWORK_MODULE:

• registers gRPC handler for DiscoverCnf and GetPuntRequests

case STONEWORK:

• waits few seconds for all CNFs to write pid files
• then for each CNF:
• creates grpcConnection with the CNF
• obtains models using the meta service
• register models (exposed by Cnf)
• creates CnfDescriptorProxy for each module

func (*Plugin) RegisterCnfModel

func (p *Plugin) RegisterCnfModel(model models.KnownModel, descriptor *kvs.KVDescriptor, callbacks *CnfModelCallbacks) error

RegisterCnfModel registers configuration model implemented by the CNF. This method should be used by a SW-Module CNF in the Init phase to convey the definition of a CNF NB API model into StoneWork, which will then act as a proxy for all the operations over that model.

type PuntManagerAPI

type PuntManagerAPI interface {
	// AddPunt is used by StoneWork or standalone CNF to configure punt between VPP and the CNF.
	AddPunt(cnfMsLabel, key string, punt *puntmgr.PuntRequest) error
	// DelPunt is used by StoneWork or standalone CNF to un-configure punt between VPP and the CNF.
	DelPunt(cnfMsLabel, key string, label string) error
	// GetPuntDependencies returns dependencies that have to be satisfied before the punt can be added.
	GetPuntDependencies(cnfMsLabel string, punt *puntmgr.PuntRequest) (deps []kvs.Dependency)
}

APIs of Punt Manager that are used by CNF registry.

type PuntRequestsClb

type PuntRequestsClb func(configItem proto.Message) *puntmgr.PuntRequests

PuntRequestsClb is used to inform the plugin about how the packet punting should be configured for a given configuration item.

type StoneWorkAPI

type StoneWorkAPI interface {
	// Returns gRPC connection established with the given SW-Module CNF.
	GetCnfGrpcConn(cnfMsLabel string) (conn grpc.ClientConnInterface, err error)
	// Returns remote configuration client connected with the given SW-Module CNF.
	GetCnfCfgClient(cnfMsLabel string) (cfgClient client.GenericClient, err error)
}

API to be used by StoneWork.