calc

package
v3.5.5+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 29, 2019 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

The calc package implements a calculation graph for Felix's dynamic state. The graph filters and transforms updates from the backend Syncer into a stream of host-specific updates to policies, profiles, endpoints and IP sets.

The graph is available either with a synchronous callback API or as a channel-based async API. The async version of the API is recommended because it includes and EventBuffer to efficiently batch IP set updates. In addition, it converts the callbacks into structs from the felix/proto package, which are ready to be marshaled directly to the felix front-end.

// Using the async API.
asyncCalcGraph := calc.NewAsyncCalcGraph("hostname", outputChannel, nil)
syncer := fc.datastore.Syncer(asyncCalcGraph)
syncer.Start()
asyncCalcGraph.Start()
for event := range outputChannel {
	switch event := event.(type) {
	case *proto.XYZ:
		...
	...
}

The best explanation of the wiring of the calculation graph nodes is in the code comments inside NewCalculationGraph.

Copyright (c) 2016-2018 Tigera, Inc. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source
const (
	// Compromise: shorter is better for occupancy and readability. Longer is better for
	// collision-resistance.  16 chars gives us 96 bits of entropy, which is fairly collision
	// resistant.
	RuleIDLength = 16
)

Variables

View Source
var AllSelector selector.Selector

AllSelector is a pre-calculated copy of the "all()" selector.

View Source
var (
	DummyDropRules = model.ProfileRules{
		InboundRules:  []model.Rule{{Action: "deny"}},
		OutboundRules: []model.Rule{{Action: "deny"}},
	}
)

Functions

func ModelHostEndpointToProto

func ModelHostEndpointToProto(ep *model.HostEndpoint, tiers, untrackedTiers, preDNATTiers []*proto.TierInfo, forwardTiers []*proto.TierInfo) *proto.HostEndpoint

func ModelWorkloadEndpointToProto

func ModelWorkloadEndpointToProto(ep *model.WorkloadEndpoint, tiers []*proto.TierInfo) *proto.WorkloadEndpoint

func NewTierInfo

func NewTierInfo(name string) *tierInfo

func ParsedRulesToActivePolicyUpdate

func ParsedRulesToActivePolicyUpdate(key model.PolicyKey, rules *ParsedRules) *proto.ActivePolicyUpdate

Types

type ActiveRulesCalculator

type ActiveRulesCalculator struct {

	// Callback objects.
	RuleScanner           ruleScanner
	PolicyMatchListener   PolicyMatchListener
	OnPolicyCountsChanged func(numPolicies, numProfiles, numALPPolicies int)
	// contains filtered or unexported fields
}

ActiveRulesCalculator calculates the set of policies and profiles (i.e. the rules) that are active for the particular endpoints that it's been told about. It emits events when the set of active rules changes.

For example, if the ActiveRulesCalculator is fed *all* the policies/profiles along with the endpoints that are on the local host then its output (via the callback objects) will indicate exactly which policies/profiles are active on the local host.

When looking at policies, the ActiveRules calculator is only interested in the selector attached to the policy itself (which determines the set of endpoints that it applies to). The rules in a policy may also contain selectors; those are are ignored here; they are mapped to IP sets by the RuleScanner.

func NewActiveRulesCalculator

func NewActiveRulesCalculator() *ActiveRulesCalculator

func (*ActiveRulesCalculator) OnStatusUpdate

func (arc *ActiveRulesCalculator) OnStatusUpdate(status api.SyncStatus)

func (*ActiveRulesCalculator) OnUpdate

func (arc *ActiveRulesCalculator) OnUpdate(update api.Update) (_ bool)

func (*ActiveRulesCalculator) RegisterWith

func (arc *ActiveRulesCalculator) RegisterWith(localEndpointDispatcher, allUpdDispatcher *dispatcher.Dispatcher)

type AsyncCalcGraph

type AsyncCalcGraph struct {
	*CalcGraph
	// contains filtered or unexported fields
}

func NewAsyncCalcGraph

func NewAsyncCalcGraph(
	conf *config.Config,
	outputChannels []chan<- interface{},
	healthAggregator *health.HealthAggregator,
) *AsyncCalcGraph

func (*AsyncCalcGraph) OnStatusUpdated

func (acg *AsyncCalcGraph) OnStatusUpdated(status api.SyncStatus)

func (*AsyncCalcGraph) OnUpdates

func (acg *AsyncCalcGraph) OnUpdates(updates []api.Update)

func (*AsyncCalcGraph) Start

func (acg *AsyncCalcGraph) Start()

type CalcGraph

type CalcGraph struct {
	// AllUpdDispatcher is the input node to the calculation graph.
	AllUpdDispatcher *dispatcher.Dispatcher
	// contains filtered or unexported fields
}

func NewCalculationGraph

func NewCalculationGraph(callbacks PipelineCallbacks, hostname string) *CalcGraph

type ConfigBatcher

type ConfigBatcher struct {
	// contains filtered or unexported fields
}

func NewConfigBatcher

func NewConfigBatcher(hostname string, callbacks configCallbacks) *ConfigBatcher

func (*ConfigBatcher) OnDatamodelStatus

func (cb *ConfigBatcher) OnDatamodelStatus(status api.SyncStatus)

func (*ConfigBatcher) OnUpdate

func (cb *ConfigBatcher) OnUpdate(update api.Update) (filterOut bool)

func (*ConfigBatcher) RegisterWith

func (cb *ConfigBatcher) RegisterWith(allUpdDispatcher *dispatcher.Dispatcher)

type DataplanePassthru

type DataplanePassthru struct {
	// contains filtered or unexported fields
}

DataplanePassthru passes through some datamodel updates to the dataplane layer, removing some duplicates along the way. It maps OnUpdate() calls to dedicated method calls for consistency with the rest of the dataplane API.

func NewDataplanePassthru

func NewDataplanePassthru(callbacks passthruCallbacks) *DataplanePassthru

func (*DataplanePassthru) OnUpdate

func (h *DataplanePassthru) OnUpdate(update api.Update) (filterOut bool)

func (*DataplanePassthru) RegisterWith

func (h *DataplanePassthru) RegisterWith(dispatcher *dispatcher.Dispatcher)

type DatastoreNotReady

type DatastoreNotReady struct{}

type EndpointKeyToProfileIDMap

type EndpointKeyToProfileIDMap struct {
	// contains filtered or unexported fields
}

EndpointKeyToProfileIDMap is a specialised map that calculates the deltas to the profile IDs when making an update.

func NewEndpointKeyToProfileIDMap

func NewEndpointKeyToProfileIDMap() *EndpointKeyToProfileIDMap

func (EndpointKeyToProfileIDMap) Update

func (idx EndpointKeyToProfileIDMap) Update(
	key model.Key,
	profileIDs []string,
) (
	removedIDs, addedIDs map[string]bool,
)

type EventHandler

type EventHandler func(message interface{})

type EventSequencer

type EventSequencer struct {
	Callback EventHandler
	// contains filtered or unexported fields
}

EventSequencer buffers and coalesces updates from the calculation graph then flushes them when Flush() is called. It flushed updates in a dependency-safe order.

func NewEventSequencer

func NewEventSequencer(conf configInterface) *EventSequencer

func (*EventSequencer) Flush

func (buf *EventSequencer) Flush()

func (*EventSequencer) OnConfigUpdate

func (buf *EventSequencer) OnConfigUpdate(globalConfig, hostConfig map[string]string)

func (*EventSequencer) OnDatastoreNotReady

func (buf *EventSequencer) OnDatastoreNotReady()

func (*EventSequencer) OnEndpointTierUpdate

func (buf *EventSequencer) OnEndpointTierUpdate(key model.Key,
	endpoint interface{},
	filteredTiers []tierInfo,
)

func (*EventSequencer) OnHostIPRemove

func (buf *EventSequencer) OnHostIPRemove(hostname string)

func (*EventSequencer) OnHostIPUpdate

func (buf *EventSequencer) OnHostIPUpdate(hostname string, ip *net.IP)

func (*EventSequencer) OnIPPoolRemove

func (buf *EventSequencer) OnIPPoolRemove(key model.IPPoolKey)

func (*EventSequencer) OnIPPoolUpdate

func (buf *EventSequencer) OnIPPoolUpdate(key model.IPPoolKey, pool *model.IPPool)

func (*EventSequencer) OnIPSetAdded

func (buf *EventSequencer) OnIPSetAdded(setID string, ipSetType proto.IPSetUpdate_IPSetType)

func (*EventSequencer) OnIPSetMemberAdded

func (buf *EventSequencer) OnIPSetMemberAdded(setID string, member labelindex.IPSetMember)

func (*EventSequencer) OnIPSetMemberRemoved

func (buf *EventSequencer) OnIPSetMemberRemoved(setID string, member labelindex.IPSetMember)

func (*EventSequencer) OnIPSetRemoved

func (buf *EventSequencer) OnIPSetRemoved(setID string)

func (*EventSequencer) OnNamespaceRemove

func (buf *EventSequencer) OnNamespaceRemove(id proto.NamespaceID)

func (*EventSequencer) OnNamespaceUpdate

func (buf *EventSequencer) OnNamespaceUpdate(update *proto.NamespaceUpdate)

func (*EventSequencer) OnPolicyActive

func (buf *EventSequencer) OnPolicyActive(key model.PolicyKey, rules *ParsedRules)

func (*EventSequencer) OnPolicyInactive

func (buf *EventSequencer) OnPolicyInactive(key model.PolicyKey)

func (*EventSequencer) OnProfileActive

func (buf *EventSequencer) OnProfileActive(key model.ProfileRulesKey, rules *ParsedRules)

func (*EventSequencer) OnProfileInactive

func (buf *EventSequencer) OnProfileInactive(key model.ProfileRulesKey)

func (*EventSequencer) OnServiceAccountRemove

func (buf *EventSequencer) OnServiceAccountRemove(id proto.ServiceAccountID)

func (*EventSequencer) OnServiceAccountUpdate

func (buf *EventSequencer) OnServiceAccountUpdate(update *proto.ServiceAccountUpdate)

type FelixSender

type FelixSender interface {
	SendUpdateToFelix(update model.KVPair)
}

type IPSetData

type IPSetData struct {
	// The selector and named port that this IP set represents.  To represent an unfiltered named
	// port, set selector to AllSelector.  If NamedPortProtocol == ProtocolNone then
	// this IP set represents a selector only, with no named port component.
	Selector selector.Selector
	// NamedPortProtocol identifies the protocol (TCP or UDP) for a named port IP set.  It is
	// set to ProtocolNone for a selector-only IP set.
	NamedPortProtocol labelindex.IPSetPortProtocol
	// NamedPort contains the name of the named port represented by this IP set or "" for a
	// selector-only IP set
	NamedPort string
	// contains filtered or unexported fields
}

func (*IPSetData) DataplaneProtocolType

func (d *IPSetData) DataplaneProtocolType() proto.IPSetUpdate_IPSetType

DataplaneProtocolType returns the dataplane driver protocol type of this IP set. One of the proto.IPSetUpdate_IPSetType constants.

func (*IPSetData) UniqueID

func (d *IPSetData) UniqueID() string

type ParsedRule

type ParsedRule struct {
	Action string

	IPVersion *int

	Protocol *numorstring.Protocol

	SrcNets              []*net.IPNet
	SrcPorts             []numorstring.Port
	SrcNamedPortIPSetIDs []string
	DstNets              []*net.IPNet
	DstPorts             []numorstring.Port
	DstNamedPortIPSetIDs []string
	ICMPType             *int
	ICMPCode             *int
	SrcIPSetIDs          []string
	DstIPSetIDs          []string

	NotProtocol             *numorstring.Protocol
	NotSrcNets              []*net.IPNet
	NotSrcPorts             []numorstring.Port
	NotSrcNamedPortIPSetIDs []string
	NotDstNets              []*net.IPNet
	NotDstPorts             []numorstring.Port
	NotDstNamedPortIPSetIDs []string
	NotICMPType             *int
	NotICMPCode             *int
	NotSrcIPSetIDs          []string
	NotDstIPSetIDs          []string

	// These fields allow us to pass through the raw match criteria from the V3 datamodel,
	// unmodified. The selectors above are formed in the update processor layer by combining the
	// original selectors, namespace selectors an service account matches into one.
	OriginalSrcSelector               string
	OriginalSrcNamespaceSelector      string
	OriginalDstSelector               string
	OriginalDstNamespaceSelector      string
	OriginalNotSrcSelector            string
	OriginalNotDstSelector            string
	OriginalSrcServiceAccountNames    []string
	OriginalSrcServiceAccountSelector string
	OriginalDstServiceAccountNames    []string
	OriginalDstServiceAccountSelector string

	// These fields allow us to pass through the HTTP match criteria from the V3 datamodel. The iptables dataplane
	// does not implement the match, but other dataplanes such as Dikastes do.
	HTTPMatch *model.HTTPMatch
}

ParsedRule is like a backend.model.Rule, except the tag and selector matches and named ports are replaced with pre-calculated ipset IDs.

type ParsedRules

type ParsedRules struct {
	// For NetworkPolicies, Namespace is set to the original namespace of the NetworkPolicy.
	// For GlobalNetworkPolicies and Profiles, "".
	Namespace string

	InboundRules  []*ParsedRule
	OutboundRules []*ParsedRule

	// Untracked is true if these rules should not be "conntracked".
	Untracked bool

	// PreDNAT is true if these rules should be applied before any DNAT.
	PreDNAT bool
}

ParsedRules holds our intermediate representation of either a policy's rules or a profile's rules. As part of its processing, the RuleScanner converts backend rules into ParsedRules. Where backend rules contain selectors, tags and named ports, ParsedRules only contain IPSet IDs. The RuleScanner calculates the relevant IDs as it processes the rules and diverts the details of the active tags, selectors and named ports to the named port index, which figures out the members that should be in those IP sets.

type PipelineCallbacks

type PipelineCallbacks interface {
	// contains filtered or unexported methods
}

type PolKV

type PolKV struct {
	Key   model.PolicyKey
	Value *model.Policy
	// contains filtered or unexported fields
}

Note: PolKV is really internal to the calc package. It is named with an initial capital so that the test package calc_test can also use it.

func (*PolKV) GovernsEgress

func (p *PolKV) GovernsEgress() bool

func (*PolKV) GovernsIngress

func (p *PolKV) GovernsIngress() bool

func (PolKV) String

func (p PolKV) String() string

type PolicyByOrder

type PolicyByOrder []PolKV

func (PolicyByOrder) Len

func (a PolicyByOrder) Len() int

func (PolicyByOrder) Less

func (a PolicyByOrder) Less(i, j int) bool

func (PolicyByOrder) Swap

func (a PolicyByOrder) Swap(i, j int)

type PolicyMatchListener

type PolicyMatchListener interface {
	OnPolicyMatch(policyKey model.PolicyKey, endpointKey interface{})
	OnPolicyMatchStopped(policyKey model.PolicyKey, endpointKey interface{})
}

type PolicyResolver

type PolicyResolver struct {
	Callbacks PolicyResolverCallbacks
	InSync    bool
	// contains filtered or unexported fields
}

PolicyResolver marries up the active policies with local endpoints and calculates the complete, ordered set of policies that apply to each endpoint. As policies and endpoints are added/removed/updated, it emits events via the PolicyResolverCallbacks with the updated set of matching policies.

The PolicyResolver doesn't figure out which policies are currently active, it expects to be told via its OnPolicyMatch(Stopped) methods which policies match which endpoints. The ActiveRulesCalculator does that calculation.

func NewPolicyResolver

func NewPolicyResolver() *PolicyResolver

func (*PolicyResolver) OnDatamodelStatus

func (pr *PolicyResolver) OnDatamodelStatus(status api.SyncStatus)

func (*PolicyResolver) OnPolicyMatch

func (pr *PolicyResolver) OnPolicyMatch(policyKey model.PolicyKey, endpointKey interface{})

func (*PolicyResolver) OnPolicyMatchStopped

func (pr *PolicyResolver) OnPolicyMatchStopped(policyKey model.PolicyKey, endpointKey interface{})

func (*PolicyResolver) OnUpdate

func (pr *PolicyResolver) OnUpdate(update api.Update) (filterOut bool)

func (*PolicyResolver) RegisterWith

func (pr *PolicyResolver) RegisterWith(allUpdDispatcher, localEndpointDispatcher *dispatcher.Dispatcher)

type PolicyResolverCallbacks

type PolicyResolverCallbacks interface {
	OnEndpointTierUpdate(endpointKey model.Key, endpoint interface{}, filteredTiers []tierInfo)
}

type PolicySorter

type PolicySorter struct {
	// contains filtered or unexported fields
}

func NewPolicySorter

func NewPolicySorter() *PolicySorter

func (*PolicySorter) OnUpdate

func (poc *PolicySorter) OnUpdate(update api.Update) (dirty bool)

func (*PolicySorter) Sorted

func (poc *PolicySorter) Sorted() *tierInfo

type ProfileDecoder

type ProfileDecoder struct {
	// contains filtered or unexported fields
}

ProfileDecoder takes updates from a dispatcher, determines if the profile is a Kubernetes Service Account or Kubernetes Namespace, and if it is, generates a dataplane update or remove for it.

func NewProfileDecoder

func NewProfileDecoder(callbacks passthruCallbacks) *ProfileDecoder

func (*ProfileDecoder) OnUpdate

func (p *ProfileDecoder) OnUpdate(update api.Update) (filterOut bool)

func (*ProfileDecoder) RegisterWith

func (p *ProfileDecoder) RegisterWith(d *dispatcher.Dispatcher)

type RuleScanner

type RuleScanner struct {
	OnIPSetActive   func(ipSet *IPSetData)
	OnIPSetInactive func(ipSet *IPSetData)

	RulesUpdateCallbacks rulesUpdateCallbacks
	// contains filtered or unexported fields
}

RuleScanner scans the rules sent to it by the ActiveRulesCalculator, looking for tags and selectors. It calculates the set of active tags and selectors and emits events when they become active/inactive.

Previously, Felix tracked tags and selectors separately, with a separate tag and label index. However, we found that had a high occupancy cost. The current code uses a shared index and maps tags onto labels, so a tag named tagName, becomes a label tagName="". The RuleScanner maps tags to label selectors of the form "has(tagName)", taking advantage of the mapping. Such a selector is almost equivalent to having the tag; the only case where the behaviour would differ is if the user was using the same name for a tag and a label and the label and tags of the same name were applied to different endpoints. Since tags are being deprecated, we can live with that potential aliasing issue in return for a significant occupancy improvement at high scale.

The RuleScanner also emits events when rules are updated: since the input rule structs contain tags and selectors but downstream, we only care about IP sets, the RuleScanner converts rules from model.Rule objects to calc.ParsedRule objects. The latter share most fields, but the tags and selector fields are replaced by lists of IP sets.

The RuleScanner only calculates which selectors and tags are active/inactive. It doesn't match endpoints against tags/selectors. (That is done downstream in a labelindex.InheritIndex created in NewCalculationGraph.)

func NewRuleScanner

func NewRuleScanner() *RuleScanner

func (*RuleScanner) OnPolicyActive

func (rs *RuleScanner) OnPolicyActive(key model.PolicyKey, policy *model.Policy)

func (*RuleScanner) OnPolicyInactive

func (rs *RuleScanner) OnPolicyInactive(key model.PolicyKey)

func (*RuleScanner) OnProfileActive

func (rs *RuleScanner) OnProfileActive(key model.ProfileRulesKey, profile *model.ProfileRules)

func (*RuleScanner) OnProfileInactive

func (rs *RuleScanner) OnProfileInactive(key model.ProfileRulesKey)

type StatsCollector

type StatsCollector struct {
	Callback func(StatsUpdate) error
	// contains filtered or unexported fields
}

func NewStatsCollector

func NewStatsCollector(callback func(StatsUpdate) error) *StatsCollector

func (*StatsCollector) OnStatusUpdate

func (s *StatsCollector) OnStatusUpdate(status api.SyncStatus)

func (*StatsCollector) OnUpdate

func (s *StatsCollector) OnUpdate(update api.Update) (filterOut bool)

func (*StatsCollector) RegisterWith

func (s *StatsCollector) RegisterWith(calcGraph *CalcGraph)

func (*StatsCollector) UpdatePolicyCounts

func (s *StatsCollector) UpdatePolicyCounts(numPolicies, numProfiles, numALPPolicies int)

type StatsUpdate

type StatsUpdate struct {
	NumHosts             int
	NumWorkloadEndpoints int
	NumHostEndpoints     int
	NumPolicies          int
	NumProfiles          int
	NumALPPolicies       int
}

func (StatsUpdate) String

func (s StatsUpdate) String() string

type SyncerCallbacksDecoupler

type SyncerCallbacksDecoupler struct {
	// contains filtered or unexported fields
}

func NewSyncerCallbacksDecoupler

func NewSyncerCallbacksDecoupler() *SyncerCallbacksDecoupler

func (*SyncerCallbacksDecoupler) OnStatusUpdated

func (a *SyncerCallbacksDecoupler) OnStatusUpdated(status api.SyncStatus)

func (*SyncerCallbacksDecoupler) OnUpdates

func (a *SyncerCallbacksDecoupler) OnUpdates(updates []api.Update)

func (*SyncerCallbacksDecoupler) SendTo

type ValidationFilter

type ValidationFilter struct {
	// contains filtered or unexported fields
}

func NewValidationFilter

func NewValidationFilter(sink api.SyncerCallbacks) *ValidationFilter

func (*ValidationFilter) OnStatusUpdated

func (v *ValidationFilter) OnStatusUpdated(status api.SyncStatus)

func (*ValidationFilter) OnUpdates

func (v *ValidationFilter) OnUpdates(updates []api.Update)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL