fqdn

package
v1.7.0-rc4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 13, 2020 License: Apache-2.0 Imports: 24 Imported by: 12

Documentation

Overview

Package fqdn handles DNS based policy enforcment. This is expressed via ToFQDN rules and implements a DNS polling scheme with DNS lookups originating from the Cilium agent.

Note: We add a ToFQDN-UUID label to rules when we process a ToFQDN section. This has the source cilium-generated and should not be modified outside pkg/fqdn

The poller will update imported policy rules that contain ToFQDN sections with matching ToCIDRSet sections (in the same egress rule, thus inheriting the same L4/L7 policy). Each CIDR is a fully qualified IP (i.e. a /32 or /128) and each IP returned in the DNS lookup creates a corresponding CIDR. The package relies on the internal policy logic to return early/trigger no regenerations if the policy is not actually different (e.g. a more broad/permissive rule already applies to an endpoint so any IP changes are irrelevant).

Index

Constants

View Source 
const DNSPollerInterval = 5 * time.Second

DNSPollerInterval is the time between 2 complete DNS lookup runs of the DNSPoller controller Note: This cannot be less than 1*time.Second, as it is used as a default for MinTTL in DNSPollerConfig

Variables

This section is empty.

Functions

func ConfigFromResolvConf added in v1.5.0 

func ConfigFromResolvConf() error

ConfigFromResolvConf parses the configuration in /etc/resolv.conf and sets the configuration for pkg/fqdn. nameservers and opt timeout are supported. search and ndots are NOT supported. This call is not thread safe.

func DNSLookupDefaultResolver added in v1.5.0 

func DNSLookupDefaultResolver(dnsNames []string) (DNSIPs map[string]*DNSIPRecords, DNSErrors map[string]error)

DNSLookupDefaultResolver sequentially and synchronously runs a DNS lookup for every name in dnsNames. It does not use net.DefaultResolver, but dnsConfig from this package (populated from resolv.conf). The configured servers are always queried in the same order, only moving to the next on errors (such as timeouts). The names are queried in random order (as the map iteration is random) but, for each name, A records and then AAAA records are requested in that order. It will return: DNSIPs: a map of DNS names to their IPs and associated smallest TTL (only contains successful lookups). CNAME records in the response are collapsed into the IPs later in the response data. The CNAME TTL is considered when finding the smallest TTL. DNSErrors: a map of DNS names to lookup errors.

DNSLookupDefaultResolver is used by DNSPoller when no alternative LookupDNSNames is provided.

func KeepUniqueNames added in v1.5.0 

func KeepUniqueNames(names []string) []string

KeepUniqueNames it gets a array of strings and return a new array of strings with the unique names.

func SetDNSConfig added in v1.5.0 

func SetDNSConfig(conf *dns.ClientConfig)

SetDNSConfig store conf in pkg/fqdn as the global config. It also creates the global UDP and TCP clients used for DNS lookups in DNSLookupDefaultResolver. Only .Servers and .Timeout are utilized from conf. This call is not thread safe.

func StartDNSPoller added in v1.5.0 

func StartDNSPoller(poller *DNSPoller)

StartDNSPoller spawns a singleton DNS polling controller. The controller will, periodically, run a DNS lookup for each ToFQDN target DNS name inserted with StartPollForDNSName. Note: Repeated calls will replace earlier instances of the controller.

Types

type Config 

type Config struct {
	// MinTTL is the time used by the poller to cache information.
	// When set to 0, 2*DNSPollerInterval is used.
	MinTTL int

	// OverLimit is the number of max entries that a host can have in the DNS cache.
	OverLimit int

	// Cache is where the poller stores DNS data used to generate rules.
	// When set to nil, it uses fqdn.DefaultDNSCache, a global cache instance.
	Cache *DNSCache

	// DNSConfig includes the Resolver IPs, port, timeout and retry count. It is
	// expected to be  generated from /etc/resolv.conf.
	DNSConfig *dns.ClientConfig

	// LookupDNSNames is a callback to run the provided DNS lookups.
	// When set to nil, fqdn.DNSLookupDefaultResolver is used.
	LookupDNSNames func(dnsNames []string) (DNSIPs map[string]*DNSIPRecords, errorDNSNames map[string]error)

	// UpdateSelectors is a callback to update the mapping of FQDNSelector to
	// sets of IPs.
	UpdateSelectors func(ctx context.Context, selectorsWithIPs map[api.FQDNSelector][]net.IP, selectorsWithoutIPs []api.FQDNSelector) (*sync.WaitGroup, error)

	// PollerResponseNotify is used when the poller receives DNS data in response
	// to a successful poll.
	// Note: This function doesn't do much, as the poller is still wired to
	// NameManager directly right now.
	PollerResponseNotify func(lookupTime time.Time, qname string, response *DNSIPRecords)
}

Config is a simple configuration structure to set how pkg/fqdn subcomponents behave. DNSPoller relies on LookupDNSNames to control how DNS lookups are done, and UpdateSelectors to control how generated policy rules are emitted.

type DNSCache 

type DNSCache struct {
	lock.RWMutex
	// contains filtered or unexported fields
}

DNSCache manages DNS data that will expire after a certain TTL. Information is tracked per-IP address, retaining the latest-expiring DNS data for each address. For most real-world DNS data, the entry per name remains small because newer lookups replace older ones. Large TTLs may cause entries to grow if many unique IPs are returned in separate lookups. It is critical to run .GC periodically. This cleans up expired entries and steps forward the time used to determine that entries are expired. This means that the Lookup functions may return expired entries until GC is called. Redundant entries are removed on insert.

func NewDNSCache 

func NewDNSCache(minTTL int) *DNSCache

NewDNSCache returns an initialized DNSCache

func NewDNSCacheWithLimit 

func NewDNSCacheWithLimit(minTTL int, limit int) *DNSCache

NewDNSCache returns an initialized DNSCache and set the max host limit to the given argument

func (*DNSCache) Dump 

func (c *DNSCache) Dump() (lookups []*cacheEntry)

Dump returns unexpired cache entries in the cache. They are deduplicated, but not usefully sorted. These objects should not be modified.

func (*DNSCache) ForceExpire 

func (c *DNSCache) ForceExpire(expireLookupsBefore time.Time, nameMatch *regexp.Regexp) (namesAffected []string)

ForceExpire is used to clear entries from the cache before their TTL is over. This operation does not keep previous guarantees that, for each IP, the most recent lookup to provide that IP is used. Note that all parameters must match, if provided. `time.Time{}` is the match-all time parameter. For example: 

ForceExpire(time.Time{}, 'cilium.io') expires all entries for cilium.io.
ForceExpire(time.Now(), 'cilium.io') expires all entries for cilium.io
that expired before the current time.

expireLookupsBefore requires a lookup to have a LookupTime before it in order to remove it. nameMatch will remove any DNS names that match.

func (*DNSCache) ForceExpireByNames added in v1.5.0 

func (c *DNSCache) ForceExpireByNames(expireLookupsBefore time.Time, names []string) (namesAffected []string)

ForceExpireByNames is the same function as ForceExpire but uses the exact names to delete the entries.

func (*DNSCache) GC 

func (c *DNSCache) GC(now time.Time, zombies *DNSZombieMappings) (affectedNames []string)

GC cleans TTL expired entries up to now, and overlimit entries, returning both sets. If zombies is passed in, expired IPs are inserted into it. GC and other management of zombies is left to the caller. Note: zombies use the original lookup's ExpirationTime for DeletePendingAt, not the now parameter. This allows better ordering in zombie GC.

func (*DNSCache) Lookup 

func (c *DNSCache) Lookup(name string) (ips []net.IP)

Lookup returns a set of unique IPs that are currently unexpired for name, if any exist. An empty list indicates no valid records exist. The IPs are returned sorted.

func (*DNSCache) LookupByRegexp 

func (c *DNSCache) LookupByRegexp(re *regexp.Regexp) (matches map[string][]net.IP)

LookupByRegexp returns all non-expired cache entries that match re as a map of name -> IPs

func (*DNSCache) LookupIP 

func (c *DNSCache) LookupIP(ip net.IP) (names []string)

LookupIP returns all DNS names in entries that include that IP. The cache maintains the latest-expiring entry per-name per-IP. This means that multiple names referrring to the same IP will expire from the cache at different times, and only 1 entry for each name-IP pair is internally retained.

func (*DNSCache) MarshalJSON 

func (c *DNSCache) MarshalJSON() ([]byte, error)

MarshalJSON serialises the set of DNS lookup cacheEntries needed to reconstruct this cache instance. Note: Expiration times are honored and the reconstructed cache instance is expected to return the same values as the original at that point in time.

func (*DNSCache) ReplaceFromCacheByNames 

func (c *DNSCache) ReplaceFromCacheByNames(namesToUpdate []string, updates ...*DNSCache)

ReplaceFromCacheByNames operates as an atomic combination of ForceExpire and multiple UpdateFromCache invocations. The result is to collect all entries for DNS names in namesToUpdate from each DNSCache in updates, replacing the current entries for each of those names.

func (*DNSCache) UnmarshalJSON 

func (c *DNSCache) UnmarshalJSON(raw []byte) error

UnmarshalJSON rebuilds a DNSCache from serialized JSON. Note: This is destructive to any currect data. Use UpdateFromCache for bulk updates.

func (*DNSCache) Update 

func (c *DNSCache) Update(lookupTime time.Time, name string, ips []net.IP, ttl int) bool

Update inserts a new entry into the cache. After insertion cache entries for name are expired and redundant entries evicted. This is O(number of new IPs) for eviction, and O(number of IPs for name) for expiration. lookupTime is the time the DNS information began being valid. It should be in the past. name is used as is and may be an unqualified name (e.g. myservice.namespace). ips may be an IPv4 or IPv6 IP. Duplicates will be removed. ttl is the DNS TTL for ips and is a seconds value.

func (*DNSCache) UpdateFromCache 

func (c *DNSCache) UpdateFromCache(update *DNSCache, namesToUpdate []string)

UpdateFromCache is a utility function that allows updating a DNSCache instance with all the internal entries of another. Latest-Expiration still applies, thus the merged outcome is consistent with adding the entries individually. When namesToUpdate has non-zero length only those names are updated from update, otherwise all DNS names in update are used.

type DNSIPRecords 

type DNSIPRecords struct {
	// TTL is the time, in seconds, that these IPs are valid for
	TTL int

	// IPs are the IPs associated with a DNS Name
	IPs []net.IP
}

DNSIPRecords mimics the RR data from an A or AAAA response. My kingdom for a DNS IP RR type that isn't hidden in the stdlib or has a million layers of type indirection.

type DNSPoller added in v1.5.0 

type DNSPoller struct {
	lock.Mutex // this guards both maps and their contents

	// DNSHistory is the collection of still-valid DNS responses intercepted
	// for the poller.
	// This is not protected by the mutex due to internally is using a mutex.
	DNSHistory *DNSCache
	// contains filtered or unexported fields
}

DNSPoller periodically runs lookups for registered DNS names. It will emit regenerated policy rules when the IPs change. CNAMEs (and DNAMEs) are not handled directly, but will depend on the resolver's behavior. fqdn.Config can be opitonally used to set how the DNS lookups are executed (via LookupDNSNames) and how generated policy rules are handled (via UpdateSelectors).

func NewDNSPoller added in v1.5.0 

func NewDNSPoller(config Config, ruleManager *NameManager) *DNSPoller

NewDNSPoller creates an initialized DNSPoller. It does not start the controller (use .Start)

func (*DNSPoller) LookupUpdateDNS added in v1.5.0 

func (poller *DNSPoller) LookupUpdateDNS(ctx context.Context) error

LookupUpdateDNS runs a DNS lookup for each stored DNS name, storing updates into ruleManager, which may emit regenerated policy rules. The general steps are: 1- take a snapshot of DNS names to lookup from .ruleManager, into dnsNamesToPoll 2- Do a DNS lookup for each DNS name (map key) in poller via LookupDNSNames 3- Update IPs for each dnsName in .ruleManager. If the IPs have changed for the name, it will generate and emit them.

type DNSZombieMapping 

type DNSZombieMapping struct {
	// Names is the list of names that had DNS lookups with this IP. These may
	// derive from unrelated DNS lookups. The list is maintained de-duplicated.
	Names []string `json:"names,omitempty"`

	// IP is an address that is pending for delete but may be in-use by a
	// connection.
	IP net.IP `json:"ip,omitempty"`

	// AliveAt is the last time this IP was marked alive via
	// DNSZombieMappings.MarkAlive.
	// When AliveAt is later than DNSZombieMappings.lastCTGCUpdate the zombie is
	// considered alive.
	AliveAt time.Time `json:"alive-at,omitempty"`

	// DeletePendingAt is the time at which this IP was most-recently scheduled
	// for deletion. This can be updated if an IP expires from the DNS caches
	// multiple times.
	// When DNSZombieMappings.lastCTGCUpdate is earlier than DeletePendingAt a
	// zombie is alive.
	DeletePendingAt time.Time `json:"delete-pending-at,omitempty"`
}

DNSZombieMapping is an IP that has expired or been evicted from a DNS cache. It records the DNS name and IP, along with other bookkeeping timestamps that help determine when it can be finally deleted. Zombies are dead when they are not marked alive by CT GC. Special handling exists when the count of zombies is large. Overlimit zombies are deleted in GC with the following preferences (this is cumulative and in order of precedence):

  • Zombies with zero AliveAt are evicted before those with a non-zero value (i.e. known connections marked by CT GC are evicted last)
  • Zombies with an earlier DeletePendingAtTime are evicted first. Note: Upsert sets DeletePendingAt on every update, thus making GC prefer to evict IPs with less DNS churn on them.
  • Zombies with the lowest count of DNS names in them are evicted first

func (*DNSZombieMapping) DeepCopy 

func (zombie *DNSZombieMapping) DeepCopy() *DNSZombieMapping

DeepCopy returns a copy of zombie that does not share any internal pointers or fields

type DNSZombieMappings 

type DNSZombieMappings struct {
	lock.Mutex
	// contains filtered or unexported fields
}

DNSZombieMappings collects DNS Name->IP mappings that may be inactive and evicted, and so may be deleted. They are periodically marked alive by the CT GC goroutine. When .GC is called, alive and dead zombies are returned, allowing us to skip deleting an IP from the global DNS cache to avoid breaking connections that outlast the DNS TTL.

func NewDNSZombieMappings 

func NewDNSZombieMappings(max int) *DNSZombieMappings

NewDNSZombieMappings constructs a DNSZombieMappings that is read to use

func (*DNSZombieMappings) DumpAlive 

func (zombies *DNSZombieMappings) DumpAlive() (alive []*DNSZombieMapping)

DumpAlive returns copies of still-alive zombies

func (*DNSZombieMappings) ForceExpire 

func (zombies *DNSZombieMappings) ForceExpire(expireLookupsBefore time.Time, nameMatch *regexp.Regexp, cidr *net.IPNet) (namesAffected []string)

ForceExpire is used to clear zombies irrespective of their alive status. Only zombies with DeletePendingAt times before expireLookupBefore are considered for deletion. Each name in an zombie is matched against nameMatcher (nil is match all) and when an zombie no longer has any valid names will it be removed outright. Note that all parameters must match, if provided. `time.Time{}` is the match-all time parameter. expireLookupsBefore requires an zombie to have been enqueued before the specified time in order to remove it. For example: 

ForceExpire(time.Time{}, 'cilium.io') expires all entries for cilium.io.
ForceExpire(time.Now(), 'cilium.io') expires all entries for cilium.io
that expired before the current time.

nameMatch will remove that specific DNS name from zombies that include it, deleting it when no DNS names remain.

func (*DNSZombieMappings) ForceExpireByNameIP 

func (zombies *DNSZombieMappings) ForceExpireByNameIP(expireLookupsBefore time.Time, name string, ips ...net.IP) error

ForceExpireByNameIP wraps ForceExpire to simplify clearing all IPs from a new DNS lookup. The error return is for errors compiling the internal regexp. This should never happen.

func (*DNSZombieMappings) GC 

func (zombies *DNSZombieMappings) GC() (alive, dead []*DNSZombieMapping)

GC returns alive and dead DNSZombieMapping entries. This removes dead zombies interally, and repeated calls will return different data. Zombies are alive if they have been marked alive (with MarkAlive). When SetCTGCTime is called and an zombie not marked alive, it becomes dead. Calling Upsert on a dead zombie will make it alive again. Alive zombies are limited by zombies.max. 0 means no zombies are allowed, disabling the behavior. It is expected to be a large value and is in place to avoid runaway zombie growth when CT GC is at a large interval.

func (*DNSZombieMappings) MarkAlive 

func (zombies *DNSZombieMappings) MarkAlive(now time.Time, ip net.IP)

MarkAlive makes an zombie alive and not dead. When now is later than the time set with SetCTGCTime the zombie remains alive.

func (*DNSZombieMappings) MarshalJSON 

func (zombies *DNSZombieMappings) MarshalJSON() ([]byte, error)

MarshalJSON encodes DNSZombieMappings into JSON. Only the DNSZombieMapping entries are encoded.

func (*DNSZombieMappings) SetCTGCTime 

func (zombies *DNSZombieMappings) SetCTGCTime(now time.Time)

SetCTGCTime marks the start of the most recent CT GC. This must be set after all MarkAlive calls complete to avoid a race between the DNS garbage collector and the CT GC. This would occur when a DNS zombie that has not been visited by the CT GC run is seen by a concurrent DNS garbage collector run, and then deleted. When now is later than an alive timestamp, set with MarkAlive, the zombie is no longer alive. Thus, this call acts as a gating function for what data is returned by GC.

func (*DNSZombieMappings) UnmarshalJSON 

func (zombies *DNSZombieMappings) UnmarshalJSON(raw []byte) error

UnmarshalJSON rebuilds a DNSZombieMappings from serialized JSON. It resets the AliveAt timestamps, requiring a CT GC cycle to occur before any zombies are deleted (by not being marked alive). Note: This is destructive to any currect data

func (*DNSZombieMappings) Upsert 

func (zombies *DNSZombieMappings) Upsert(now time.Time, ipStr string, qname ...string) (updatedExisting bool)

Upsert enqueues the ip -> qname as a possible deletion updatedExisting is true when an earlier enqueue existed and was updated

type NameManager 

type NameManager struct {
	lock.Mutex
	// contains filtered or unexported fields
}

NameManager maintains state DNS names, via FQDNSelector or exact match for polling, need to be tracked. It is the main structure which relates the FQDN subsystem to the policy subsystem for plumbing the relation between a DNS name and the corresponding IPs which have been returned via DNS lookups. When DNS updates are given to a NameManager it update cached selectors as required via UpdateSelectors. DNS information is cached, respecting TTL.

func NewNameManager 

func NewNameManager(config Config) *NameManager

NewNameManager creates an initialized NameManager. When config.Cache is nil, the global fqdn.DefaultDNSCache is used.

func (*NameManager) CompleteBootstrap 

func (n *NameManager) CompleteBootstrap()

func (*NameManager) ForceGenerateDNS 

func (n *NameManager) ForceGenerateDNS(ctx context.Context, namesToRegen []string) (wg *sync.WaitGroup, err error)

ForceGenerateDNS unconditionally regenerates all rules that refer to DNS names in namesToRegen. These names are FQDNs and toFQDNs.matchPatterns or matchNames that match them will cause these rules to regenerate.

func (*NameManager) GenerateSelectorUpdates added in v1.6.0 

func (n *NameManager) GenerateSelectorUpdates(fqdnSelectors map[api.FQDNSelector]struct{}) (namesMissingIPs []api.FQDNSelector, selectorIPMapping map[api.FQDNSelector][]net.IP)

GenerateSelectorUpdates iterates over all names in the DNS cache managed by gen and figures out to which FQDNSelectors managed by the cache these names map. Returns the set of FQDNSelectors which map to no IPs, and a mapping of FQDNSelectors to IPs.

func (*NameManager) GetDNSCache 

func (n *NameManager) GetDNSCache() *DNSCache

GetDNSCache returns the DNSCache used by the NameManager

func (*NameManager) GetDNSNames added in v1.6.0 

func (n *NameManager) GetDNSNames() (dnsNames []string)

GetDNSNames returns a snapshot of the DNS names managed by this NameManager

func (*NameManager) GetModel 

func (n *NameManager) GetModel() *models.NameManager

GetModel returns the API model of the NameManager.

func (*NameManager) RegisterForIdentityUpdates added in v1.6.0 

func (n *NameManager) RegisterForIdentityUpdates(selector api.FQDNSelector) []identity.NumericIdentity

RegisterForIdentityUpdates exposes this FQDNSelector so that identities for IPs contained in a DNS response that matches said selector can be propagated back to the SelectorCache via `UpdateFQDNSelector`. All DNS names contained within the NameManager's cache are iterated over to see if they match the FQDNSelector. All IPs which correspond to the DNS names which match this Selector will be returned as CIDR identities, as other DNS Names which have already been resolved may match this FQDNSelector.

func (*NameManager) UnregisterForIdentityUpdates added in v1.6.0 

func (n *NameManager) UnregisterForIdentityUpdates(selector api.FQDNSelector)

UnregisterForIdentityUpdates removes this FQDNSelector from the set of FQDNSelectors which are being tracked by the NameManager. No more updates for IPs which correspond to said selector are propagated.

func (*NameManager) UpdateDNSIPs added in v1.6.0 

func (n *NameManager) UpdateDNSIPs(lookupTime time.Time, updatedDNSIPs map[string]*DNSIPRecords) (affectedSelectors map[api.FQDNSelector]struct{}, updatedNames map[string][]net.IP)

UpdateDNSIPs updates the IPs for each DNS name in updatedDNSIPs. It returns: affectedSelectors: a set of all FQDNSelectors which match DNS Names whose corresponding set of IPs has changed. updatedNames: a map of DNS names to all the valid IPs we store for each.

func (*NameManager) UpdateGenerateDNS 

func (n *NameManager) UpdateGenerateDNS(ctx context.Context, lookupTime time.Time, updatedDNSIPs map[string]*DNSIPRecords) (wg *sync.WaitGroup, err error)

UpdateGenerateDNS inserts the new DNS information into the cache. If the IPs have changed for a name, store which rules must be updated in rulesToUpdate, regenerate them, and emit via UpdateSelectors.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.