dubber

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 22, 2023 License: Apache-2.0 Imports: 33 Imported by: 0

README

WARNING dubber is in it's infancy. It may not even be a great idea at all.

Dubber

A tool for provisioning DNS records based on dynamic config from various sources.

What Does Dubber Do?

Dubber queries various sources of information (currently Marathon, or Kubernetes) for the state of tasks and services running within them.

This state is then passed to user supplied templates (text/template with github.com/Masterminds/sprig) enabled).

Output of the template is parsed as RFC 1035 Zone file content using github.com/miekg.

The resulting zone file is then passed to DNS provisioners (currently route53 and gcloud DNS), which reconcile the provided zone content with the running zone.

Template can use the comments on a DNS record to pass hints to the provisioner.

Dubber will only delete records under the following two circumstances:

  • A record of the same Name, Class, RR Type and Grouping Flag values (see below), already exists, and has a different value.
  • No records of the same Name, Class, RR Type and Grouping Flag values (see below), were request, and all the flags mentioned in the ownerFlags in the provisioner configuration are set, and match the configured regexp of each flag.

So by settings some grouping flag on a record, (e.g. route53.SetID), to a value that can be specific to a given instance of dubber, you can then allow dubber to delete records that match that specific value, if they are no longer needed.

Record Flags

Dubber uses DNS comments to translate into non-traditional DNS options supported by the provisioners. Some flags are used to group records so that conflicting records within a group can be remove/replaced, but conflicting records in different groups are treated a separate entitied.

route53
  • route53.SetID: (Grouping Flag) Associate these records with a Set
  • route53.Weight: Set a weight for the set
  • route53.Alias: "HOSTEDZONEID:ALIASNAME"
  • route53.EvalTargetHealth: "true" will enable target health evaluation
GCloud DNS

TBD

An example

discoverers:
  kubernetes:
    - template: |
      {{ $cluster := env `CLUSTER` }}
      ; From Ingresses
      {{- range $ing := .Ingresses }}
        {{- $setID := or (index $ing.ObjectMeta.Labels `route53SetId`) $cluster }}
        {{- $weight := or (index $ing.ObjectMeta.Labels `route53Weight`) `0` }}
        {{- if len $ing.Status.LoadBalancer.Ingress }}
        {{-   $sing := index $ing.Status.LoadBalancer.Ingress 0 }}
        {{-   range $rule := $ing.Spec.Rules }}
        {{-     if $rule.Host }}
        {{-       if $sing.IP }}
      {{ $rule.Host }} 60 A {{$sing.IP}}; route53.SetID={{ $setID }} route53.Weight={{ $weight }}
        {{-       else }}
      {{ $rule.Host }} CNAME {{ $sing.Hostname }};
        {{-       end }}
        {{-     end }}
        {{-   end }}
        {{- end  }}
      {{- end }}
  marathon:
    - endpoints:
      -  http://marathon.mesos.example.com/api
      template: |
        {{- $publicLB := `Z1234:dualstack.exanple-public.elb.amazonaws.com.`}}
        {{- $privateLB := `Z1234:dualstack.exanple-private.elb.amazonaws.com.`}}
        {{- range .Applications }}
        {{-   if (index .Labels `dnsName`) }}
        {{-     if eq (index .Labels `externalAccess`) `public` }}
        {{ index .Labels `dnsName`}} 0 A 0.0.0.0 ; route53.Alias={{$publicLB}}
        {{-     else }}
        {{ index .Labels `dnsName`}} 0 A 0.0.0.0 ; route53.Alias={{$privateLB}}
        {{-     end }}
        {{-   end }}
        {{- end }}

provisioners:
  route53:
    - zone: example.com.
      zoneid: Z56789
      ownerFlags:
        "route53.SetID": "{{ env `CLUSTER` }}"

Will create a route53 alias record, based on the dnsName and externalAccess labels on a Narathon task.

Different disoveres can provide different data, and any number of records can be created for different elements.

TODO

  • Watch rather than poll
  • Possibly unify all data and pass it to a single template, rather than each discoverer having it's own template.
  • Template functions to help build the records.
  • Purging - possibly track the state of the records and allow purging of anything we created (this is somewhat achievable via the ownerFlags)

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BaseDiscovererConfig 

type BaseDiscovererConfig struct {
	Disabled bool         `yaml:"disabled" json:"disabled"`
	Template JSONTemplate `yaml:"template" json:"template"`
}

BaseDiscovererConfig is configuration common to all discoverers

type BaseProvisionerConfig 

type BaseProvisionerConfig struct {
	Zone           string                  `yaml:"zone" json:"zone"`
	OwnerFlagsStrs map[string]JSONTemplate `yaml:"ownerFlags"`
	// contains filtered or unexported fields
}

BaseProvisionerConfig is the configuration that is common to all provisioners

func (*BaseProvisionerConfig) OwnerFlags added in v0.6.0 

func (bp *BaseProvisionerConfig) OwnerFlags() (map[string]*regexp.Regexp, error)

type ByRR 

type ByRR Zone

ByRR is a Zone ordered by it's resource records.

func (ByRR) Compare 

func (z ByRR) Compare(i, j int) int

Compare compares two elements in a Zone.

func (ByRR) Dedupe 

func (z ByRR) Dedupe() ByRR

Dedupe z , z must already be sorted.

func (ByRR) Len 

func (z ByRR) Len() int

Len implements Sorter for Zone

func (ByRR) Less 

func (z ByRR) Less(i, j int) bool

Less implements Sorter for Zone

func (ByRR) Swap 

func (z ByRR) Swap(i, j int)

Swap implements Sorter for Zone

type Config 

type Config struct {
	Discoverers struct {
		Marathon   []MarathonConfig   `yaml:"marathon" json:"marathon"`
		Kubernetes []KubernetesConfig `yaml:"kubernetes" json:"kubernetes"`
	} `yaml:"discoverers" json:"discoverers"`
	Provisioners struct {
		Route53   []Route53Config   `yaml:"route53" json:"route53"`
		GCloudDNS []GCloudDNSConfig `yaml:"gcloud" json:"gcloud"`
	} `yaml:"provisioners" json:"provisioners"`

	XXX `json:",omitempty" yaml:",omitempty,inline"`

	DryRun       bool          `json:"-"  yaml:"-"`
	OneShot      bool          `json:"-"  yaml:"-"`
	PollInterval time.Duration `json:"-"  yaml:"-"`
}

Config describes the base configuration for dubber

func FromYAML 

func FromYAML(r io.Reader) (Config, error)

FromYAML creates a dubber config from a YAML config file

func (Config) BuildDiscoveres 

func (cfg Config) BuildDiscoveres() ([]Discoverer, error)

BuildDiscoveres returns the set of discoveres for this config

func (Config) BuildProvisioners 

func (cfg Config) BuildProvisioners() (map[string]Provisioner, error)

BuildProvisioners returns the set of provisioners for this config

type Discoverer 

type Discoverer struct {
	StatePuller
	State interface{}
	JSONTemplate
}

Discoverer combined zone data and state into a Zone

func (*Discoverer) Discover 

func (d *Discoverer) Discover(ctx context.Context) (Zone, error)

Discover pulls the state from a StatePuller and renders the state into Zone data

type GCloudDNS added in v0.5.3 

type GCloudDNS struct {
	GCloudDNSConfig
	// contains filtered or unexported fields
}

GCloudDNS is an Google Cloud DNS provider.

func NewGCloudDNS added in v0.5.3 

func NewGCloudDNS(cfg GCloudDNSConfig) *GCloudDNS

NewGCloudDNS creates a gcloud dns provisioner.

func (*GCloudDNS) GroupFlags added in v0.5.3 

func (r *GCloudDNS) GroupFlags() []string

GroupFlags is empty for GCloud DNS

func (*GCloudDNS) RemoteZone added in v0.5.3 

func (r *GCloudDNS) RemoteZone() (Zone, error)

RemoteZone creates a Zone from a GCloudDNS Zone.

func (*GCloudDNS) UpdateZone added in v0.5.3 

func (r *GCloudDNS) UpdateZone(wanted, unwanted, desired, remote Zone) error

UpdateZone updates a GCloudDNS zone, removing the unwanted records, and adding any unwanted records.

type GCloudDNSConfig added in v0.5.3 

type GCloudDNSConfig struct {
	BaseProvisionerConfig `json:",omitempty,inline" yaml:",omitempty,inline"`
	Project               string `yaml:"project" json:"project"`
	ZoneID                string `yaml:"zoneID" json:"zoneID"`
}

GCloudDNSConfig describes the settings required for controlling a Google Cloud DNS zone.

type JSONTemplate 

type JSONTemplate struct {
	*template.Template
}

JSONTemplate provides a means of directly unmarshaling a template

func (JSONTemplate) MarshalJSON 

func (t JSONTemplate) MarshalJSON() ([]byte, error)

MarshalJSON implements the yaml Marshaler interface for JSON template

func (JSONTemplate) MarshalYAML 

func (t JSONTemplate) MarshalYAML() (interface{}, error)

MarshalYAML implements the yaml Marshaler interface for JSON template

func (*JSONTemplate) UnmarshalJSON 

func (t *JSONTemplate) UnmarshalJSON(bs []byte) error

UnmarshalJSON implements the yaml Unmarshaler interface for JSON Regex

func (*JSONTemplate) UnmarshalYAML 

func (t *JSONTemplate) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements the yaml Unmarshaler interface for JSON Regex

type Kubernetes 

type Kubernetes struct {
	*kubernetes.Clientset

	sync.Mutex
	// contains filtered or unexported fields
}

Kubernetes implements discovery of applications and dns names from https://github.com/mesosphere/marathon

func NewKubernetes 

func NewKubernetes(cfg KubernetesConfig) (*Kubernetes, error)

NewKubernetes creates a new marathon discoverer

func (*Kubernetes) StatePull 

func (m *Kubernetes) StatePull(ctx context.Context) (State, error)

StatePull watches, or polls, marathon for new applications. Any matching the requires constraints are returned. THe first call to Discover returns all the known apps, Subsequent calls block until an individial update is found.

type KubernetesConfig 

type KubernetesConfig struct {
	BaseDiscovererConfig `json:",omitempty" yaml:",omitempty,inline"`
	FileName             string `json:"kubeconfig" yaml:"kubeconfig"`
	Context              string `json:"context" yaml:"context"`
	XXX                  `json:",omitempty" yaml:",omitempty,inline"`
}

KubernetesConfig describes configuration options for the marathon discoverer.

type KubernetesState 

type KubernetesState struct {
	Nodes     map[string]v1.Node
	Ingresses map[string]netv1.Ingress
	Services  map[string]v1.Service
	Endpoints map[string]v1.Endpoints
}

KubernetesState holds the state information we will pass to the configuration template.

type Marathon 

type Marathon struct {
	marathon.Marathon

	sync.Mutex
	// contains filtered or unexported fields
}

Marathon implements discovery of applications and dns names from https://github.com/mesosphere/marathon

func NewMarathon 

func NewMarathon(cfg MarathonConfig) (*Marathon, error)

NewMarathon creates a new marathon discoverer

func (*Marathon) StatePull 

func (m *Marathon) StatePull(ctx context.Context) (State, error)

StatePull watches, or polls, marathon for new applications. Any matching the requires constraints are returned. THe first call to Discover returns all the known apps, Subsequent calls block until an individial update is found.

type MarathonConfig 

type MarathonConfig struct {
	BaseDiscovererConfig `json:",omitempty" yaml:",omitempty,inline"`
	Endpoint             []string `json:"endpoints" yaml:"endpoints"`
	BasicAuth            struct {
		Username string `json:"username" yaml:"username"`
		Password string `json:"password" yaml:"password"`
	} `json:"basic_auth" yaml:"basic_auth"`
	XXX `json:",omitempty" yaml:",omitempty,inline"`
}

MarathonConfig describes configuration options for the marathon discoverer.

type MarathonState 

type MarathonState struct {
	Applications map[string]marathon.Application
	Tasks        map[string]marathon.Task
}

MarathonState holds the state information we will pass to the configuration template.

type Provisioner 

type Provisioner interface {
	RemoteZone() (Zone, error)
	UpdateZone(wanted, unwanted, desired, remote Zone) error
	GroupFlags() []string
	OwnerFlags() (map[string]*regexp.Regexp, error)
}

A Provisioner can manage a zone. RemoteZone should include exactly 1 SOA record. It is assumed that Zones do not change without that Serial Number being changed. In the event that records must be added/removed from the Zone retuned by RemoteZone, UpdateZone will be called with the relevant changes, plus an update to the SOA record. It is assumed that an update will fail if the SOA serial from the remote list does not match the SOA of the current remote zone state.

type Record 

type Record struct {
	dns.RR
	Flags RecordFlags
}

Record represents a DNS record we wish to be present, along with a Flags string which may contain hints to the provisioner

func (*Record) Compare 

func (r *Record) Compare(r2 *Record) int

Compare two Records

func (*Record) String 

func (r *Record) String() string

String implements fmt.Stringer for a Record

type RecordFlags 

type RecordFlags map[string]string

RecordFlags is a set of KV pairs, parsed from the comments of a record. They are used to pass hints to the provisioners.

func ParseRecordFlags 

func ParseRecordFlags(str string) (RecordFlags, error)

ParseRecordFlags parses simple K=V pairs from a comment on a record. Any bare words are included and are assumed to have a "" value.

func (RecordFlags) Compare 

func (rf RecordFlags) Compare(rf2 RecordFlags) int

Compare two sets of RecordFlags. Retursn > 0 if mthere are more flags in rf2 than rf. If the number of flags is the same, the String representations are compared.

func (RecordFlags) String 

func (rf RecordFlags) String() string

String implements Stringer for a RecordFlags, rendering the strings in sorted order

type RecordSetKey 

type RecordSetKey struct {
	Name       string
	Class      uint16
	Rrtype     uint16
	GroupFlags string
}

RecordSetKey is used to group records by name, type and class, along with any grouping keys.

type Route53 

type Route53 struct {
	sync.Mutex
	Route53Config
	// contains filtered or unexported fields
}

Route53 is an AWS Route53 DNS record provisioner. This provision uses the following flags:

  • route53.SetID: Associate these records with a Set
  • route53.Weight: Set a weight for the set
  • route53.Alias: "HOSTEDZONEID:ALIASNAME"
  • route53.EvalTargetHealth: "true" will enable target health evaluation on an alias

func NewRoute53 

func NewRoute53(cfg Route53Config) *Route53

NewRoute53 creates a route53 provisioner. Currently this uses the default client setup from the aws-sdk.

func (*Route53) GroupFlags 

func (r *Route53) GroupFlags() []string

RemoteZone creates a Zone from an AWS Route53 Hosted Zone.

func (*Route53) RemoteZone 

func (r *Route53) RemoteZone() (Zone, error)

RemoteZone creates a Zone from an AWS Route53 Hosted Zone.

func (*Route53) UpdateZone 

func (r *Route53) UpdateZone(wanted, unwanted, desired, remote Zone) error

UpdateZone updates a Route53 zone, removing the unwanted records, and adding any wanted records.

type Route53Config 

type Route53Config struct {
	BaseProvisionerConfig `json:",omitempty,inline" yaml:",omitempty,inline"`
	ZoneID                string `json:"zoneid,omitempty" yaml:"zoneid,omitempty"`
}

Route53Config is used to provide settings for a Route53 provisioner. If the ZoneID is not set it will be looked up in the clients default region.

type Server 

type Server struct {
	*http.ServeMux
	*prometheus.Registry
	MetricActiveDicoverers      prometheus.Gauge
	MetricDiscovererRuns        *prometheus.CounterVec
	MetricDiscoveredZoneSerial  *prometheus.GaugeVec
	MetricProvisionedZoneSerial *prometheus.GaugeVec
	MetricReconcileRuns         *prometheus.CounterVec
	MetricReconcileTimes        *prometheus.HistogramVec
	// contains filtered or unexported fields
}

Server wraps the configuration and basic functionality.

func New 

func New(cfg *Config) *Server

New creates a new dubber server.

func (*Server) ReconcileZone 

func (srv *Server) ReconcileZone(p Provisioner, desired Zone) error

ReconcileZone attempts to ensure that the set of records in the desired zone are present in the Provisioner's zone.

  • Records are grouped by Name.
  • Records from the provisioner that are not listed in the desired set are ignored.
  • Records of a given "Name, Type , Class" combination that are in the remote zone, but not in the desired zone are removed.
  • Records of a given "Name, Type , Class" combination that are in the desired zone, but not in the remote zone are added.

func (*Server) Run 

func (srv *Server) Run(ctx context.Context) error

Run process the configuration, passing updates form discoverers, managing state, and request action from provisioners.

type State 

type State interface{}

State is any data passed from the discoverer to the template to generate DNS records

type StatePuller 

type StatePuller interface {
	StatePull(context.Context) (State, error)
}

StatePuller reads the state from the remote service. The call should block until an updated state is available

type XXX 

type XXX map[string]interface{}

XXX catches unknown Rule settings

type Zone 

type Zone []*Record

Zone is a collection of related Records

func ParseZoneData 

func ParseZoneData(r io.Reader) (Zone, error)

ParseZoneData parses the text from the provided reader into zone data. All errors encountered during parsing are collected into the err response.

func (Zone) Diff 

func (z Zone) Diff(z2 Zone) (Zone, Zone, Zone)

Diff enumerates the differences between two zones. Both zones should be sorted before calling. The first return argument are those items only in the original zone The Second return argument are those items common to both zones The Third return argument are those items present only in the argument zone

func (Zone) FindSet 

func (z Zone) FindSet(name string, class uint16, rrtype uint16) Zone

FindSet finds the set of records matching the provided name, class and type

func (Zone) Group 

func (z Zone) Group(groupFlags []string) map[RecordSetKey]Zone

Group all the records by Name,Class and Type, and a set of grouping flags.

func (Zone) Partition 

func (z Zone) Partition(domains []string) map[string]Zone

Partition splits a zones data into separate zones based on a list of domains. Records are assigned to the longest matching domain.

func (Zone) String 

func (z Zone) String() string

String renders the text version of the Zone data

type ZoneError 

type ZoneError []error

ZoneError is the set of errors seen when parsing Zone data

func (ZoneError) Error 

func (z ZoneError) Error() string

Error implenebts the Error interface for a set of errors found when parsing a zone.

Source Files

View all Source files

Directories

Path Synopsis
cmd
dubber

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.