xconnect

README

xconnect - declaring microservices connectivity

Xconnect is a structure in YAML that describes how a service can accept connections and what connections a service needs to operate. With this information, a static overview of the landscape of intercconnected services can be constructed.

specification in YAML

how does it work

Every application/service uses some kind of configuration to specifiy what other services is connects to. This can be a database, another service, a filesystem etc. By standardising a part of that configuration, a tool can extract connectivity information from that configuration. This is called the xconnect section.

xonnect section

The xconnect section of a configuration (now YAML only), consists of 3 parts:

• description of what a service can provide by connecting to it, the listen part.
• description of what a service needs to consume or produce to using a connection, the connect part
• metadata about the service such as version, ownership and custom labels.

Not all application configuration is related to connectivity ; the section for connectivity can be part of it. So instead of keeping connection related information separate from the rest of the configuration, with the inevitable effect of becoming out of date, the xconnect section should be integrated in the complete configuration. The actual format of the xconnect data is free-form YAML, meaning that users are free to add their own service metadata if desired.

Example

xconnect:
  meta:
    # name for discovery
    name: account-service
    # tagged version of the implementation
    version: v1.2.3
    # team that owns the code and operates it
    owner: team accounts
    labels:
      - account
      - registration
      - search    
  listen:
    api:
      host: localhost
      scheme: grpc
      port: 9443
    web:
      host: localhost
      scheme: http
      tls: true
      port: 443
  connect:
    some-db:
      url: jdbc:postgresql://localhost:5432/postgres?reWriteBatchedInserts=true
    some-cache:
      host: #REDIS_IP
      port: 6379
    variant-publish:
      gcp.pubsub:
        topic: VariantToAssortment_Push_v1-topic          
    variant-pull:
      gcp.pubsub:
        subscription: Variant_v1-subscription
        test:
          topic: Variant_v1-topic

Use as Go package

import ( "github.com/emicklei/xconnect" )

This example uses gopkg.in/yaml.v2 for parsing the configuration.

content, err := ioutil.ReadFile("your-app.yaml")
var doc xconnect.Document
err := yaml.Unmarshal(content, &doc)
cfg := doc.Config

version := cfg.Meta.Version
// alternative: cfg.Meta.FindString("version")

webPort := cfg.Listen["web"].Port
// alternative: cfg.Listen["web"].FindInt("port")

variantPullTestTopic := cfg.Connect["variant-pull"].FindString("gcp.pubsub/test/topic")

Sprint Boot application configration

A Spring configuration needs it own root element in a YAML file. To use an xconnect section in this file, relevant property values should be referenced using the ${..} notation supported by Spring.

xconnect:
  meta: 
  listen:
  connect:
    some-db:
      url: jdbc:postgresql://localhost:5432/postgres?reWriteBatchedInserts=true

spring:
  datasource:
    # use a reference to the actual value, within this document
    url: ${xconnect.connect.some-db.url}

extract

To extract the xconnect section using the command line tool:

xconnect -input application.yml -target file://xconnect-from-application.yml

Kubernetes configration (ConfigMap)

A Kubernetes configuration as its defined structure in a YAML file. To use an xconnect section in this file, its content must be inside the data field.

apiVersion: v1
data:
    xconnect:
        meta: 
            ...
        listen:
            ...
        connect:
            ...
        
kind: ConfigMap
metadata:
    creationTimestamp: null
    name: some
    namespace: somewhere

extract

To extract the xconnect section using the command line tool:

xconnect -input configmap.yml -k8s -target file://xconnect-from-configmap.yml

Getting the extra fields

See xconnect_test.go

Inspiration

• https://dzone.com/articles/cataloguing-microservices

© 2019, ernestmicklei.com. MIT License. Contributions welcome.

Documentation

Index

• type Config
• type ConnectEntry
  • func (e ConnectEntry) FindInt(path string) int
  • func (e ConnectEntry) FindString(path string) string
• type Document
• type K8SConfiguration
  • func (k K8SConfiguration) ExtractConfig() (x Config, err error)
• type ListenEntry
  • func (e ListenEntry) FindInt(path string) int
  • func (e ListenEntry) FindString(path string) string
• type MetaConfig
  • func (m MetaConfig) FindString(path string) string

Types

type Config

type Config struct {
	Meta    MetaConfig              `yaml:"meta" json:"meta"`
	Listen  map[string]ListenEntry  `yaml:"listen" json:"listen"`
	Connect map[string]ConnectEntry `yaml:"connect" json:"connect"`
}

Config represents the xconnect data section of a YAML document. See spec-xconnect.yaml.

type ConnectEntry

type ConnectEntry struct {
	Scheme      string                 `yaml:"scheme,omitempty" json:"scheme,omitempty"`
	TLS         *bool                  `yaml:"tls,omitempty" json:"tls,omitempty"`
	Host        string                 `yaml:"host,omitempty" json:"host,omitempty"`
	Port        *int                   `yaml:"port,omitempty" json:"port,omitempty"`
	URL         string                 `yaml:"url,omitempty" json:"url,omitempty"`
	Disabled    bool                   `yaml:"disabled,omitempty" json:"disabled,omitempty"`
	ExtraFields map[string]interface{} `yaml:"-,inline"`
}

ConnectEntry is a list element in the xconnect.connect config.

func (ConnectEntry) FindInt

func (e ConnectEntry) FindInt(path string) int

FindInt returns an int for a given path (using slashes).

func (ConnectEntry) FindString

func (e ConnectEntry) FindString(path string) string

FindString return a string for a give dotted path.

type Document

type Document struct {
	Config Config `yaml:"xconnect"`
}

Document is the root YAML element

type K8SConfiguration

type K8SConfiguration struct {
	APIVersion string                 `yaml:"apiVersion"`
	Data       map[string]interface{} `yaml:"data"`
	Kind       string                 `yaml:"kind" `
	Metadata   struct {
		Name              string    `yaml:"name" `
		Namespace         string    `yaml:"namespace"`
		CreationTimestamp time.Time `yaml:"creationTimestamp"`
	} `yaml:"metadata"`
}

K8SConfiguration represents a Kubernetes configuration.

func (K8SConfiguration) ExtractConfig

func (k K8SConfiguration) ExtractConfig() (x Config, err error)

ExtractConfig expects a "xconnect" key in the data map and parses that part into a xconnect.Config.

type ListenEntry

type ListenEntry struct {
	Scheme      string                 `yaml:"scheme,omitempty" json:"scheme,omitempty"`
	Host        string                 `yaml:"host,omitempty" json:"host,omitempty"`
	Port        *int                   `yaml:"port,omitempty" json:"port,omitempty"`
	TLS         *bool                  `yaml:"tls,omitempty" json:"tls,omitempty"`
	Disabled    bool                   `yaml:"disabled,omitempty" json:"disabled,omitempty"`
	ExtraFields map[string]interface{} `yaml:"-,inline"`
}

ListenEntry is a list element in the xconnect.accept config.

func (ListenEntry) FindInt

func (e ListenEntry) FindInt(path string) int

FindInt returns an int for a given path (using slashes).

func (ListenEntry) FindString

func (e ListenEntry) FindString(path string) string

FindString return a string for a given path (using slashes).

type MetaConfig

type MetaConfig struct {
	Name        string                 `yaml:"name,omitempty" json:"name,omitempty"`
	Version     string                 `yaml:"version,omitempty" json:"version,omitempty"`
	Owner       string                 `yaml:"owner,omitempty" json:"owner,omitempty"`
	Labels      []string               `yaml:"labels,omitempty" json:"labels,omitempty"`
	ExtraFields map[string]interface{} `yaml:"-,inline"`
}

MetaConfig represents the meta element in the xconnect data section.

func (MetaConfig) FindString

func (m MetaConfig) FindString(path string) string

FindString return a string for a given slash path.