producer

Documentation

Overview

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Index

• Variables

Variables

var BlueprintGuide = `
{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}

*Reusable, customizable components can be built and shared as blueprints.*

## Overview

Blueprints are a **pattern for developing reusable, customizable
configuration**.  Blueprints are typically published and consumed by
different teams.

{{% pageinfo color="primary" %}}
Because packages can be updated to new versions, blueprint consumers
can pull in changes to a blueprint after fetching it.
{{% /pageinfo %}}

### Example use cases for blueprints

- **Languages**: Java / Node / Ruby / Python / Golang application
- **Frameworks**: Spring, Express, Rails, Django
- **Platforms**: Kubeflow, Spark
- **Applications / Stacks**:
  - Rails Backend + Node Frontend + Prometheus
  - Spring Cloud Microservices (discovery-server, config-server, api-gateway, 
    admin-server, hystrix, various backends) 
- **Infrastructure Stacks**: CloudSQL + Pubsub + GKE

{{< svg src="images/blueprint" >}}

  # Optional: copy the mysql-kustomize blueprint to follow along
  kpt pkg get https://github.com/GoogleContainerTools/kpt.git/package-examples/mysql-kustomize mysql

## Factor / denormalize the configuration data

Structuring blueprints into separate publisher and consumer focused pieces
provides a clean UX for consumers to modifys the package.

Example: provide separate directories with pieces consumers are expected to
edit (replicas) vs publisher implementation (health check command).

As a package publisher, it is important to think about **where and how you
want to promote customization.**

We will use [kustomize] to structure the package:

1. **Factoring out a common field value**
   - Example: ` + "`" + `namespace` + "`" + `, ` + "`" + `commonLabels` + "`" + `, ` + "`" + `commonAnnotations` + "`" + `
2. **Factoring a single resource into multiple files**
   - Example: ` + "`" + `resources` + "`" + ` + ` + "`" + `patches` + "`" + `

{{% pageinfo color="info" %}}
Remote kustomize bases may be used to reference the publisher focused pieces
directly from a git repository rather than including them in the package.

One disadvantage of this approach is that it creates a dependency on the
remote package being accessible in order to push -- if you can't fetch the
remote package, then you can't push changes.
{{% /pageinfo %}}

Example package structure:

  $ tree mysql/
  mysql/
  ├── Kptfile
  ├── README.md
  ├── instance
  │   ├── kustomization.yaml
  │   ├── service.yaml
  │   └── statefulset.yaml
  └── upstream
      ├── kustomization.yaml
      ├── service.yaml
      └── statefulset.yaml

The ` + "`" + `upstream` + "`" + ` directory acts as a kustomize base to the ` + "`" + `instance` + "`" + ` directory.
Upstream contains things most **consumers are unlikely to modify** --
e.g. the image (for off the shelf software), health check endpoints, etc.

The ` + "`" + `instance` + "`" + ` directory contains patches with fields populated for things
most **consumers are expected to modify** -- e.g. namespace, cpu, memory,
user, password, etc.

{{% pageinfo color="warning" %}}
While the package is structured into publisher and consumer focused pieces,
it is still possible for the package consumer to modify (via direct edits)
or override (via patches) any part of the package.

**Factoring is for UX, not for enforcement of specific configuration values.**
{{% /pageinfo %}}

## Commands, Args and Environment Variables

How do you configure applications in a way that can be extended or overridden --
how can consumers of a package specify new args, flags, environment variables
or configuration files and merge those with those defined by the package
publisher?

### Notes

- Commands and Args are non-associative arrays so it is not possible to
  target specific elements -- any changes replace the entire list of elements.
- Commands and Args are separate fields that are concatenated
- Commands and Args can use values from environment variables
- Environment variables are associative arrays, so it is possible to target
  specific elements within the list to be overridden or added.
- Environment variables can be pulled from ConfigMaps and Secrets
- Kustomize merges ConfigMaps and Secrets per-key (deep merges of the
  values is not supported).
- ConfigMaps and Secrets can be read from apps via environment variables or
  volumes.

Flags and arguments may be factored into publisher and consumer focused pieces
by **specifying the ` + "`" + `command` + "`" + ` in the ` + "`" + `upstream` + "`" + ` base dir and the ` + "`" + `args` + "`" + ` in the 
` + "`" + `instance` + "`" + ` dir**.  This allows consumers to set and add flags using ` + "`" + `args` + "`" + `
without erasing those defined by the publisher in the ` + "`" + `command` + "`" + `.

When **specifying values for arguments or flag values, it is best to use an
environment variable read from a generated ConfigMap.**  This enables overriding
the value using kustomize's generators.

Example: Enable setting ` + "`" + `--skip-grant-tables` + "`" + ` as a flag on mysql.

*` + "`" + `# {"$ref": ...` + "`" + ` comments are setter references, defined in the next section.*

  # mysql/instance/statefulset.yaml
  # Wire ConfigMap value from kustomization.yaml to
  # an environment variable used by an arg
  apiVersion: apps/v1
  kind: StatefulSet
  ...
  spec:
    template:
      metadata:
        labels:
          app: release-name-mysql
      spec:
        containers:
        - name: mysql
          ...
          args:
          - --skip-grant-tables=$(SKIP_GRANT_TABLES)
          ...
          env:
          - name: SKIP_GRANT_TABLES
            valueFrom:
              configMapKeyRef:
                name: mysql
                key: skip-grant-tables

  # mysql/instance/kustomization.yaml
  # Changing the literal changes the StatefulSet behavior
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    # for bootstrapping the root table grants -- set to false after bootstrapped
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}

### Generating ConfigMaps and Secrets

Kustomize supports generating ConfigMaps and Secrets from the
kustomization.yaml.

- Generated objects have a suffix applied so that the name is unique for
  the data.  This ensures a rollout of Deployments and StatefulSets occurs.
- Generated object may have their values overridden by downstream consumers.

Example Upstream:

  # mysql/upstream
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    literals:
      - skip-grant-tables=true

Example Instance:

  # mysql/instance
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}
    - "mysql-user=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-user"}
    - "mysql-database=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-database"}

## Setters and Substitutions

It may be desirable to provide user friendly commands for customizing the
package rather than exclusively through text editors and sed:

- Setting a value in several different patches at once
- Setting common or required values -- e.g. the image name for a Java app
  blueprint
- Setting the image tag to match the digest of an image that was just build.
- Setting a value from the environment when the package is fetched the first
  time -- e.g. GCP project.

Setters and substitutions are a way to define user and automation friendly
commands for performing structured edits of a configuration.

Combined with the preceding techniques, setters or substitutions can be used
to modify generated ConfigMaps and patches in the ` + "`" + `instance` + "`" + ` dir.

See the [setter] and [substitution] guides for details.

  # mysql/instance/kustomization.yaml
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  #
  # namespace is the namespace the mysql instance is run in
  namespace: "" # {"$ref":"#/definitions/io.k8s.cli.setters.namespace"}
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    # for bootstrapping the root table grants -- set to false after bootstrapped
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}
    - "mysql-user=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-user"}
    - "mysql-database=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-database"}
  ...

  # mysql/instance/statefulset.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: mysql
  spec:
    template:
      spec:
        containers:
        - name: mysql
  ...
          ports:
          - name: mysql
            containerPort: 3306 # {"$ref":"#/definitions/io.k8s.cli.setters.port"}
          resources:
            requests:
              cpu: 100m # {"$ref":"#/definitions/io.k8s.cli.setters.cpu"}
              memory: 256Mi # {"$ref":"#/definitions/io.k8s.cli.setters.memory"}

  # mysql/instance/service.yaml
  apiVersion: v1
  kind: Service
  ...
  spec:
    ports:
    - name: mysql
      port: 3306 # {"$ref":"#/definitions/io.k8s.cli.setters.port"}
      targetPort: mysql

## Updates

Individual directories may have their own package versions by prefixing the
version with the directory path -- e.g.
` + "`" + `package-examples/mysql-kustomize/v0.1.0` + "`" + `.

When publishing a new version of a package, publishers should think about how
their changes will be merged into existing packages.

Changing values in the instance package is not recommended, but adding them
may be ok -- changes to fields will overwrite user changes to those same
fields, whereas adds will only conflict if the user added the same field.
`

var BootstrapGuide = `` /* 1208-byte string literal not displayed */

var InitGuide = `
A kpt package is published as a git subdirectory containing configuration
files (YAML). Publishes of kpt packages can create or generate YAML files
however they like using the tool of their choice.

Publishing a package is done by pushing the git directory
(and optionally tagging it with a version).

{{% pageinfo color="primary" %}}
Multiple packages may exist in a single repo under separate subdirectories.

Packages may be nested -- both parent (composite) and child
(component) directories may be fetched as a kpt package.

A package is versioned by tagging the git repo as one of:

- ` + "`" + `package-subdirectory/package-version` + "`" + ` (directory scoped versioning)
-  ` + "`" + `package-version` + "`" + ` (repo scoped versioning)
{{% /pageinfo %}}

{{< svg src="images/producer-guide" >}}

## Steps

1. [Create a git repo](#create-a-git-repo)
2. [Create the package contents](#create-the-package)
2. [Create configuration](#create-configuration)
3. [Publish package to git](#publish-package-to-git)


## Create a git repo

  git clone REPO_URL # or create a new repo with ` + "`" + `git init` + "`" + `
  cd REPO_NAME

## Create the package

  mkdir nginx

Recommended: initialize the package with metadata

  kpt pkg init nginx --tag kpt.dev/app=nginx --description "kpt nginx package"

## Create configuration

  $ curl https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/controllers/nginx-deployment.yaml --output nginx/nginx-deployment.yaml

## Publish package to git

  git add .
  git commit -m "Add nginx package"

Recommended: tag the commit as a release

  # tag as DIR/VERSION for per-directory versioning
  git tag nginx/v0.1.0
  git push nginx/v01.0`

var ProducerGuide = `{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}


{{< svg src="images/producer-guide" >}}`

var SettersGuide = `Setters provide a solution for template-free setting or substitution of field
values through package metadata (OpenAPI).  They are a safer alternative to
other substitution techniques which do not have the context of the
structured data -- e.g. using ` + "`" + `sed` + "`" + ` to replace values.

The OpenAPI definitions for setters are defined in a Kptfile and referenced by
a fields through comments on the fields.

Setters may be invoked to programmatically modify the configuration
using ` + "`" + `kpt cfg set` + "`" + ` to set and/or substitute values.

{{% pageinfo color="primary" %}}
Creating a setter requires that the package has a Kptfile.  If one does
not exist for the package, run ` + "`" + `kpt pkg init DIR/` + "`" + ` to create one.
{{% /pageinfo %}}

## Setters explained

Following is a short explanation of the command that will be demonstrated
in this guide.

### Data model

- Fields reference setters through OpenAPI definitions specified as
  line comments -- e.g. ` + "`" + `# { "$ref": "#/definitions/..." }` + "`" + `
- OpenAPI definitions are provided through the Kptfile

### Command control flow

1. Read the package Kptfile and resources.
2. Change the setter OpenAPI value in the Kptfile
3. Locate all fields which reference the setter and change their values.
4. Write both the modified Kptfile and resources back to the package.

{{< svg src="images/set-command" >}}

#### Creating a Setter

Setters may be created either manually (by editing the Kptfile directly), or
programmatically (through the ` + "`" + `create-setter` + "`" + ` command).  The ` + "`" + `create-setter` + "`" + `
command will:

1. create a new OpenAPI definition for a setter in the Kptfile
2. create references to the setter definition on the resource fields

  # Kptfile -- original
  openAPI:
    definitions: {}

  # deployment.yaml -- original
  kind: Deployment
  metadata:
    name: foo
  spec:
    replicas: 3

  # create or update a setter named "replicas"
  # match fields with the value "3"
  kpt cfg create-setter hello-world/ replicas 3

  # Kptfile -- updated
  openAPI:
    definitions:
      io.k8s.cli.setters.replicas:
        x-k8s-cli:
          setter:
            name: "replicas"
            value: "3"

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    replicas: 3 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

#### Invoking a Setter

  # deployment.yaml -- original
  kind: Deployment
  metadata:
   name: helloworld-gke
   labels:
     app: hello
  spec:
   replicas: 3 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

  # set the replicas field to 5
  kpt cfg set DIR/ replicas 5

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
   name: helloworld-gke
   labels:
     app: hello
  spec:
   replicas: 5 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

#### Types

Setters may have types specified which ensure that the configuration is always
serialized correctly as yaml 1.1 -- e.g. if a string field such as an
annotation or arg has the value "on", then it would need to be quoted otherwise
it will be parsed as a bool by yaml 1.1.

This may be done by modifying the Kptfile OpenAPI definitions as shown here:

  openAPI:
    definitions:
      io.k8s.cli.setters.version:
        x-k8s-cli:
          setter:
            name: "version"
            value: "3"
        type: string

Set would change the configuration like this:

  kind: Deployment
  metadata:
    name: foo
    annotations:
      version: "3" # {"$ref":"#/definitions/io.k8s.cli.setters.version"}

#### Enumerations

Setters may be configured to map an enum input to a different value set
in the configuration.

e.g. users set ` + "`" + `small` + "`" + `, ` + "`" + `medium` + "`" + `, ` + "`" + `large` + "`" + ` cpu sizes, and these are mapped
to numeric values set in the configuration.

This may be done by modifying the Kptfile OpenAPI definitions as shown here:

  openAPI:
    definitions:
      io.k8s.cli.setters.cpu:
        x-k8s-cli:
          setter:
            name: "cpu"
            value: "small"
            # enumValues will replace the user provided key with the
            # map value when setting fields.
            enumValues:
              small: "0.5"
              medium: "2"
              large: "4"

Set would change the configuration like this:

  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: foo
      resources:
        requests:
          cpu: "0.5" # {"$ref":"#/definitions/io.k8s.cli.setters.cpu"}`

var SubstitutionsGuide = `

Substitutions provide a solution for template-free substitution of field values
built on top of [setters].  They enable substituting values into part of a
field, including combining multiple setters into a single value.

Much like setters, substitutions are defined using OpenAPI.

Substitutions may be invoked to programmatically modify the configuration
using ` + "`" + `kpt cfg set` + "`" + ` to substitute values which are derived from the setter.

Substitutions are computed by substituting setter values into a pattern.
They are composed of 2 parts: a pattern and a list of values.

- The pattern is a string containing markers which will be replaced with
  1 or more setter values.
- The values are pairs of markers and setter references.  The *set* command
  retrieves the values from the referenced setters, and replaces the markers
  with the setter values.

{{% pageinfo color="primary" %}}
Creating a substitution requires that the package has a Kptfile.  If one does
not exist for the package, run ` + "`" + `kpt pkg init DIR/` + "`" + ` to create one.
{{% /pageinfo %}}

## Substitutions explained

Following is a short explanation of the command that will be demonstrated
in this guide.

### Data model

- Fields reference substitutions through OpenAPI definitions specified as
  line comments -- e.g. ` + "`" + `# { "$ref": "#/definitions/..." }` + "`" + `
- OpenAPI definitions are provided through the Kptfile
- Substitution OpenAPI definitions contain patterns and values to compute
  the field value

### Command control flow

1. Read the package Kptfile and resources.
2. Change the setter OpenAPI value in the Kptfile
3. Locate all fields which reference the setter indirectly through a 
   substitution.
4. Compute the new substitution value by substituting the setter values into
   the pattern.
5. Write both the modified Kptfile and resources back to the package.

{{< svg src="images/substitute-command" >}}

## Creating a Substitution

Substitution may be created either manually (by editing the Kptfile directly),
or programmatically (with ` + "`" + `create-subst` + "`" + `).  The ` + "`" + `create-subst` + "`" + ` command will:

1. Create a new OpenAPI definition for a substitution in the Kptfile
2. Create references to the substitution OpenAPI definition on the resource
   fields

### Example

  # Kptfile -- original
  openAPI:
    definitions: {}

  # deployment.yaml -- original
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

  # create an image substitution and a setter that populates it
  kpt cfg create-subst hello-world/ image-tag nginx:1.7.9 \
    --pattern nginx:TAG_SETTER --value TAG_SETTER=tag

  # Kptfile -- updated
  openAPI:
    definitions:
      io.k8s.cli.setters.tag:
        x-k8s-cli:
          setter:
            name: "tag"
            value: "1.7.9"
      io.k8s.cli.substitutions.image-value:
        x-k8s-cli:
          substitution:
            name: image-value
            pattern: nginx:TAG_SETTER
            values:
            - marker: TAG_SETTER
              ref: '#/definitions/io.k8s.cli.setters.tag'

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

This substitution defines how the value for a field may be produced by
substituting the ` + "`" + `tag` + "`" + ` setter value into the pattern.

Any time the ` + "`" + `tag` + "`" + ` value is changed via the *set* command, then then
substitution value will be re-calculated for referencing fields.

## Creation semantics

By default create-subst will create referenced setters if they do not already
exist.  It will infer the current setter value from the pattern and value.

If setters already exist before running the create-subst command, then those
setters are used and left unmodified.

If a setter does not exist and create-subst cannot infer the setter value,
then it will throw and error, and the setter must be manually created.

## Invoking a Substitution

Substitutions are invoked by running ` + "`" + `kpt cfg set` + "`" + ` on a setter used by the
substitution.

  kpt cfg set hello-world/ image 1.8.1

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.8.1 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

{{% pageinfo color="primary" %}}
When setting a field through a substitution, the names of the setters
are used *not* the name of the substitution.  The name of the substitution is
*only used in the configuration field references*.
{{% /pageinfo %}}

`

var VariantGuide = `
{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}`