producer

Documentation

Overview

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Code generated by "mdtogo"; DO NOT EDIT.

Index

• Variables

Variables

var BlueprintGuide = `
{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}

*Reusable, customizable components can be built and shared as blueprints.*

## Overview

Blueprints are a **pattern for developing reusable, customizable
configuration**.  Blueprints are typically published and consumed by
different teams.

{{% pageinfo color="primary" %}}
Because packages can be updated to new versions, blueprint consumers
can pull in changes to a blueprint after fetching it.
{{% /pageinfo %}}

### Example use cases for blueprints

- **Languages**: Java / Node / Ruby / Python / Golang application
- **Frameworks**: Spring, Express, Rails, Django
- **Platforms**: Kubeflow, Spark
- **Applications / Stacks**:
  - Rails Backend + Node Frontend + Prometheus
  - Spring Cloud Microservices (discovery-server, config-server, api-gateway, 
    admin-server, hystrix, various backends) 
- **Infrastructure Stacks**: CloudSQL + Pubsub + GKE

{{< svg src="images/blueprint" >}}

  # Optional: copy the mysql-kustomize blueprint to follow along
  kpt pkg get https://github.com/GoogleContainerTools/kpt.git/package-examples/mysql-kustomize mysql

## Factor / denormalize the configuration data

Structuring blueprints into separate publisher and consumer focused pieces
provides a clean UX for consumers to modifys the package.

Example: provide separate directories with pieces consumers are expected to
edit (replicas) vs publisher implementation (health check command).

As a package publisher, it is important to think about **where and how you
want to promote customization.**

We will use [kustomize] to structure the package:

1. **Factoring out a common field value**
   - Example: ` + "`" + `namespace` + "`" + `, ` + "`" + `commonLabels` + "`" + `, ` + "`" + `commonAnnotations` + "`" + `
2. **Factoring a single resource into multiple files**
   - Example: ` + "`" + `resources` + "`" + ` + ` + "`" + `patches` + "`" + `

{{% pageinfo color="info" %}}
Remote kustomize bases may be used to reference the publisher focused pieces
directly from a git repository rather than including them in the package.

One disadvantage of this approach is that it creates a dependency on the
remote package being accessible in order to push -- if you can't fetch the
remote package, then you can't push changes.
{{% /pageinfo %}}

Example package structure:

  $ tree mysql/
  mysql/
  ├── Kptfile
  ├── README.md
  ├── instance
  │   ├── kustomization.yaml
  │   ├── service.yaml
  │   └── statefulset.yaml
  └── upstream
      ├── kustomization.yaml
      ├── service.yaml
      └── statefulset.yaml

The ` + "`" + `upstream` + "`" + ` directory acts as a kustomize base to the ` + "`" + `instance` + "`" + ` directory.
Upstream contains things most **consumers are unlikely to modify** --
e.g. the image (for off the shelf software), health check endpoints, etc.

The ` + "`" + `instance` + "`" + ` directory contains patches with fields populated for things
most **consumers are expected to modify** -- e.g. namespace, cpu, memory,
user, password, etc.

{{% pageinfo color="warning" %}}
While the package is structured into publisher and consumer focused pieces,
it is still possible for the package consumer to modify (via direct edits)
or override (via patches) any part of the package.

**Factoring is for UX, not for enforcement of specific configuration values.**
{{% /pageinfo %}}

## Commands, Args and Environment Variables

How do you configure applications in a way that can be extended or overridden --
how can consumers of a package specify new args, flags, environment variables
or configuration files and merge those with those defined by the package
publisher?

### Notes

- Commands and Args are non-associative arrays so it is not possible to
  target specific elements -- any changes replace the entire list of elements.
- Commands and Args are separate fields that are concatenated
- Commands and Args can use values from environment variables
- Environment variables are associative arrays, so it is possible to target
  specific elements within the list to be overridden or added.
- Environment variables can be pulled from ConfigMaps and Secrets
- Kustomize merges ConfigMaps and Secrets per-key (deep merges of the
  values is not supported).
- ConfigMaps and Secrets can be read from apps via environment variables or
  volumes.

Flags and arguments may be factored into publisher and consumer focused pieces
by **specifying the ` + "`" + `command` + "`" + ` in the ` + "`" + `upstream` + "`" + ` base dir and the ` + "`" + `args` + "`" + ` in the 
` + "`" + `instance` + "`" + ` dir**.  This allows consumers to set and add flags using ` + "`" + `args` + "`" + `
without erasing those defined by the publisher in the ` + "`" + `command` + "`" + `.

When **specifying values for arguments or flag values, it is best to use an
environment variable read from a generated ConfigMap.**  This enables overriding
the value using kustomize's generators.

Example: Enable setting ` + "`" + `--skip-grant-tables` + "`" + ` as a flag on mysql.

*` + "`" + `# {"$ref": ...` + "`" + ` comments are setter references, defined in the next section.*

  # mysql/instance/statefulset.yaml
  # Wire ConfigMap value from kustomization.yaml to
  # an environment variable used by an arg
  apiVersion: apps/v1
  kind: StatefulSet
  ...
  spec:
    template:
      metadata:
        labels:
          app: release-name-mysql
      spec:
        containers:
        - name: mysql
          ...
          args:
          - --skip-grant-tables=$(SKIP_GRANT_TABLES)
          ...
          env:
          - name: SKIP_GRANT_TABLES
            valueFrom:
              configMapKeyRef:
                name: mysql
                key: skip-grant-tables

  # mysql/instance/kustomization.yaml
  # Changing the literal changes the StatefulSet behavior
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    # for bootstrapping the root table grants -- set to false after bootstrapped
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}

### Generating ConfigMaps and Secrets

Kustomize supports generating ConfigMaps and Secrets from the
kustomization.yaml.

- Generated objects have a suffix applied so that the name is unique for
  the data.  This ensures a rollout of Deployments and StatefulSets occurs.
- Generated object may have their values overridden by downstream consumers.

Example Upstream:

  # mysql/upstream
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    literals:
      - skip-grant-tables=true

Example Instance:

  # mysql/instance
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}
    - "mysql-user=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-user"}
    - "mysql-database=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-database"}

## Setters and Substitutions

It may be desirable to provide user friendly commands for customizing the
package rather than exclusively through text editors and sed:

- Setting a value in several different patches at once
- Setting common or required values -- e.g. the image name for a Java app
  blueprint
- Setting the image tag to match the digest of an image that was just build.
- Setting a value from the environment when the package is fetched the first
  time -- e.g. GCP project.

Setters and substitutions are a way to define user and automation friendly
commands for performing structured edits of a configuration.

Combined with the preceding techniques, setters or substitutions can be used
to modify generated ConfigMaps and patches in the ` + "`" + `instance` + "`" + ` dir.

See the [setter] and [substitution] guides for details.

  # mysql/instance/kustomization.yaml
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  #
  # namespace is the namespace the mysql instance is run in
  namespace: "" # {"$ref":"#/definitions/io.k8s.cli.setters.namespace"}
  configMapGenerator:
  - name: mysql
    behavior: merge
    literals:
    # for bootstrapping the root table grants -- set to false after bootstrapped
    - "skip-grant-tables=true" # {"$ref":"#/definitions/io.k8s.cli.substitutions.skip-grant-tables"}
    - "mysql-user=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-user"}
    - "mysql-database=" # {"$ref":"#/definitions/io.k8s.cli.substitutions.mysql-database"}
  ...

  # mysql/instance/statefulset.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: mysql
  spec:
    template:
      spec:
        containers:
        - name: mysql
  ...
          ports:
          - name: mysql
            containerPort: 3306 # {"$ref":"#/definitions/io.k8s.cli.setters.port"}
          resources:
            requests:
              cpu: 100m # {"$ref":"#/definitions/io.k8s.cli.setters.cpu"}
              memory: 256Mi # {"$ref":"#/definitions/io.k8s.cli.setters.memory"}

  # mysql/instance/service.yaml
  apiVersion: v1
  kind: Service
  ...
  spec:
    ports:
    - name: mysql
      port: 3306 # {"$ref":"#/definitions/io.k8s.cli.setters.port"}
      targetPort: mysql

## Updates

Individual directories may have their own package versions by prefixing the
version with the directory path -- e.g.
` + "`" + `package-examples/mysql-kustomize/v0.1.0` + "`" + `.

When publishing a new version of a package, publishers should think about how
their changes will be merged into existing packages.

Changing values in the instance package is not recommended, but adding them
may be ok -- changes to fields will overwrite user changes to those same
fields, whereas adds will only conflict if the user added the same field.
`

var BootstrapGuide = `` /* 1208-byte string literal not displayed */

var ContainerGuide = `
Functions may be written as container images.  The container must run a program which:

- Reads a ResourceList from STDIN
- Writes a ResourceList to STDOUT
- Exits non-0 on failure
- Writes error messages to users on STDERR

By default containers are run without network access, and without the ability to write to volumes
outside the container.

While container functions may be written in any language so long as they adhere to the io
specification (read / write ResourceList), the [kpt-functions-sdk] provides a solution for
writing functions using typescript.

## Imperative Run

Container functions may be run imperatively using the ` + "`" + `--image` + "`" + ` flag with ` + "`" + `kpt fn run` + "`" + `.  This
will create a container from the image, then write to its STDIN a ResourceList and read from
it's STDOUT a ResourceList.

## Declarative Run

Container functions may be run by declaring the function in a resource using the
` + "`" + `config.kubernetes.io/function` + "`" + ` annotation.

  apiVersion: example.com/v1beta1
  kind: ExampleKind
  metadata:
    name: function-input
    annotations:
      config.kubernetes.io/function: |
        container:
          image: gcr.io/a/b:v1

To run the declared function use ` + "`" + `kpt fn run DIR/` + "`" + ` on the directory containing the example.

## Network

By default, container functions cannot access network or volumes.  Functions may enable network
access using the ` + "`" + `--network` + "`" + ` flag, and specifying that a network is required in the functionConfig.

  apiVersion: example.com/v1beta1
  kind: ExampleKind
  metadata:
    name: function-input
    annotations:
      config.kubernetes.io/function: |
        container:
          image: gcr.io/a/b:v1
          network:
            required: true

  kpt fn run DIR/ --network
`

var ExecGuide = `
Functions may alternatively be run as executables outside of containers.  Exec
functions read input and write output the same as container functions, but are
run outside of a container.

Running functions as executables can be useful for function development, or for
running trusted executables.

Exec functions are disabled by default, and must be enabled with ` + "`" + `--enable-exec` + "`" + `.

{{% pageinfo color="info" %}}
Exec functions may be converted to container functions by building the executable
into a container and invoking it as the container ` + "`" + `CMD` + "`" + `.
{{% /pageinfo %}}

## Imperative Run

Exec functions may be run imperatively using the ` + "`" + `--exec-path` + "`" + ` flag with ` + "`" + `kpt fn run` + "`" + `.

  kpt fn run DIR/ --enable-exec --exec-path /path/to/executable

This is similar to building ` + "`" + `/path/to/executable` + "`" + ` into the container image
` + "`" + `gcr.io/project/image:tag` + "`" + ` and running -- except that the executable has access
to the local machine.

  kpt fn run DIR/ --image gcr.io/project/image:tag

Just like container functions, exec functions accept input as arguments after ` + "`" + `--` + "`" + `

  kpt fn run DIR/ --enable-exec --exec-path /path/to/executable -- foo=bar

## Declarative Run

Exec functions may also be run by declaring the function in a resource using the
` + "`" + `config.kubernetes.io/function` + "`" + ` annotation.

To run the declared function use ` + "`" + `kpt fn run DIR/ --enable-exec` + "`" + ` on the directory containing
the example.

  apiVersion: example.com/v1beta1
  kind: ExampleKind
  metadata:
    name: function-input
    annotations:
      config.kubernetes.io/function: |
        exec:
          path: /path/to/executable

Note: if the ` + "`" + `--enable-exec` + "`" + ` flag is not provided, ` + "`" + `kpt fn run DIR/` + "`" + ` will ignore the exec
function and exit 0.
`

var FunctionsGuide = `
Kpt functions are conceptually similar to Kubernetes *controllers* and *validating webhooks* --
they are programs which read resources as input, then write resources as output (creating,
modifying, deleting, or validating resources).

Unlike controllers and validating webhooks, kpt functions can be run outside of the Kubernetes
control plane.  This allows them to run in more contexts or embedded in other systems.
For example, functions could be:

- manually run locally
- automatically run locally as part of *make*, *mvn*, *go generate*, etc
- automatically run in CICD systems
- run by controllers as reconcile implementations

{{< svg src="images/fn" >}}

{{% pageinfo color="primary" %}}
Unlikely pure-templating and DSL approaches, functions must be able to both *read* and *write*
resources, and specifically should be able to read resources they have previously written --
updating the inputs rather generating new resources.

This mirrors the level-triggered controller architecture used in the Kubernetes control plane.
A function reconciles the desired state of the resource configuration to match the declared state
specified by the functionConfig.

Functions that implement abstractions should update resources they have generated in the past
by reading them from the input.
{{% /pageinfo %}}
 
The following function runtimes are available in kpt:

| Runtime    | Read Resources From | Write Resources To | Write Error Messages To | Validation Failure | Maturity |
|------------|---------------------|--------------------|-------------------------|--------------------|----------|
| Containers | STDIN               | STDOUT             | STDERR                  | Exit Code          | Beta     |
| Starlark   | ` + "`" + `ctx.resource_list` + "`" + ` | ` + "`" + `ctx.resource_list` + "`" + `| ` + "`" + `log` + "`" + `                   | Exit Code          | Alpha    |


## Input / Output

Functions read a ` + "`" + `ResourceList` + "`" + `, modify it, and write it back out.

### ResourceList.items

The ResourceList contains:
 
- (input+output) a list of resource ` + "`" + `items` + "`" + `
- (input) configuration for the function
- (output) validation results

Items are resources read from some source -- such as a package directory.

After a function adds, deletes or modifies items, the items will be written to a sink.
In most cases the sink will be the same as the source (directory).

  kind: ResourceList
  items:
  - apiVersion: apps/v1
    kind: Deployment
    spec:
    ...
  - apiVersion: v1
    kind: Service
    spec:
    ...

### ResourceList.functionConfig

Functions may optionally be configured using the ` + "`" + `ResourceList.functionConfig` + "`" + `
field.  ` + "`" + `functionConfig` + "`" + ` is analogous to a Deployment, and ` + "`" + `items` + "`" + ` 
is analogous to the set of all resources in the Deployment controller in-memory cache
(e.g. all the resources in the cluster) -- this includes the ReplicaSets created, updated
and deleted for that Deployment.

*Functions are responsible for identifying which resources from the ` + "`" + `items` + "`" + ` that they own.*

  kind: ResourceList
  functionConfig:
    apiVersion: example.com/v1alpha1
    kind: Foo
    spec:
      foo: bar
      ...
  items:
    ...

{{% pageinfo color="primary" %}}
Some functions use a ConfigMap as the functionConfig instead of introducing a new resource type
bespoke to the function.
{{% /pageinfo %}}

## Running Functions

Functions may be run either imperatively using the form ` + "`" + `kpt fn run DIR/ --image fn` + "`" + `, or they
may be run declaratively using the form ` + "`" + `kpt fn run DIR/` + "`" + `.

Either way, ` + "`" + `kpt fn run` + "`" + ` will 

1. read the package directory as input
2. encapsulate the package resources in a ` + "`" + `ResourceList` + "`" + `
3. run the function(s), providing the ResourceList as input
4. write the function(s) output back to the package directory; creating, deleting, or updating
   resources

### Imperative Run

Functions can be run imperatively by specifying the ` + "`" + `--image` + "`" + ` flag.

` + "`" + `kpt fn run DIR/ --image some-image:version` + "`" + ` will:

- create a container from the image
- read all resources from the package directory
- provide the resources as input to the function (container)
- write the output items back to the package directory

  kpt pkg get https://github.com/GoogleContainerTools/kpt-functions-sdk.git/example-configs example-configs
  mkdir results/
  kpt fn run example-configs/ --results-dir results/ --image gcr.io/kpt-functions/validate-rolebinding:results -- subject_name=bob@foo-corp.com

{{% pageinfo color="primary" %}}
If key-value pairs are provided after a ` + "`" + `--` + "`" + ` argument, then a ConfigMap will be generated
with these values as ` + "`" + `data` + "`" + ` elements, and the ConfigMap will be set as the functionConfig
field.
{{% /pageinfo %}}

### Declarative Run

Functions can be specified declaratively using the ` + "`" + `config.kubernetes.io/function` + "`" + `
annotation on a resource serving as the functionConfig.

` + "`" + `kpt fn run DIR/` + "`" + ` (without ` + "`" + `--image` + "`" + `) will:
 
- read all resources from the package directory
- for each resource with this annotation, kpt will run the specified function (using the resource as
  the functionConfig)
  - functions are run sequentially, with the output of each function provided as input
    to the next
- write the output items back to the package directory

{{% pageinfo color="primary" %}}
If multiple functions are specified in the same yaml file (e.g. separated by ` + "`" + `---` + "`" + `),
then they are run in the order they are specified.
{{% /pageinfo %}}

## Declaring Functions

Functions may be run by declaratively through the ` + "`" + `config.kubernetes.io/function` + "`" + ` annotation.

The following is a declaration to run the function implemented by the
` + "`" + `gcr.io/example.com/image:v1.0.0` + "`" + ` image, and to provide this ExampleFunction as the functionConfig.

  apiVersion: example.com/v1alpha1
  kind: ExampleFunction
  metadata:
    annotations:
      config.kubernetes.io/function: |
        container:
          image: gcr.io/example.com/image:v1.0.0

## Function Scoping

Functions are scoped to resources that live in their same directory, or subdirectories of their
directory.

` + "`" + `kpt fn run DIR/` + "`" + ` will recursively traverse DIR/ looking for declared functions and invoking
them -- passing in only those resources scoped to the function.

## Validation

### ResourceList.results

Functions may define validation results through the ` + "`" + `results` + "`" + ` field.
When functions are run using the ` + "`" + `--results-dir` + "`" + `, each function's results field will be written
to a file under the specified directory.

  kind: ResourceList
  functionConfig:
    apiVersion: example.com/v1alpha1
    kind: Foo
    spec:
      foo: bar
      ...
  results:
  - name: "kubeval"
    items:  
    - severity: error # one of ["error", "warn", "info"] -- error code should be non-0 if there are 1 or more errors
      tags: # arbitrary metadata about the result
        error-type: "field"
      message: "Value exceeds the namespace quota, reduce the value to make the pod schedulable"
      resourceRef: # key to lookup the resource
        apiVersion: apps/v1
        kind: Deployment
        name: foo
        namespace: bar
      file:
        # optional if present as annotation
        path: deploy.yaml # read from annotation if present
        # optional if present as annotation
        index: 0 # read from annotation if present
      field:
        path: "spec.template.spec.containers[3].resources.limits.cpu"
        currentValue: "200" # number | string | boolean
        suggestedValue: "2" # number | string | boolean
    - severity: warn
      ...
  - name: "something else"
    items:
    - severity: info
       ...

## Deferring Failure

When running multiple validation functions, it may be desired to defer failures until all functions
have been run so that the results of all functions are written to the results directory.
Functions can specify that failures should be deferred by specifying ` + "`" + `deferFailure` + "`" + ` in the
declaration.

  apiVersion: example.com/v1alpha1
  kind: ExampleFunction
  metadata:
    annotations:
      config.kubernetes.io/function: |
        container:
          image: gcr.io/example.com/image:v1.0.0
        deferFailure: true # continue running functions if this fails, and fail at the end

## FunctionConfig for imperative runs

When functions are run imperatively, the functionConfig may be generated from commandline arguments.

When functions are run using the form ` + "`" + `kpt fn run DIR/ --image foo:v1 -- a=b c=d` + "`" + `, the arguments
following ` + "`" + `--` + "`" + ` are parsed into a ConfigMap which is set as the functionConfig.

  kind: ResourceList
  functionConfig:
    apiVersion: v1
    kind: ConfigMap
    spec:
      a: b
      ...
  items:
    ...

If the first argument after ` + "`" + `--` + "`" + ` is *not* a key=value pair, it will be used as the
functionConfig type.

` + "`" + `kpt fn run DIR/ --image foo:v1 -- Foo a=b c=d` + "`" + `

  kind: ResourceList
  functionConfig:
    kind: Foo
    spec:
      a: b
      ...
  items:
    ...`

var GolangGuide = `
Writing exec and container functions in Golang.

### Hello World Go Function

Go libraries:

| Library | Purpose  |
|---|---|
| [sigs.k8s.io/kustomize/kyaml/fn/framework]  | Setup function command |
| [sigs.k8s.io/kustomize/kyaml/yaml]  | Modify resources  |

#### Create the go module

  go mod init github.com/user/repo
  go get sigs.k8s.io/kustomize/kyaml

#### Create the ` + "`" + `main.go` + "`" + `

  // main.go
  package main
  
  import (
  	"os"
  
  	"sigs.k8s.io/kustomize/kyaml/fn/framework"
  	"sigs.k8s.io/kustomize/kyaml/yaml"
  )
  
  var value string
  
  func main() {
      resourceList := &framework.ResourceList{}
  	cmd := framework.Command(resourceList, func() error {
          // cmd.Execute() will parse the ResourceList.functionConfig into cmd.Flags from
  		// the ResourceList.functionConfig.data field.
  		for i := range resourceList.Items {
              // modify the resources using the kyaml/yaml library:
              // https://pkg.go.dev/sigs.k8s.io/kustomize/kyaml/yaml
  			if err := resourceList.Items[i].PipeE(yaml.SetAnnotation("value", value)); err != nil {
  				return err
  			}
  		}
  		return nil
  	})
  	cmd.Flags().StringVar(&value, "value", "", "flag value")
  	if err := cmd.Execute(); err != nil {
  		os.Exit(1)
  	}
  }

### Build and test the function

Build the go binary and test it by running it as an executable function.

  go build -o my-fn .

  # run the my-fn function against the configuration in PACKAGE_DIR/
  kpt fn run PACKAGE_DIR/ --enable-exec --exec-path ./my-fn -- value=foo

### Publish the function

Build the function into a container image.

  # optional: generate a Dockerfile to contain the function
  go run ./main.go gen ./

  # build the function into an image
  docker build . -t gcr.io/project/fn-name:tag
  # optional: push the image to a container registry
  docker push gcr.io/project/fn-name:tag

Run the function as a container

  kpt fn run PACKAGE_DIR/ --image gcr.io/project/fn-name:tag -- value=foo

### Declarative function configuration

#### Run the function declaratively

Run as a container function:

  # PACKAGE_DIR/example.yaml
  apiVersion: example.com/v1alpha1
  kind: Example
  metadata:
    name: foo
    annotations:
      config.kubernetes.io/function: |
        container:
          image: gcr.io/project/fn-name:tag
  data:
    value: a

  kpt fn run PACKAGE_DIR/

Or as an exec function:

  # PACKAGE_DIR/example.yaml
  apiVersion: example.com/v1alpha1
  kind: Example
  metadata:
    name: foo
    annotations:
      config.kubernetes.io/function: |
        exec:
          path: /path/to/my-fn
  data:
    value: a

  kpt fn run PACKAGE_DIR/ --enable-exec

#### Implement the function using declarative input

Functions may alternatively be written using a struct for parsing the functionConfig rather than
flags.  The example shown below explicitly implements what the preceding example implements
implicitly.

  package main
  
  import (
  	"os"
  
  	"sigs.k8s.io/kustomize/kyaml/fn/framework"
  	"sigs.k8s.io/kustomize/kyaml/yaml"
  )
  
  func main() {
  	type Data struct {
          Value string ` + "`" + `yaml:"value,omitempty"` + "`" + `
  	}
  	type Example struct {
          // Data contains the function configuration (e.g. client-side CRD).  Using "data"
          // as the field name to contain key-value pairs enables the function to be invoked
          // imperatively via ` + "`" + `kpt fn run DIR/ --image img:tag -- key=value` + "`" + ` and the
          // key=value arguments will be parsed into the functionConfig.data field.
          // If the function does not need to be invoked imperatively, other field names
          // may be used.
          Data Data ` + "`" + `yaml:"data,omitempty"` + "`" + `
  	}
  	functionConfig := &Example{}
      resourceList := &framework.ResourceList{FunctionConfig: functionConfig}
  
  	cmd := framework.Command(resourceList, func() error {
  		for i := range resourceList.Items {
              // use the kyaml libraries to modify each resource by applying transformations
  			err := resourceList.Items[i].PipeE(
                  yaml.SetAnnotation("value", functionConfig.Data.Value),
              )
              if err != nil {
  				return nil, err
  			}
  		}
  		return items, nil
  	})
  
  	if err := cmd.Execute(); err != nil {
  		os.Exit(1)
  	}
  }

Note: functionConfig need not read from the ` + "`" + `data` + "`" + ` field if it is not going to be run
imperatively with ` + "`" + `kpt fn run DIR/ --image gcr.io/some/image -- foo=bar` + "`" + ` or 
` + "`" + `kpt fn run DIR/ --exec-path /some/bin --enable-exec -- foo=bar` + "`" + `.  This is more appropriate
for functions implementing abstractions (e.g. client-side CRD equivalents).

  ...
  	type NestedValue struct {
  		Value string ` + "`" + `yaml:"value,omitempty"` + "`" + `
  	}
  	type Spec struct {
          NestedValue string ` + "`" + `yaml:"nestedValue,omitempty"` + "`" + `
          MapValues map[string]string  ` + "`" + `yaml:"mapValues,omitempty"` + "`" + `
          ListItems []string  ` + "`" + `yaml:"listItems,omitempty"` + "`" + `
  	}
  	type Example struct {
          Spec Spec ` + "`" + `yaml:"spec,omitempty"` + "`" + `
  	}
  	functionConfig := &Example{}
  ...

  # PACKAGE_DIR/example.yaml
  apiVersion: example.com/v1alpha1
  kind: Example
  metadata:
    name: foo
    annotations:
      config.kubernetes.io/function: |
        exec:
          path: /path/to/my-fn
  spec:
    nestedValue:
      value: something
    mapValues:
      key: value
    listItems:
    - a
    - b
### kyaml

Functions written in go should use the [sigs.k8s.io/kustomize/kyaml] libraries for modifying
resource configuration.

The [sigs.k8s.io/kustomize/kyaml/yaml] library offers utilities for reading and modifying
yaml configuration, while retaining comments and structure.

To use the kyaml/yaml library, become familiar with:
 
- The ` + "`" + `*yaml.RNode` + "`" + ` type, which represents a configuration object or field
  - [link](https://pkg.go.dev/sigs.k8s.io/kustomize/kyaml/yaml?tab=doc#RNode)
- The ` + "`" + `Pipe` + "`" + ` and ` + "`" + `PipeE` + "`" + ` functions, which apply a series of pipelined operations to the ` + "`" + `*RNode` + "`" + `.
  - [link](https://pkg.go.dev/sigs.k8s.io/kustomize/kyaml/yaml?tab=doc#RNode.Pipe)


#### Workflow

To modify a *yaml.RNode call PipeE() on the *RNode, passing in the operations to be performed.

  // Set the spec.replicas field to 3 if it exists
  var node *yaml.RNode
  ...
  err := node.PipeE(yaml.Lookup("spec", "replicas"), yaml.FieldSetter{StringValue: "3"})

  // Set the spec.replicas field to 3, creating it if it doesn't exist
  var node *yaml.RNode
  ...
  // pass in the type of the node to create if it doesn't exist (e.g. Sequence, Map, Scalar)
  err := node.PipeE(yaml.LookupCreate(yaml.ScalarNode, "spec", "replicas"), yaml.FieldSetter{StringValue: "3"})

To read a value from a *yaml.RNode call Pipe() on the RNode, passing in the operations to
lookup a field.

  // Read the spec.replicas field
  var node *yaml.RNode
  ...
  replicas, err := node.Pipe(yaml.Lookup("spec", "replicas"))

{{% pageinfo color="info" %}}
Operations are any types implementing the ` + "`" + `yaml.Filter` + "`" + ` interface, so it is simple to
define custom operations and provide them to ` + "`" + `Pipe` + "`" + `, combining them with the built-in operations.
{{% /pageinfo %}}


#### Visiting Fields and Elements

Maps (i.e. Objects) and Sequences (i.e. Lists) support functions for visiting their fields and
elements.

  // Visit each of the elements in a Sequence (i.e. a List)
  err := node.VisitElements(func(elem *yaml.RNode) error {
      // do something with each element in the list
      return nil	
  })

  // Visit each of the fields in a Map (i.e. an Object)
  err := node.VisitFields(func(n *yaml.MapNode) error {
      // do something with each field in the map / object
      return nil	
  })

### Validation

Go functions can implement high fidelity validation results by setting a ` + "`" + `framework.Result` + "`" + `
on the ` + "`" + `ResourceList` + "`" + `.

If run using ` + "`" + `kpt fn run --results-dir SOME_DIR/` + "`" + `, the result will be written to a file
in the specified directory.

If the result is returned and contains an item with severity of ` + "`" + `framework.Error` + "`" + `, the function
will exit non-0.  Otherwise it will exit 0.


  cmd := framework.Command(resourceList, func() error {
      ...
      if ... {
          // return validation results to be written under the results dir
      	resourceList.Result = framework.Result{...}
  
          // return the results as an error if desireds
          return resourceList.Result
      }
      ...
  })
  
`

var InitGuide = `
A kpt package is published as a git subdirectory containing configuration
files (YAML). Publishes of kpt packages can create or generate YAML files
however they like using the tool of their choice.

Publishing a package is done by pushing the git directory
(and optionally tagging it with a version).

{{% pageinfo color="primary" %}}
Multiple packages may exist in a single repo under separate subdirectories.

Packages may be nested -- both parent (composite) and child
(component) directories may be fetched as a kpt package.

A package is versioned by tagging the git repo as one of:

- ` + "`" + `package-subdirectory/package-version` + "`" + ` (directory scoped versioning)
-  ` + "`" + `package-version` + "`" + ` (repo scoped versioning)
{{% /pageinfo %}}

{{< svg src="images/producer-guide" >}}

## Steps

1. [Create a git repo](#create-a-git-repo)
2. [Create the package contents](#create-the-package)
2. [Create configuration](#create-configuration)
3. [Publish package to git](#publish-package-to-git)


## Create a git repo

  git clone REPO_URL # or create a new repo with ` + "`" + `git init` + "`" + `
  cd REPO_NAME

## Create the package

  mkdir nginx

Recommended: initialize the package with metadata

  kpt pkg init nginx --tag kpt.dev/app=nginx --description "kpt nginx package"

## Create configuration

  $ curl https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/controllers/nginx-deployment.yaml --output nginx/nginx-deployment.yaml

## Publish package to git

  git add .
  git commit -m "Add nginx package"

Recommended: tag the commit as a release

  # tag as DIR/VERSION for per-directory versioning
  git tag nginx/v0.1.0
  git push nginx/v0.1.0`

var ProducerGuide = `{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}


{{< svg src="images/producer-guide" >}}`

var SettersGuide = `Setters provide a solution for template-free setting or substitution of field
values through package metadata (OpenAPI).  They are a safer alternative to
other substitution techniques which do not have the context of the
structured data -- e.g. using ` + "`" + `sed` + "`" + ` to replace values.

The OpenAPI definitions for setters are defined in a Kptfile and referenced by
a fields through comments on the fields.

Setters may be invoked to programmatically modify the configuration
using ` + "`" + `kpt cfg set` + "`" + ` to set and/or substitute values.

{{% pageinfo color="primary" %}}
Creating a setter requires that the package has a Kptfile.  If one does
not exist for the package, run ` + "`" + `kpt pkg init DIR/` + "`" + ` to create one.
{{% /pageinfo %}}

## Setters explained

Following is a short explanation of the command that will be demonstrated
in this guide.

### Data model

- Fields reference setters through OpenAPI definitions specified as
  line comments -- e.g. ` + "`" + `# { "$ref": "#/definitions/..." }` + "`" + `
- OpenAPI definitions are provided through the Kptfile

### Command control flow

1. Read the package Kptfile and resources.
2. Change the setter OpenAPI value in the Kptfile
3. Locate all fields which reference the setter and change their values.
4. Write both the modified Kptfile and resources back to the package.

{{< svg src="images/set-command" >}}

#### Creating a Setter

Setters may be created either manually (by editing the Kptfile directly), or
programmatically (through the ` + "`" + `create-setter` + "`" + ` command).  The ` + "`" + `create-setter` + "`" + `
command will:

1. create a new OpenAPI definition for a setter in the Kptfile
2. create references to the setter definition on the resource fields

  # Kptfile -- original
  openAPI:
    definitions: {}

  # deployment.yaml -- original
  kind: Deployment
  metadata:
    name: foo
  spec:
    replicas: 3

  # create or update a setter named "replicas"
  # match fields with the value "3"
  kpt cfg create-setter hello-world/ replicas 3

  # Kptfile -- updated
  openAPI:
    definitions:
      io.k8s.cli.setters.replicas:
        x-k8s-cli:
          setter:
            name: "replicas"
            value: "3"

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    replicas: 3 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

#### Invoking a Setter

  # deployment.yaml -- original
  kind: Deployment
  metadata:
   name: helloworld-gke
   labels:
     app: hello
  spec:
   replicas: 3 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

  # set the replicas field to 5
  kpt cfg set DIR/ replicas 5

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
   name: helloworld-gke
   labels:
     app: hello
  spec:
   replicas: 5 # {"$ref":"#/definitions/io.k8s.cli.setters.replicas"}

#### Types

Setters may have types specified which ensure that the configuration is always
serialized correctly as yaml 1.1 -- e.g. if a string field such as an
annotation or arg has the value "on", then it would need to be quoted otherwise
it will be parsed as a bool by yaml 1.1.

This may be done by modifying the Kptfile OpenAPI definitions as shown here:

  openAPI:
    definitions:
      io.k8s.cli.setters.version:
        x-k8s-cli:
          setter:
            name: "version"
            value: "3"
        type: string

Set would change the configuration like this:

  kind: Deployment
  metadata:
    name: foo
    annotations:
      version: "3" # {"$ref":"#/definitions/io.k8s.cli.setters.version"}

#### OpenAPI Validations
Users can input any additional validation constraints during ` + "`" + `create-setter` + "`" + `
operation in the form of openAPI schema. Relevant openAPI specification
constraints can be provided in json file format. The ` + "`" + `set` + "`" + ` operation validates
the input value against provided schema during setter creation and throws an
error if the input value doesn't meet any of the constraints. 

  $ cat /path/to/file.json
  {"maximum": 10, "type": "integer"}
  
  # create setter with openAPI property constraints
  kpt cfg create-setter replicas 3 --schema-path /path/to/file.json

The command creates setter with the following definition

  openAPI:
    definitions:
      io.k8s.cli.setters.replicas:
        maximum: 10
        type: integer
        x-k8s-cli:
          setter:
            name: replicas
            value: "3"

  # try to set value not adhering to the constraints
  kpt cfg set replicas 11

  error: The input value doesn't validate against provided OpenAPI schema:
  validation failure list: replicas in body should be less than or equal to 10

  Relevant fixtures from openAPI JSON schema suite
  
  {"minLength", "maxLength", "pattern", "type", "minimum", "maximum", "multipleOf",
  "enum", "maxItems", "minItems", "format", "patternProperties", "uniqueItems",
  "allOf", "not", "oneOf", "anyOf"}

#### Setting Lists

It is possible to create setters for fields which are a list of strings.
The setter type must be ` + "`" + `array` + "`" + `, and the reference must be on the list field.
The list setter will take variable args for its value rather than a single value.

**Note:** Currently ` + "`" + `create-setter` + "`" + ` will not directly create a setter reference for a
list field.  The simplest way to create a list setter is to create a setter for one of
the elements, and then move the reference to the list field.


  # example.yaml
  apiVersion: example.com/v1beta1
  kind: Example
  spec:
    list:
    - "a"

  # Kptfile
  kind: Kptfile


` + "`" + `$ kpt cfg create-setter --type array . list a` + "`" + `

**Note:** Move the setter reference from the element (` + "`" + `- "a"` + "`" + `) to the list (` + "`" + `list: ` + "`" + `)

  # example.yaml
  apiVersion: example.com/v1beta1
  kind: Example
  spec:
    list: # {"$ref":"#/definitions/io.k8s.cli.setters.list"}
    - "a"

  # Kptfile
  kind: Kptfile
  openAPI:
    definitions:
      io.k8s.cli.setters.list:
        type: array
        x-k8s-cli:
          setter:
            name: list
            listValues:
            - "a"

` + "`" + `$ kpt cfg set . list a b c` + "`" + `

  # example.yaml
  apiVersion: example.com/v1beta1
  kind: Example
  spec:
    list: # {"$ref":"#/definitions/io.k8s.cli.setters.list"}
    - "a"
    - "b"
    - "c"

  # Kptfile
  kind: Kptfile
  openAPI:
    definitions:
      io.k8s.cli.setters.list:
        type: array
        x-k8s-cli:
          setter:
            name: list
            listValues:
            - "a"
            - "b"
            - "c"

#### Enumerations

Setters may be configured to map an enum input to a different value set
in the configuration.

e.g. users set ` + "`" + `small` + "`" + `, ` + "`" + `medium` + "`" + `, ` + "`" + `large` + "`" + ` cpu sizes, and these are mapped
to numeric values set in the configuration.

This may be done by modifying the Kptfile OpenAPI definitions as shown here:

  openAPI:
    definitions:
      io.k8s.cli.setters.cpu:
        x-k8s-cli:
          setter:
            name: "cpu"
            value: "small"
            # enumValues will replace the user provided key with the
            # map value when setting fields.
            enumValues:
              small: "0.5"
              medium: "2"
              large: "4"

Set would change the configuration like this:

  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: foo
      resources:
        requests:
          cpu: "0.5" # {"$ref":"#/definitions/io.k8s.cli.setters.cpu"}`

var StarlarkGuide = `
{{% pageinfo color="warning" %}}
The Starlark runtime is Alpha, and must be enabled with the ` + "`" + `--enable-star` + "`" + ` flag.
{{% /pageinfo %}}

Functions may be written as Starlark scripts which modify a ResourceList provided as
a variable.

#### Imperative Run

Starlark functions can be run imperatively by providing the Starlark script as a flag
on ` + "`" + `kpt fn run` + "`" + `.  Following is an example of a Starlark function which adds a "foo" annotation
to every resource in the package.

  # c.star
  # set the foo annotation on each resource
  def run(r, an):
    for resource in r:
      # mutate the resource
      resource["metadata"]["annotations"]["foo"] = an
  
  # get the value of the annotation to add
  an = ctx.resource_list["functionConfig"]["data"]["value"]
  
  run(ctx.resource_list["items"], an)

Run the Starlark function with:

  # run c.star as a function, generating a ConfigMap with value=bar as the functionConfig
  kpt fn run . --enable-star --star-path c.star -- value=bar

Any resource under ` + "`" + `.` + "`" + ` will have the ` + "`" + `foo: bar` + "`" + ` annotation added.

#### Declarative Run

Starlark functions can also be run declaratively using the ` + "`" + `config.kubernetes.io/function` + "`" + `
annotation.  This annotation indicates that the resource is functionConfig that should
be provided to a function.

Following is an example of a Starlark function which adds a "foo" annotation to
each resource in its package.  The ExampleKind resource will be set as the
ResourceList.functionConfig.

  # example.yaml
  apiVersion: example.com/v1beta1
  kind: ExampleKind
  metadata:
    name: function-input
    annotations:
      config.kubernetes.io/function: |
        starlark: {path: c.star, name: example-name}
  spec:
    value: "hello world"

Example Starlark function to which will add an annotation to each resource scoped to ` + "`" + `example.yaml` + "`" + `
(those under the directory containing ` + "`" + `example.yaml` + "`" + `):

  # c.star
  # set the foo annotation on each resource
  def run(r, an):
    for resource in r:
      resource["metadata"]["annotations"]["foo"] = an
  
  an = ctx.resource_list["functionConfig"]["spec"]["value"]
  run(ctx.resource_list["items"], an)

Run them on the directory containing ` + "`" + `example.yaml` + "`" + ` using:

  kpt fn run DIR/ --enable-star

## Debugging Functions

It is possible to debug Starlark functions using ` + "`" + `print` + "`" + `

  # c.star
  print(ctx.resource_list["items"][0]["metadata"]["name"])

  kpt fn run . --enable-star --star-path c.star

> foo

## OpenAPI

The OpenAPI known to kpt is provided to the Starlark program through the ` + "`" + `ctx.open_api` + "`" + ` variable.
This may contain metadata about the resources and their types.

  #c.star
  print(ctx.open_api["definitions"]["io.k8s.api.apps.v1.Deployment"]["description"])

  kpt fn run . --enable-star --star-path c.star

> Deployment enables declarative updates for Pods and ReplicaSets.

## Retaining YAML Comments

While Starlark programs are unable to retain comments on resources, kpt will
attempt to retain comments by copying them from the function inputs to the function outputs.

It is not possible at this time to add, modify or delete comments from Starlark scripts.`

var SubstitutionsGuide = `

Substitutions provide a solution for template-free substitution of field values
built on top of [setters].  They enable substituting values into part of a
field, including combining multiple setters into a single value.

Much like setters, substitutions are defined using OpenAPI.

Substitutions may be invoked to programmatically modify the configuration
using ` + "`" + `kpt cfg set` + "`" + ` to substitute values which are derived from the setter.

Substitutions are computed by substituting setter values into a pattern.
They are composed of 2 parts: a pattern and a list of values.

- The pattern is a string containing markers which will be replaced with
  1 or more setter values.
- The values are pairs of markers and setter references.  The *set* command
  retrieves the values from the referenced setters, and replaces the markers
  with the setter values.

{{% pageinfo color="primary" %}}
Creating a substitution requires that the package has a Kptfile.  If one does
not exist for the package, run ` + "`" + `kpt pkg init DIR/` + "`" + ` to create one.
{{% /pageinfo %}}

## Substitutions explained

Following is a short explanation of the command that will be demonstrated
in this guide.

### Data model

- Fields reference substitutions through OpenAPI definitions specified as
  line comments -- e.g. ` + "`" + `# { "$ref": "#/definitions/..." }` + "`" + `
- OpenAPI definitions are provided through the Kptfile
- Substitution OpenAPI definitions contain patterns and values to compute
  the field value

### Command control flow

1. Read the package Kptfile and resources.
2. Change the setter OpenAPI value in the Kptfile
3. Locate all fields which reference the setter indirectly through a 
   substitution.
4. Compute the new substitution value by substituting the setter values into
   the pattern.
5. Write both the modified Kptfile and resources back to the package.

{{< svg src="images/substitute-command" >}}

## Creating a Substitution

Substitution may be created either manually (by editing the Kptfile directly),
or programmatically (with ` + "`" + `create-subst` + "`" + `).  The ` + "`" + `create-subst` + "`" + ` command will:

1. Create a new OpenAPI definition for a substitution in the Kptfile
2. Create references to the substitution OpenAPI definition on the resource
   fields

### Example

  # Kptfile -- original
  openAPI:
    definitions: {}

  # deployment.yaml -- original
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

  # create an image substitution and a setter that populates it
  kpt cfg create-subst hello-world/ image-value --field-value nginx:1.7.9 \
    --pattern nginx:\${TAG_SETTER}

  # Kptfile -- updated
  openAPI:
    definitions:
      io.k8s.cli.setters.TAG_SETTER:
        x-k8s-cli:
          setter:
            name: TAG_SETTER
            value: 1.7.9
      io.k8s.cli.substitutions.image-value:
        x-k8s-cli:
          substitution:
            name: image-value
            pattern: nginx:${TAG_SETTER}
            values:
            - marker: ${TAG_SETTER}
              ref: '#/definitions/io.k8s.cli.setters.TAG_SETTER'

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.7.9 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

This substitution defines how the value for a field may be produced by
substituting the ` + "`" + `tag` + "`" + ` setter value into the pattern.

Any time the ` + "`" + `tag` + "`" + ` value is changed via the *set* command, then then
substitution value will be re-calculated for referencing fields.

## Creation semantics

By default create-subst will create referenced setters if they do not already
exist.  It will infer the current setter value from the pattern and value.

If setters already exist before running the create-subst command, then those
setters are used and left unmodified.

If a setter does not exist and create-subst cannot infer the setter value,
then it will throw and error, and the setter must be manually created.

## Invoking a Substitution

Substitutions are invoked by running ` + "`" + `kpt cfg set` + "`" + ` on a setter used by the
substitution.

  kpt cfg set hello-world/ TAG_SETTER 1.8.1

  # deployment.yaml -- updated
  kind: Deployment
  metadata:
    name: foo
  spec:
    template:
      spec:
        containers:
        - name: nginx
          image: nginx:1.8.1 # {"$ref":"#/definitions/io.k8s.cli.substitutions.image-value"}

{{% pageinfo color="primary" %}}
When setting a field through a substitution, the names of the setters
are used *not* the name of the substitution.  The name of the substitution is
*only used in the configuration field references*.
{{% /pageinfo %}}

`

var TsGuide = `` /* 145-byte string literal not displayed */

var VariantGuide = `
{{% pageinfo color="warning" %}}
# Notice: Under Development
{{% /pageinfo %}}`