kio

package
v0.13.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 1, 2022 License: Apache-2.0 Imports: 22 Imported by: 244

Documentation

Overview

Package kio contains libraries for reading and writing collections of Resources.

Reading Resources

Resources are Read using a kio.Reader function. Examples: 

[kio.LocalPackageReader{}, kio.ByteReader{}]

Resources read using a LocalPackageReader will have annotations applied so they can be written back to the files they were read from.

Modifying Resources

Resources are modified using a kio.Filter. The kio.Filter accepts a collection of Resources as input, and returns a new collection as output. It is recommended to use the yaml package for manipulating individual Resources in the collection.

Writing Resources

Resources are Read using a kio.Reader function. Examples: 

[kio.LocalPackageWriter{}, kio.ByteWriter{}]

ReadWriters

It is preferred to use a ReadWriter when reading and writing from / to the same source.

Building Pipelines

The preferred way to transforms a collection of Resources is to use kio.Pipeline to Read, Modify and Write the collection of Resources. Pipeline will automatically sequentially invoke the Read, Modify, Write steps, returning and error immediately on any failure.

Package kio contains low-level libraries for reading, modifying and writing Resource Configuration and packages.

Example 
package main

import (
	"bytes"
	"log"
	"os"

	"sigs.k8s.io/kustomize/kyaml/kio"
	"sigs.k8s.io/kustomize/kyaml/yaml"
)

func main() {
	input := bytes.NewReader([]byte(`apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
`))

	// setAnnotationFn
	setAnnotationFn := kio.FilterFunc(func(operand []*yaml.RNode) ([]*yaml.RNode, error) {
		for i := range operand {
			resource := operand[i]
			_, err := resource.Pipe(yaml.SetAnnotation("foo", "bar"))
			if err != nil {
				return nil, err
			}
		}
		return operand, nil
	})

	err := kio.Pipeline{
		Inputs:  []kio.Reader{&kio.ByteReader{Reader: input}},
		Filters: []kio.Filter{setAnnotationFn},
		Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
	}.Execute()
	if err != nil {
		log.Fatal(err)
	}

}

Output:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
  annotations:
    foo: 'bar'
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  annotations:
    foo: 'bar'
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

Index

Examples

Constants

View Source 
const (
	ResourceListKind       = "ResourceList"
	ResourceListAPIVersion = "config.kubernetes.io/v1"
)

Variables

View Source 
var DefaultMatch = []string{"*.yaml", "*.yml"}
View Source 
var GraphStructures = []string{string(TreeStructureGraph), string(TreeStructurePackage)}
View Source 
var JSONMatch = []string{"*.json"}
View Source 
var MatchAll = append(DefaultMatch, JSONMatch...)

Functions

func FromBytes added in v0.10.2 

func FromBytes(bs []byte) ([]*yaml.RNode, error)

FromBytes reads from a byte slice.

func ParseAll added in v0.10.1 

func ParseAll(inputs ...string) ([]*yaml.RNode, error)

ParseAll reads all of the inputs into resources

func PreprocessResourcesForInternalAnnotationMigration added in v0.13.1 

func PreprocessResourcesForInternalAnnotationMigration(result []*yaml.RNode) (map[string]map[string]string, error)

PreprocessResourcesForInternalAnnotationMigration returns a mapping from id to all internal annotations, so that we can use it to reconcile the annotations later. This is necessary because currently both internal-prefixed annotations and legacy annotations are currently supported, and a change to one must be reflected in the other if needed.

func ReconcileInternalAnnotations added in v0.13.1 

func ReconcileInternalAnnotations(result []*yaml.RNode, nodeAnnosMap map[string]map[string]string) error

ReconcileInternalAnnotations reconciles the annotation format for path, index and id annotations. It will ensure the output annotation format matches the format in the input. e.g. if the input format uses the legacy format and the output will be converted to the legacy format if it's not.

func StringAll added in v0.10.1 

func StringAll(resources []*yaml.RNode) (string, error)

StringAll writes all of the resources to a string

Types

type ByteReadWriter 

type ByteReadWriter struct {
	// Reader is where ResourceNodes are decoded from.
	Reader io.Reader

	// Writer is where ResourceNodes are encoded.
	Writer io.Writer

	// OmitReaderAnnotations will configures Read to skip setting the config.kubernetes.io/index
	// annotation on Resources as they are Read.
	OmitReaderAnnotations bool

	// KeepReaderAnnotations if set will keep the Reader specific annotations when writing
	// the Resources, otherwise they will be cleared.
	KeepReaderAnnotations bool

	// PreserveSeqIndent if true adds kioutil.SeqIndentAnnotation to each resource
	PreserveSeqIndent bool

	// Style is a style that is set on the Resource Node Document.
	Style yaml.Style

	// WrapBareSeqNode wraps the bare sequence node document with map node,
	// kyaml uses reader annotations to track resources, it is not possible to
	// add them to bare sequence nodes, this option enables wrapping such bare
	// sequence nodes into map node with key yaml.BareSeqNodeWrappingKey
	// note that this wrapping is different and not related to ResourceList wrapping
	WrapBareSeqNode bool

	FunctionConfig *yaml.RNode

	Results *yaml.RNode

	NoWrap             bool
	WrappingAPIVersion string
	WrappingKind       string
}

ByteReadWriter reads from an input and writes to an output.

func (*ByteReadWriter) Read 

func (rw *ByteReadWriter) Read() ([]*yaml.RNode, error)

func (*ByteReadWriter) Write 

func (rw *ByteReadWriter) Write(nodes []*yaml.RNode) error

type ByteReader 

type ByteReader struct {
	// Reader is where ResourceNodes are decoded from.
	Reader io.Reader

	// OmitReaderAnnotations will configures Read to skip setting the config.kubernetes.io/index
	// and internal.config.kubernetes.io/seqindent annotations on Resources as they are Read.
	OmitReaderAnnotations bool

	// PreserveSeqIndent if true adds kioutil.SeqIndentAnnotation to each resource
	PreserveSeqIndent bool

	// SetAnnotations is a map of caller specified annotations to set on resources as they are read
	// These are independent of the annotations controlled by OmitReaderAnnotations
	SetAnnotations map[string]string

	FunctionConfig *yaml.RNode

	Results *yaml.RNode

	// DisableUnwrapping prevents Resources in Lists and ResourceLists from being unwrapped
	DisableUnwrapping bool

	// WrappingAPIVersion is set by Read(), and is the apiVersion of the object that
	// the read objects were originally wrapped in.
	WrappingAPIVersion string

	// WrappingKind is set by Read(), and is the kind of the object that
	// the read objects were originally wrapped in.
	WrappingKind string

	// WrapBareSeqNode wraps the bare sequence node document with map node,
	// kyaml uses reader annotations to track resources, it is not possible to
	// add them to bare sequence nodes, this option enables wrapping such bare
	// sequence nodes into map node with key yaml.BareSeqNodeWrappingKey
	// note that this wrapping is different and not related to ResourceList wrapping
	WrapBareSeqNode bool

	// AnchorsAweigh set to true attempts to replace all YAML anchor aliases
	// with their definitions (anchor values) immediately after the read.
	AnchorsAweigh bool
}

ByteReader decodes ResourceNodes from bytes. By default, Read will set the config.kubernetes.io/index annotation on each RNode as it is read so they can be written back in the same order.

func (*ByteReader) Read 

func (r *ByteReader) Read() ([]*yaml.RNode, error)

type ByteWriter 

type ByteWriter struct {
	// Writer is where ResourceNodes are encoded.
	Writer io.Writer

	// KeepReaderAnnotations if set will keep the Reader specific annotations when writing
	// the Resources, otherwise they will be cleared.
	KeepReaderAnnotations bool

	// ClearAnnotations is a list of annotations to clear when writing the Resources.
	ClearAnnotations []string

	// Style is a style that is set on the Resource Node Document.
	Style yaml.Style

	// FunctionConfig is the function config for an ResourceList.  If non-nil
	// wrap the results in an ResourceList.
	FunctionConfig *yaml.RNode

	Results *yaml.RNode

	// WrappingKind if set will cause ByteWriter to wrap the Resources in
	// an 'items' field in this kind.  e.g. if WrappingKind is 'List',
	// ByteWriter will wrap the Resources in a List .items field.
	WrappingKind string

	// WrappingAPIVersion is the apiVersion for WrappingKind
	WrappingAPIVersion string

	// Sort if set, will cause ByteWriter to sort the the nodes before writing them.
	Sort bool
}

ByteWriter writes ResourceNodes to bytes. Generally YAML encoding will be used but in the special case of writing a single, bare yaml.RNode that has a kioutil.PathAnnotation indicating that the target is a JSON file JSON encoding is used. See shouldJSONEncodeSingleBareNode below for more information.

func (ByteWriter) Write 

func (w ByteWriter) Write(inputNodes []*yaml.RNode) error

type Filter 

type Filter interface {
	Filter([]*yaml.RNode) ([]*yaml.RNode, error)
}

Filter modifies a collection of Resource Configuration by returning the modified slice. When possible, Filters should be serializable to yaml so that they can be described as either data or code.

Analogous to http://www.linfo.org/filters.html

func FilterAll 

func FilterAll(filter yaml.Filter) Filter

FilterAll runs the yaml.Filter against all inputs

type FilterFunc 

type FilterFunc func([]*yaml.RNode) ([]*yaml.RNode, error)

FilterFunc implements a Filter as a function.

func (FilterFunc) Filter 

func (fn FilterFunc) Filter(o []*yaml.RNode) ([]*yaml.RNode, error)

type LocalPackageReadWriter 

type LocalPackageReadWriter struct {
	Kind string `yaml:"kind,omitempty"`

	KeepReaderAnnotations bool `yaml:"keepReaderAnnotations,omitempty"`

	// PreserveSeqIndent if true adds kioutil.SeqIndentAnnotation to each resource
	PreserveSeqIndent bool

	// PackagePath is the path to the package directory.
	PackagePath string `yaml:"path,omitempty"`

	// PackageFileName is the name of file containing package metadata.
	// It will be used to identify package.
	PackageFileName string `yaml:"packageFileName,omitempty"`

	// MatchFilesGlob configures Read to only read Resources from files matching any of the
	// provided patterns.
	// Defaults to ["*.yaml", "*.yml"] if empty.  To match all files specify ["*"].
	MatchFilesGlob []string `yaml:"matchFilesGlob,omitempty"`

	// IncludeSubpackages will configure Read to read Resources from subpackages.
	// Subpackages are identified by presence of PackageFileName.
	IncludeSubpackages bool `yaml:"includeSubpackages,omitempty"`

	// ErrorIfNonResources will configure Read to throw an error if yaml missing missing
	// apiVersion or kind is read.
	ErrorIfNonResources bool `yaml:"errorIfNonResources,omitempty"`

	// OmitReaderAnnotations will cause the reader to skip annotating Resources with the file
	// path and mode.
	OmitReaderAnnotations bool `yaml:"omitReaderAnnotations,omitempty"`

	// SetAnnotations are annotations to set on the Resources as they are read.
	SetAnnotations map[string]string `yaml:"setAnnotations,omitempty"`

	// NoDeleteFiles if set to true, LocalPackageReadWriter won't delete any files
	NoDeleteFiles bool `yaml:"noDeleteFiles,omitempty"`

	// FileSkipFunc is a function which returns true if reader should ignore
	// the file
	FileSkipFunc LocalPackageSkipFileFunc

	// FileSystem can be used to mock the disk file system.
	FileSystem filesys.FileSystemOrOnDisk

	// WrapBareSeqNode wraps the bare sequence node document with map node,
	// kyaml uses reader annotations to track resources, it is not possible to
	// add them to bare sequence nodes, this option enables wrapping such bare
	// sequence nodes into map node with key yaml.BareSeqNodeWrappingKey
	// note that this wrapping is different and not related to ResourceList wrapping
	WrapBareSeqNode bool
	// contains filtered or unexported fields
}

LocalPackageReadWriter reads and writes Resources from / to a local directory. When writing, LocalPackageReadWriter will delete files if all of the Resources from that file have been removed from the output.

func (*LocalPackageReadWriter) Read 

func (r *LocalPackageReadWriter) Read() ([]*yaml.RNode, error)

func (*LocalPackageReadWriter) Write 

func (r *LocalPackageReadWriter) Write(nodes []*yaml.RNode) error

type LocalPackageReader 

type LocalPackageReader struct {
	Kind string `yaml:"kind,omitempty"`

	// PackagePath is the path to the package directory.
	PackagePath string `yaml:"path,omitempty"`

	// PackageFileName is the name of file containing package metadata.
	// It will be used to identify package.
	PackageFileName string `yaml:"packageFileName,omitempty"`

	// MatchFilesGlob configures Read to only read Resources from files matching any of the
	// provided patterns.
	// Defaults to ["*.yaml", "*.yml"] if empty.  To match all files specify ["*"].
	MatchFilesGlob []string `yaml:"matchFilesGlob,omitempty"`

	// IncludeSubpackages will configure Read to read Resources from subpackages.
	// Subpackages are identified by presence of PackageFileName.
	IncludeSubpackages bool `yaml:"includeSubpackages,omitempty"`

	// ErrorIfNonResources will configure Read to throw an error if yaml missing missing
	// apiVersion or kind is read.
	ErrorIfNonResources bool `yaml:"errorIfNonResources,omitempty"`

	// OmitReaderAnnotations will cause the reader to skip annotating Resources with the file
	// path and mode.
	OmitReaderAnnotations bool `yaml:"omitReaderAnnotations,omitempty"`

	// SetAnnotations are annotations to set on the Resources as they are read.
	SetAnnotations map[string]string `yaml:"setAnnotations,omitempty"`

	// FileSkipFunc is a function which returns true if reader should ignore
	// the file
	FileSkipFunc LocalPackageSkipFileFunc

	// PreserveSeqIndent if true adds kioutil.SeqIndentAnnotation to each resource
	PreserveSeqIndent bool

	// FileSystem can be used to mock the disk file system.
	FileSystem filesys.FileSystemOrOnDisk

	// WrapBareSeqNode wraps the bare sequence node document with map node,
	// kyaml uses reader annotations to track resources, it is not possible to
	// add them to bare sequence nodes, this option enables wrapping such bare
	// sequence nodes into map node with key yaml.BareSeqNodeWrappingKey
	// note that this wrapping is different and not related to ResourceList wrapping
	WrapBareSeqNode bool
}

LocalPackageReader reads ResourceNodes from a local package.

func (LocalPackageReader) Read 

func (r LocalPackageReader) Read() ([]*yaml.RNode, error)

Read reads the Resources.

type LocalPackageSkipFileFunc added in v0.10.17 

type LocalPackageSkipFileFunc func(relPath string) bool

LocalPackageSkipFileFunc is a function which returns true if the file in the package should be ignored by reader. relPath is an OS specific relative path

type LocalPackageWriter 

type LocalPackageWriter struct {
	Kind string `yaml:"kind,omitempty"`

	// PackagePath is the path to the package directory.
	PackagePath string `yaml:"path,omitempty"`

	// KeepReaderAnnotations if set will retain the annotations set by LocalPackageReader
	KeepReaderAnnotations bool `yaml:"keepReaderAnnotations,omitempty"`

	// ClearAnnotations will clear annotations before writing the resources
	ClearAnnotations []string `yaml:"clearAnnotations,omitempty"`

	// FileSystem can be used to mock the disk file system.
	FileSystem filesys.FileSystemOrOnDisk
}

LocalPackageWriter writes ResourceNodes to a filesystem

func (LocalPackageWriter) Write 

func (r LocalPackageWriter) Write(nodes []*yaml.RNode) error

type PackageBuffer 

type PackageBuffer struct {
	Nodes []*yaml.RNode
}

PackageBuffer implements Reader and Writer, storing Resources in a local field.

func (*PackageBuffer) Read 

func (r *PackageBuffer) Read() ([]*yaml.RNode, error)

func (*PackageBuffer) Write 

func (r *PackageBuffer) Write(nodes []*yaml.RNode) error

type Pipeline 

type Pipeline struct {
	// Inputs provide sources for Resource Configuration to be read.
	Inputs []Reader `yaml:"inputs,omitempty"`

	// Filters are transformations applied to the Resource Configuration.
	// They are applied in the order they are specified.
	// Analogous to http://www.linfo.org/filters.html
	Filters []Filter `yaml:"filters,omitempty"`

	// Outputs are where the transformed Resource Configuration is written.
	Outputs []Writer `yaml:"outputs,omitempty"`

	// ContinueOnEmptyResult configures what happens when a filter in the pipeline
	// returns an empty result.
	// If it is false (default), subsequent filters will be skipped and the result
	// will be returned immediately. This is useful as an optimization when you
	// know that subsequent filters will not alter the empty result.
	// If it is true, the empty result will be provided as input to the next
	// filter in the list. This is useful when subsequent functions in the
	// pipeline may generate new resources.
	ContinueOnEmptyResult bool `yaml:"continueOnEmptyResult,omitempty"`
}

Pipeline reads Resource Configuration from a set of Inputs, applies some transformation filters, and writes the results to a set of Outputs.

Analogous to http://www.linfo.org/pipes.html

func (Pipeline) Execute 

func (p Pipeline) Execute() error

Execute executes each step in the sequence, returning immediately after encountering any error as part of the Pipeline.

func (Pipeline) ExecuteWithCallback added in v0.6.1 

func (p Pipeline) ExecuteWithCallback(callback PipelineExecuteCallbackFunc) error

ExecuteWithCallback executes each step in the sequence, returning immediately after encountering any error as part of the Pipeline. The callback will be called each time a step succeeds.

type PipelineExecuteCallbackFunc added in v0.6.1 

type PipelineExecuteCallbackFunc = func(op Filter)

PipelineExecuteCallbackFunc defines a callback function that will be called each time a step in the pipeline succeeds.

type Reader 

type Reader interface {
	Read() ([]*yaml.RNode, error)
}

Reader reads ResourceNodes. Analogous to io.Reader.

type ReaderWriter 

type ReaderWriter interface {
	Reader
	Writer
}

ReaderWriter implements both Reader and Writer interfaces

type ResourceNodeSlice 

type ResourceNodeSlice []*yaml.RNode

ResourceNodeSlice is a collection of ResourceNodes. While ResourceNodeSlice has no inherent constraints on ordering or uniqueness, specific Readers, Filters or Writers may have constraints.

func (ResourceNodeSlice) Read 

func (o ResourceNodeSlice) Read() ([]*yaml.RNode, error)

type Setup added in v0.8.0 

type Setup struct {
	// root is the tmp directory
	Root string
}

Setup creates directories and files for testing

func SetupDirectories added in v0.8.0 

func SetupDirectories(t *testing.T, dirs ...string) Setup

setupDirectories creates directories for reading test configuration from

func (Setup) Clean added in v0.8.0 

func (s Setup) Clean()

clean deletes the test config

func (Setup) WriteFile added in v0.8.0 

func (s Setup) WriteFile(t *testing.T, path string, value []byte)

writeFile writes a file under the test directory

type TrackableFilter added in v0.13.2 

type TrackableFilter interface {
	Filter
	WithMutationTracker(func(key, value, tag string, node *yaml.RNode))
}

TrackableFilter is an extension of Filter which is also capable of tracking which fields were mutated by the filter.

type TreeStructure 

type TreeStructure string
const (
	// TreeStructurePackage configures TreeWriter to generate the tree structure off of the
	// Resources packages.
	TreeStructurePackage TreeStructure = "directory"

	// TreeStructureOwners configures TreeWriter to generate the tree structure off of the
	// Resource owners.
	TreeStructureGraph TreeStructure = "owners"
)

type TreeWriter 

type TreeWriter struct {
	Writer          io.Writer
	Root            string
	Fields          []TreeWriterField
	Structure       TreeStructure
	OpenAPIFileName string
}

TreeWriter prints the package structured as a tree. TODO(pwittrock): test this package better. it is lower-risk since it is only 

used for printing rather than updating or editing.

func (TreeWriter) Write 

func (p TreeWriter) Write(nodes []*yaml.RNode) error

Write writes the ascii tree to p.Writer

type TreeWriterField 

type TreeWriterField struct {
	yaml.PathMatcher
	Name    string
	SubName string
}

TreeWriterField configures a Resource field to be included in the tree

type Writer 

type Writer interface {
	Write([]*yaml.RNode) error
}

Writer writes ResourceNodes. Analogous to io.Writer.

type WriterFunc 

type WriterFunc func([]*yaml.RNode) error

WriterFunc implements a Writer as a function.

func (WriterFunc) Write 

func (fn WriterFunc) Write(o []*yaml.RNode) error

Source Files

View all Source files

Directories

Path Synopsis
Package yamlfmt contains libraries for formatting yaml files containing Kubernetes Resource configuration.
 Package yamlfmt contains libraries for formatting yaml files containing Kubernetes Resource configuration.
testyaml
Package testyaml contains test data and libraries for formatting Kubernetes configuration
 Package testyaml contains test data and libraries for formatting Kubernetes configuration

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.