model

README

YAML API versioning internals for developers

This is a guide for abc CLI developers on how to create a new api_version.

When to bump the api_version

In any of the following cases:

• You're adding a new field to an existing YAML struct
• You're changing the semantics or interpretation of an existing yaml field
• You're adding a new type of YAML file

Every api_version represents a distinct configuration "language", and we never want to change the semantics of an existing api_version that has already been released.

Steps

Beware, these steps may be not quite right. Please fix them or file a bug if something isn't right. You can use this PR as an example.

• Announce in the abc CLI developer chat that you're bumping the api_version and binary version, so nobody else tries to make a conflicting simultaneous api_version change.
• Let "old version" mean the current api_version, and let "new version" mean the new api_version that you are creating. For these examples, let's suppose the old version is v9 and the new version is v10.
• There are a few different kinds of YAML files (e.g. Template, Manifest, GoldenTest). Each one of these has its own subdirectory under templates/model. Most api_version changes will only need to change one of these kinds. For each kind:
  • Locate the directory for that YAML kind (e.g. kind Template corresponds to templates/model/spec.
  • Create a new directory under that named after the new api_version, e.g. templates/model/spec/v10
  • Copy the contents of the previous version into your new directory (e.g. cp templates/model/spec/v9/* templates/model/spec/v10/). This includes *_test.go files.
  • Implement the Spec.Upgrade() method in the old schema (e.g. v9, in templates/model/spec/v9/upgrade.go) that specifies how to upgrade from the old schema to the new schema. See existing implementations for examples of how to do this simply using the copier library. For example, suppose you renamed a field, then you would implement the Upgrade() method so that it stored the contents of the old field in the newly renamed field.
• In templates/model/decode/decode.go, add a new entry to the end of apiVersions (If new api version is still WIP and is not ready to get released, set the unreleased field to true). See the instructions and examples there.
• Do a global replace (skip upgrade test cases as older api version is expected in those test cases) to point to your new abc cli version. For example, if you add a new api version entry called v1beta1, you'd change relevant Go files from old api version to new api version apiVersion: 'cli.abcxyz.dev/v1beta1'.
• Do a global replace of imports to point to your new version. For example, if you made changes to the template spec, you'd change \tspec "github.com/abcxyz/abc/templates/model/spec/v9" with \tspec "github.com/abcxyz/abc/templates/model/spec/v10", in all Go files. You only need to do this for the kinds you changed (if you only changed the template spec then you don't need to do this for goldentests and manifests).
• Change the tests named oldest_*_upgrades_to_newest, speculative_upgrade_* and newest_* to change references from the formerly-newest schema to the schema you just created. (e.g. specv9 -> specv10)
• Submit your work so far as a PR. This will allow the binary to understand your new work-in-progress api_version, but since we didn't announce it yet, users shouldn't start using it yet. Example PR.
• Modify the new version directory to make whatever struct changes you want to make (e.g. add a new field/feature), including tests.
• Update the "list of api_versions" section in /README.md. In templates/model/decode/decode.go, remove the line unreleased: true from your api version in the apiVersions list
• Release a new version of the abc CLI (see RELEASING.md). If you've added a field, you only need to bump the minor version number. If you've changed the meaning of a field or removed a field, then you need to bump the major version number.
• Now you can start using your new api_version and its new features in your templates.

Deprecating an api_version

TODO: write this once we do it for the first time.

Documentation

Index

• Variables
• func DecodeAndValidate(r io.Reader, filename string, outPtr Validator) error
• func IsValidRegexGroupName(s String, fieldName string) error
• func NonEmptySlice[T any](pos *ConfigPos, s []T, fieldName string) error
• func NotZero[T comparable](pos *ConfigPos, t T, fieldName string) error
• func NotZeroModel[T comparable](pos *ConfigPos, x valWithPos[T], fieldName string) error
• func OneOf[T comparable](parentPos *ConfigPos, x valWithPos[T], allowed []T, fieldName string) error
• func UnmarshalPlain(n *yaml.Node, outPtr any, outPos *ConfigPos, extraYAMLFields ...string) error
• func ValidateEach[T Validator](s []T) error
• func ValidateUnlessNil(v Validator) error
• type Bool
• type ConfigPos
  • func YAMLPos(n *yaml.Node) *ConfigPos
  • func (c *ConfigPos) Errorf(fmtStr string, args ...any) error
  • func (c ConfigPos) IsZero() bool
• type Int
• type String
• type Validator
• type ValidatorUpgrader

Variables

var ErrLatestVersion = errors.New("this is the latest version")

ErrLatestVersion is a sentinel error returned from Upgrade() meaning that there is no further upgrading to be done because the current version is already the latest version.

Functions

func DecodeAndValidate

func DecodeAndValidate(r io.Reader, filename string, outPtr Validator) error

DecodeAndValidate unmarshals the YAML text in the given Reader into the given pointer-to-struct, and calls Validate() on it. Returns any unmarshaling error or validation error.

func IsValidRegexGroupName

func IsValidRegexGroupName(s String, fieldName string) error

IsValidRegexGroupName returns whether the given string will be accepted by the regexp library as an RE2 group name.

func NonEmptySlice

func NonEmptySlice[T any](pos *ConfigPos, s []T, fieldName string) error

NonEmptySlice returns error if the given slice is empty.

func NotZero

func NotZero[T comparable](pos *ConfigPos, t T, fieldName string) error

NotZero returns error if the given value is equal to the zero value for type T.

func NotZeroModel

func NotZeroModel[T comparable](pos *ConfigPos, x valWithPos[T], fieldName string) error

NotZeroModel returns error if the given model's value is equal to the zero value for type T.

func OneOf

func OneOf[T comparable](parentPos *ConfigPos, x valWithPos[T], allowed []T, fieldName string) error

OneOf returns error if x.Val is not one of the given allowed choices.

func UnmarshalPlain

func UnmarshalPlain(n *yaml.Node, outPtr any, outPos *ConfigPos, extraYAMLFields ...string) error

UnmarshalPlain unmarshals the yaml node n into the struct pointer outPtr, as if it did not have an UnmarshalYAML method. This lets you still use the default unmarshaling logic to populate the fields of your struct, while adding custom logic before and after.

outPtr must be a pointer to a struct which will be modified by this function.

pos will be modified by this function to contain the position of this yaml node within the input file.

The `yaml:"..."` tags of the outPtr struct are used to determine the set of valid fields. Unexpected fields in the yaml are treated as an error. To allow extra yaml fields that don't correspond to a field of outPtr, provide their names in extraYAMLFields. This allows some fields to be handled specially.

func ValidateEach

func ValidateEach[T Validator](s []T) error

ValidateEach calls Validate() on each element of the input and returns all errors encountered.

func ValidateUnlessNil

func ValidateUnlessNil(v Validator) error

ValidateUnlessNil is intended to be used in a model Validate() method. Semantically it means "if this model field is present (non-nil), then validate it. If not present, then skip validation." This is useful for polymorphic models like Step that have many possible child types, only one of which will be set.

Types

type Bool

type Bool = valWithPos[bool]

Bool represents a boolean field in a model, together with its location in the input file.

type ConfigPos

type ConfigPos struct {
	Line   int
	Column int
}

ConfigPos stores the position of an config value so error messages post-validation can point to problems. The zero value means "position unknown or there is no position."

This is theoretically agnostic to input format; we could decide to support alternative serialization formats besides YAML in the future.

func YAMLPos

func YAMLPos(n *yaml.Node) *ConfigPos

YAMLPos constructs a position struct based on a YAML parse cursor.

func (*ConfigPos) Errorf

func (c *ConfigPos) Errorf(fmtStr string, args ...any) error

Errorf returns a error prepended with spec.yaml position information, if available.

Examples:

Wrapping an error: c.Errorf("foo(): %w", err)

Creating an error: c.Errorf("something went wrong doing action %s", action)

func (ConfigPos) IsZero

func (c ConfigPos) IsZero() bool

type Int

type Int = valWithPos[int]

Int represents an integer field in a model, together with its location in the input file.

type String

type String = valWithPos[string]

String represents a string field in a model, together with its location in the input file.

type Validator

type Validator interface {
	Validate() error
}

Validator is any model struct that has a validate method. It's useful for "higher order" validation functions like "validate each entry in a list".

type ValidatorUpgrader

type ValidatorUpgrader interface {
	Validator

	// Upgrade converts an old version of a YAML struct into newer version for
	// example, from api_version v1 to v2. If the struct is already the most
	// recent version, it returns (nil,LatestVersion).
	//
	// An error other than LatestVersion means that the model cannot be converted
	// because either:
	//
	//   1. something weird happened
	//   2. the YAML struct uses features in an old version that are not supported in newer
	//      versions.
	Upgrade(context.Context) (ValidatorUpgrader, error)
}

ValidatorUpgrader is the interface implemented by every kind of YAML struct (templates, golden tests, manifests, etc).