cuex

README

CueX: Extended CUE Compiler

Background

In native CUE, although there are functions that could be used in cue string to render values with underlying implementations, it is not easy to make extensions to those functions.

// example.cue
import "strings"

x: "aaa"
y: strings.ToUpper(x)

In addition to that, CUE's function are designed for idempotent execution, in other word, with the same input, the output of CUE functions are fixed.

Goal

However, there are times that user may want to extend the render capability of CUE by integrating customized function implementations.

The aim of CueX is to build a light engine for reach that goal.

Arch

Resolve Process

To allow users make extension to the native CUE rendering process, CueX uses a customized Compiler to do the Resolve work. It recursively loops over all the CUE values and find out struct as

{
  #do: string
  #provider: string

  $params: {
    ...
  }

  $returns: {
    ...
  }
}

It will search for the ProviderFn (specified by #do) in the given Provider and call the function. Note that every schema should have params and returns. CueX will execute the ProviderFn with $params value as inputs, then it will fill back the result to $returns after execution. In this way, it is possible to define customized functions for either rendering or pure executing.

Each time a ProviderFn is called, the Compiler will repeat the Resolve work after the result of last execution is filled back. Executed CUE values will not be executed again in the later process. The repeated Resolve stops when no more CUE values that needs to be run.

To help CUE users recognize the input and output scheme for the function call, there is CUETemplater aside by the Provider that holds CUE definition for the provider function. Like http.cue. It also defines the import path for use when user want to reference it.

import "vela/http"

req: http.#Get & {
  $params: {
    url: "https://cuelang.org"
  }
}

body: req.$returns.body

CUETemplater and Provider together compose Package, the basic unit for registering and discovery. By far, internal implementation of Package includes base64, http, kube, etc. Packages are managed in PackageManager which gives unified interface for access.

External Imports

In the case user wants to make integration, there is also a Kubernetes CustomResource defined as packages.cue.oam.dev. By building server and host it somewhere, you can register it as a Provider in Package.

For example, if you want to register a external package to provide mysql access in CUE, you can create a CustomResource in your Kubernetes cluster as

apiVersion: cue.oam.dev/v1alpha1
kind: Package
metadata:
  name: mysql
spec:
  path: ext/db/mysql
  provider:
    protocol: http
    endpoint: https://my-render-server/mysql
  templates:
    mysql.cue: |
      package mysql
      #Table: {
        name: string
      }
      #ListTables: {
        #do: "list-tables"
        #provider: "mysql"
        $params: {
          conn: string
          db: string
        }
        $returns: [...#Table]
      }
      ...

And you can use this package in your CUE code like

import "ext/db/mysql"

list: mysql.#ListTables & {
  $params: {
    conn: "my-connect"
    db: "my-db"
  }
}

tables: list.$returns

By default, PackageManager only loads internal packages. There are functions for it to load external packages:

1. LoadExternalPackages: Load Packages from CustomResource in the target cluster at once.
2. ListenExternalPackages: Watch CustomResource Package changes in the target cluster.

In the case that CueX only need to execute once, it is recommended to use the first option, (like Vela CLI). In other cases that updates are always needed (like Controller or WebServer), the second option is recommended.

Usage

Currently, we're using CueX as the rendering engine for KubeVela. As shown in the diagram, you can register your customized package and use it in your Definition.

Documentation

Index

• Variables
• func AddFlags(set *pflag.FlagSet)
• func CompileString(ctx context.Context, src string) (cue.Value, error)
• func CompileStringWithOptions(ctx context.Context, src string, opts ...CompileOption) (cue.Value, error)
• type CompileConfig
  • func NewCompileConfig(opts ...CompileOption) *CompileConfig
• type CompileOption
  • func WithExtraData(key string, data interface{}) CompileOption
• type Compiler
  • func NewCompilerWithDefaultInternalPackages() *Compiler
  • func NewCompilerWithInternalPackages(packages ...cuexruntime.Package) *Compiler
  • func (in *Compiler) CompileString(ctx context.Context, src string) (cue.Value, error)
  • func (in *Compiler) CompileStringWithOptions(ctx context.Context, src string, opts ...CompileOption) (cue.Value, error)
  • func (in *Compiler) Resolve(ctx context.Context, value cue.Value) (cue.Value, error)
• type DisableResolveProviderFunctions
  • func (in DisableResolveProviderFunctions) ApplyTo(cfg *CompileConfig)
• type FunctionCallError
  • func NewFunctionCallError(v cue.Value, err error) FunctionCallError
  • func (e FunctionCallError) Error() string
• type ProviderFnNotFoundErr
  • func (e ProviderFnNotFoundErr) Error() string
• type ProviderNotFoundErr
  • func (e ProviderNotFoundErr) Error() string
• type ResolveTimeoutErr
  • func (e ResolveTimeoutErr) Error() string

Variables

var (
	// EnableExternalPackageForDefaultCompiler .
	EnableExternalPackageForDefaultCompiler = true
	// EnableExternalPackageWatchForDefaultCompiler .
	EnableExternalPackageWatchForDefaultCompiler = false
)

var DefaultCompiler = singleton.NewSingleton[*Compiler](func() *Compiler {
	compiler := NewCompilerWithDefaultInternalPackages()
	if EnableExternalPackageForDefaultCompiler {
		if err := compiler.LoadExternalPackages(context.Background()); err != nil && !kerrors.IsNotFound(err) {
			klog.Errorf("failed to load external packages for cuex default compiler: %s", err.Error())
		}
	}
	if EnableExternalPackageWatchForDefaultCompiler {
		go compiler.ListenExternalPackages(nil)
	}
	return compiler
})

DefaultCompiler compiler for cuex to compile

Functions

func AddFlags

func AddFlags(set *pflag.FlagSet)

AddFlags add flags for configuring cuex default compiler

func CompileString

func CompileString(ctx context.Context, src string) (cue.Value, error)

CompileString use cuex default compiler to compile cue string

func CompileStringWithOptions

func CompileStringWithOptions(ctx context.Context, src string, opts ...CompileOption) (cue.Value, error)

CompileStringWithOptions use cuex default compiler to compile cue string with options

Types

type CompileConfig

type CompileConfig struct {
	ResolveProviderFunctions bool
	PreResolveMutators       []func(context.Context, string) (string, error)
}

CompileConfig config for running compile process

func NewCompileConfig

func NewCompileConfig(opts ...CompileOption) *CompileConfig

NewCompileConfig create new CompileConfig

type CompileOption

type CompileOption interface {
	ApplyTo(*CompileConfig)
}

CompileOption options for compile cue string

func WithExtraData

func WithExtraData(key string, data interface{}) CompileOption

WithExtraData fill the cue.Value before resolve

type Compiler

type Compiler struct {
	*cuexruntime.PackageManager
}

Compiler for compile cue strings into cue.Value

func NewCompilerWithDefaultInternalPackages

func NewCompilerWithDefaultInternalPackages() *Compiler

NewCompilerWithDefaultInternalPackages create compiler with default internal packages

func NewCompilerWithInternalPackages

func NewCompilerWithInternalPackages(packages ...cuexruntime.Package) *Compiler

NewCompilerWithInternalPackages create compiler with internal packages

func (*Compiler) CompileString

func (in *Compiler) CompileString(ctx context.Context, src string) (cue.Value, error)

CompileString compile given cue string into cue.Value

func (*Compiler) CompileStringWithOptions

func (in *Compiler) CompileStringWithOptions(ctx context.Context, src string, opts ...CompileOption) (cue.Value, error)

CompileStringWithOptions compile given cue string with extra options

func (*Compiler) Resolve

func (in *Compiler) Resolve(ctx context.Context, value cue.Value) (cue.Value, error)

Resolve runs the resolve process by calling provider functions

type DisableResolveProviderFunctions

type DisableResolveProviderFunctions struct{}

DisableResolveProviderFunctions disable ResolveProviderFunctions

func (DisableResolveProviderFunctions) ApplyTo

func (in DisableResolveProviderFunctions) ApplyTo(cfg *CompileConfig)

ApplyTo .

type FunctionCallError

type FunctionCallError struct {
	Path  string
	Value string
	Err   error
}

FunctionCallError error for executing provider function

func NewFunctionCallError

func NewFunctionCallError(v cue.Value, err error) FunctionCallError

NewFunctionCallError create a new error for executing resolved function call

func (FunctionCallError) Error

func (e FunctionCallError) Error() string

Error .

type ProviderFnNotFoundErr

type ProviderFnNotFoundErr struct {
	Provider, Fn string
}

ProviderFnNotFoundErr provider function not found error

func (ProviderFnNotFoundErr) Error

func (e ProviderFnNotFoundErr) Error() string

Error .

type ProviderNotFoundErr

type ProviderNotFoundErr string

ProviderNotFoundErr provider not found error

func (ProviderNotFoundErr) Error

func (e ProviderNotFoundErr) Error() string

Error .

type ResolveTimeoutErr

type ResolveTimeoutErr struct{}

ResolveTimeoutErr error when Resolve process timeout

func (ResolveTimeoutErr) Error

func (e ResolveTimeoutErr) Error() string

Error .