rillv1

Documentation

Index

• Constants
• Variables
• func InitEmpty(ctx context.Context, repo drivers.RepoStore, instanceID, title string) error
• func IsInit(ctx context.Context, repo drivers.RepoStore, instanceID string) bool
• func ResolveTemplate(tmpl string, data TemplateData) (string, error)
• type Connector
• type ConnectorDef
• type Diff
• type Node
• type Parser
  • func Parse(ctx context.Context, repo drivers.RepoStore, instanceID string, ...) (*Parser, error)
  • func (p *Parser) AnalyzeConnectors(ctx context.Context) ([]*Connector, error)
  • func (p *Parser) Reparse(ctx context.Context, paths []string) (*Diff, error)
• type Resource
• type ResourceKind
  • func ParseResourceKind(kind string) (ResourceKind, error)
  • func (k ResourceKind) String() string
• type ResourceName
  • func (n ResourceName) Normalized() ResourceName
  • func (n ResourceName) String() string
• type RillYAML
  • func ParseRillYAML(ctx context.Context, repo drivers.RepoStore, instanceID string) (*RillYAML, error)
• type TemplateData
• type TemplateMetadata
  • func AnalyzeTemplate(tmpl string) (*TemplateMetadata, error)
• type TemplateResource
• type VariableDef

Constants

const Version = "rillv1"

Version identifier for this compiler

Variables

var ErrInvalidProject = errors.New("parser: not a valid project (rill.yaml not found)")

ErrInvalidProject indicates a project without a rill.yaml file

Functions

func InitEmpty

func InitEmpty(ctx context.Context, repo drivers.RepoStore, instanceID, title string) error

InitEmpty initializes an empty project

func IsInit

func IsInit(ctx context.Context, repo drivers.RepoStore, instanceID string) bool

IsInit returns true if a Rill project exists in the repo

func ResolveTemplate

func ResolveTemplate(tmpl string, data TemplateData) (string, error)

ResolveTemplate resolves a template to a string using the given data.

Types

type Connector

type Connector struct {
	Driver          string
	Name            string
	Spec            drivers.Spec
	Resources       []*Resource
	AnonymousAccess bool
}

Connector contains metadata about a connector used in a Rill project

type ConnectorDef

type ConnectorDef struct {
	Type     string
	Name     string
	Defaults map[string]string
}

ConnectorDef is a subtype of RillYAML, defining connectors required by the project

type Diff

type Diff struct {
	Added            []ResourceName
	Modified         []ResourceName
	ModifiedRillYAML bool
	Deleted          []ResourceName
}

Diff shows changes to Parser.Resources following an incremental reparse.

type Node

type Node struct {
	Kind              ResourceKind
	Name              string
	Refs              []ResourceName
	Paths             []string
	YAML              *yaml.Node
	YAMLRaw           string
	YAMLPath          string
	Connector         string
	SQL               string
	SQLPath           string
	SQLAnnotations    map[string]any
	SQLUsesTemplating bool
}

Node represents one path stem in the project. It contains data derived from a YAML and/or SQL file (e.g. "/path/to/file.yaml" for "/path/to/file.sql").

type Parser

type Parser struct {
	// Options
	Repo             drivers.RepoStore
	InstanceID       string
	DuckDBConnectors []string

	// Output
	RillYAML  *RillYAML
	Resources map[ResourceName]*Resource
	Errors    []*runtimev1.ParseError
	// contains filtered or unexported fields
}

Parser parses a Rill project directory into a set of resources. After the initial parse, the parser can be used to incrementally reparse a subset of files. Parser is not concurrency safe.

func Parse

func Parse(ctx context.Context, repo drivers.RepoStore, instanceID string, duckDBConnectors []string) (*Parser, error)

Parse creates a new parser and parses the entire project.

Note on SQL parsing: For DuckDB SQL specifically, the parser can use a SQL parser to extract refs and annotations (instead of relying on templating or YAML). To enable SQL parsing for a connector, pass it in duckDBConnectors. If DuckDB SQL parsing should be used on files where no connector is specified, put an empty string in duckDBConnectors.

func (*Parser) AnalyzeConnectors

func (p *Parser) AnalyzeConnectors(ctx context.Context) ([]*Connector, error)

AnalyzeConnectors extracts connector metadata from a Rill project

func (*Parser) Reparse

func (p *Parser) Reparse(ctx context.Context, paths []string) (*Diff, error)

Reparse re-parses the indicated file paths, updating the Parser's state. If a previous call to Reparse has returned an error, the Parser may not be accessed or called again.

type Resource

type Resource struct {
	// Metadata
	Name  ResourceName
	Paths []string
	Refs  []ResourceName // Derived from rawRefs after parsing (can't contain ResourceKindUnspecified)

	// Only one of these will be non-nil
	SourceSpec      *runtimev1.SourceSpec
	ModelSpec       *runtimev1.ModelSpec
	MetricsViewSpec *runtimev1.MetricsViewSpec
	MigrationSpec   *runtimev1.MigrationSpec
	// contains filtered or unexported fields
}

Resource parsed from code files. One file may output multiple resources and multiple files may contribute config to one resource.

type ResourceKind

type ResourceKind int

ResourceKind identifies a resource type supported by the parser

const (
	ResourceKindUnspecified ResourceKind = iota
	ResourceKindSource
	ResourceKindModel
	ResourceKindMetricsView
	ResourceKindMigration
)

func ParseResourceKind

func ParseResourceKind(kind string) (ResourceKind, error)

ParseResourceKind maps a string to a ResourceKind. Note: The empty string is considered a valid kind (unspecified).

func (ResourceKind) String

func (k ResourceKind) String() string

type ResourceName

type ResourceName struct {
	Kind ResourceKind
	Name string
}

ResourceName is a unique identifier for a resource

func (ResourceName) Normalized

func (n ResourceName) Normalized() ResourceName

func (ResourceName) String

func (n ResourceName) String() string

type RillYAML

type RillYAML struct {
	Title       string
	Description string
	Connectors  []*ConnectorDef
	Variables   []*VariableDef
}

RillYAML is the parsed contents of rill.yaml

func ParseRillYAML

func ParseRillYAML(ctx context.Context, repo drivers.RepoStore, instanceID string) (*RillYAML, error)

ParseRillYAML parses only the project's rill.yaml (or rill.yml) file.

type TemplateData

type TemplateData struct {
	Claims    map[string]any
	Variables map[string]string
	Self      TemplateResource
	Resolve   func(ref ResourceName) (string, error)
	Lookup    func(name ResourceName) (TemplateResource, error)
}

TemplateData contains data for resolving a template.

type TemplateMetadata

type TemplateMetadata struct {
	Refs                     []ResourceName
	Config                   map[string]any
	UsesTemplating           bool
	ResolvedWithPlaceholders string
}

TemplateMetadata contains metadata extracted from a template.

func AnalyzeTemplate

func AnalyzeTemplate(tmpl string) (*TemplateMetadata, error)

AnalyzeTemplate parses a template and extracts metadata.

type TemplateResource

type TemplateResource struct {
	Meta  *runtimev1.ResourceMeta
	Spec  any
	State any
}

TemplateResource contains data for a resource for injection into a template.

type VariableDef

type VariableDef struct {
	Name    string
	Default string
}

VariableDef is a subtype of RillYAML, defining defaults for project variables