Documentation ¶
Overview ¶
Package lucicfg contains LUCI config generator.
All Starlark code is executed sequentially in a single goroutine from inside Generate function, thus this package doesn't used any mutexes or other synchronization primitives. It is safe to call Generate concurrently though, since there's no global shared state, each Generate call operates on its own state.
Index ¶
- Constants
- func FindTrackedFiles(dir string, patterns []string) ([]string, error)
- func TrackedSet(patterns []string) func(string) (bool, error)
- type BacktracableError
- type BlobDatum
- type CompareResult
- type ConfigSet
- type ConfigSetValidator
- type Datum
- type Error
- type Inputs
- type MessageDatum
- type Meta
- type Output
- func (o Output) Compare(dir string, semantic bool) (map[string]CompareResult, error)
- func (o Output) ConfigSets() ([]ConfigSet, error)
- func (o Output) DebugDump()
- func (o Output) DiscardChangesToUntracked(ctx context.Context, tracked []string, dir string) error
- func (o Output) Files() []string
- func (o Output) Read(dir string) error
- func (o Output) Write(dir string, force bool) (written, untouched []string, err error)
- type State
- type ValidationMessage
- type ValidationResult
Constants ¶
const ( // Version is the version of lucicfg tool. // // It ends up in CLI output and in User-Agent headers. Version = "1.43.16" // UserAgent is used for User-Agent header in HTTP requests from lucicfg. UserAgent = "lucicfg v" + Version )
Variables ¶
This section is empty.
Functions ¶
func FindTrackedFiles ¶
FindTrackedFiles recursively discovers all regular files in the given directory whose names match given patterns.
See TrackedSet for the format of `patterns`. If the directory doesn't exist, returns empty slice.
Returned file names are sorted, slash-separated and relative to `dir`.
func TrackedSet ¶
TrackedSet returns a predicate that classifies whether a slash-separated path belongs to a tracked set or not.
Each entry in `patterns` is either `<glob pattern>` (a "positive" glob) or `!<glob pattern>` (a "negative" glob). A path is considered tracked if its base name matches any of the positive globs and none of the negative globs. If `patterns` is empty, no paths are considered tracked. If all patterns are negative, single `**/*` positive pattern is implied as well.
The predicate returns an error if some pattern is malformed.
Types ¶
type BacktracableError ¶
type BacktracableError interface { error // Backtrace returns a user-friendly error message describing the stack // of calls that led to this error, along with the error message itself. Backtrace() string }
BacktracableError is an error that has a starlark backtrace attached to it.
Implemented by Error here, by starlark.EvalError and graph errors.
type BlobDatum ¶
type BlobDatum []byte
BlobDatum is a Datum which is just a raw byte blob.
type CompareResult ¶
type CompareResult int
CompareResult is returned by Datum.Compare.
const ( UnknownResult CompareResult = iota // used as a placeholder on errors Identical // datums are byte-to-byte identical SemanticallyEqual // datums are byte-to-byte different, but semantically equal Different // datums are semantically different )
type ConfigSet ¶
type ConfigSet struct { // Name is a name of this config set, e.g. "projects/something". // // It is used by LUCI Config to figure out how to validate files in the set. Name string // Data is files belonging to the config set. // // Keys are slash-separated filenames, values are corresponding file bodies. Data map[string][]byte }
ConfigSet is an in-memory representation of a single config set.
func ReadConfigSet ¶
ReadConfigSet reads all regular files in the given directory (recursively) and returns them as a ConfigSet with given name.
func (ConfigSet) AsOutput ¶
AsOutput converts this config set into Output that have it at the given root path (usually ".").
func (ConfigSet) Validate ¶
func (cs ConfigSet) Validate(ctx context.Context, val ConfigSetValidator) *ValidationResult
Validate sends the config set for validation to LUCI Config service.
Returns ValidationResult with a list of validation message (errors, warnings, etc). The list of messages may be empty if the config set is 100% valid.
If the RPC call itself failed, ValidationResult is still returned, but it has only ConfigSet and RPCError fields populated.
type ConfigSetValidator ¶
type ConfigSetValidator interface { // Validate sends the validation request to the service. // // Returns errors only on RPC errors. Actual validation errors are // communicated through []*config.ValidationResult_Message. Validate(ctx context.Context, cs ConfigSet) ([]*config.ValidationResult_Message, error) }
ConfigSetValidator is primarily implemented through config.Service, but can also be mocked in tests.
func NewRemoteValidator ¶
func NewRemoteValidator(conn *grpc.ClientConn) ConfigSetValidator
type Datum ¶
type Datum interface { // Bytes is a raw file body to put on disk. Bytes() ([]byte, error) // Compare semantically compares this datum to 'other'. Compare(other []byte) (CompareResult, error) }
Datum represents one generated output file.
type Error ¶
type Error struct { Msg string Stack *builtins.CapturedStacktrace }
Error is a single error message emitted by the config generator.
It holds a stack trace responsible for the error.
type Inputs ¶
type Inputs struct { Code interpreter.Loader // a package with the user supplied code Path string // absolute path to the main package, if known Entry string // a name of the entry point script in this package Meta *Meta // defaults for lucicfg own parameters Vars map[string]string // var values passed via `-var key=value` flags // contains filtered or unexported fields }
Inputs define all inputs for the config generator.
type MessageDatum ¶
type MessageDatum struct { Header string Message *starlarkproto.Message // contains filtered or unexported fields }
MessageDatum is a Datum constructed from a proto message.
func (*MessageDatum) Bytes ¶
func (m *MessageDatum) Bytes() ([]byte, error)
Bytes is a raw file body to put on disk.
func (*MessageDatum) Compare ¶
func (m *MessageDatum) Compare(other []byte) (CompareResult, error)
Compare deserializes `other` and compares it to `m.Message`.
If `other` can't be deserialized as a proto message at all returns Different. Returns an error if `m` can't be serialized.
type Meta ¶
type Meta struct { ConfigServiceHost string `json:"config_service_host"` // LUCI config host name ConfigDir string `json:"config_dir"` // output directory to place generated files or '-' for stdout TrackedFiles []string `json:"tracked_files"` // e.g. ["*.cfg", "!*-dev.cfg"] FailOnWarnings bool `json:"fail_on_warnings"` // true to treat validation warnings as errors LintChecks []string `json:"lint_checks"` // active lint checks // contains filtered or unexported fields }
Meta contains configuration for the configuration generator itself.
It influences how generator produces output configs. It is settable through lucicfg.config(...) statements on the Starlark side or through command line flags. Command line flags override what was set via lucicfg.config(...).
See @stdlib//internal/lucicfg.star for full meaning of fields.
func (*Meta) Copy ¶
Copy returns an "untouched" copy of `m`.
In the returned copy WasTouched reports all fields as untouched.
func (*Meta) PopulateFromTouchedIn ¶
PopulateFromTouchedIn takes all touched values in `t` and copies them to `m`, overriding what's in `m`.
func (*Meta) RebaseConfigDir ¶
RebaseConfigDir changes ConfigDir, if it is set, to be absolute by appending it to the given root.
Doesn't touch "-", which indicates "stdout".
func (*Meta) WasTouched ¶
WasTouched returns true if the field (given by its Starlark snake_case name) was explicitly set via CLI flags or via lucicfg.config(...) in Starlark.
Panics if the field is unrecognized.
type Output ¶
type Output struct { // Data is all output files. // // Keys are slash-separated filenames, values are corresponding file bodies. Data map[string]Datum // Roots is mapping "config set name => its root". // // Roots are given as slash-separated paths relative to the output root, e.g. // '.' matches ALL output files. Roots map[string]string }
Output is an in-memory representation of all generated output files.
Output may span zero or more config sets, each defined by its root directory. Config sets may intersect (though this is rare).
func (Output) Compare ¶
Compare compares files on disk to what's in the output.
If 'semantic' is true, for output files based on proto messages uses semantic comparison, i.e. loads the file on disk as a proto message and compares it to the output message. If 'semantic' is false, just always compares files as byte blobs.
For each file in the output set, the resulting map has a CompareResult describing how it compares to the file on disk. They can either be identical as byte blobs (Identical), different as byte blobs, but semantically the same (SemanticallyEqual), or totally different (Different).
Note that when 'semantic' is false, only Identical and Different can appear in the result, since we compare files as byte blobs only, so there's no notion of being "semantically the same".
Files on disk that are not in the output set are totally ignored. Files in the output set that are missing on disk as Different.
Returns an error if some file on disk can't be read or some output file can't be serialized.
func (Output) ConfigSets ¶
ConfigSets partitions this output into 0 or more config sets based on Roots.
Returns an error if some output Datum can't be serialized.
func (Output) DebugDump ¶
func (o Output) DebugDump()
DebugDump writes the output to stdout in a format useful for debugging.
func (Output) DiscardChangesToUntracked ¶
DiscardChangesToUntracked replaces bodies of the files that are in the output set, but not in the `tracked` set (per TrackedSet semantics) with what's on disk in the given `dir`.
This allows to construct partially generated output: some configs (the ones in the tracked set) are generated, others are loaded from disk.
If `dir` is "-" (which indicates that the output is going to be dumped to stdout rather then to disk), just removes untracked files from the output.
func (Output) Read ¶
Read replaces values in o.Data by reading them from disk as blobs.
Returns an error if some file can't be read.
func (Output) Write ¶
Write updates files on disk to match the output.
Returns a list of written files and a list of files that were left untouched.
If 'force' is false, compares files on disk to the generated files using the semantic comparison. If they are all up-to-date (semantically) does nothing. If at least one file is stale, rewrites *all* not already identical files. That way all output files always have consistent formatting, but `lucicfg generate` still doesn't produce noop formatting changes by default (it piggy backs formatting changes onto real changes).
If 'force' is true, compares files as byte blobs and rewrites all files that changed as blobs. No semantic comparison is done.
Creates missing directories. Not atomic. All files have mode 0666.
type State ¶
type State struct { Inputs Inputs // all inputs, exactly as passed to Generate. Output Output // all generated config files, populated at the end Meta Meta // lucicfg parameters, settable through Starlark Visited []string // visited Starlark modules from Inputs // contains filtered or unexported fields }
State is mutated throughout execution of the script and at the end contains the final execution result.
It is available in the implementation of native functions exposed to the Starlark side. Starlark code operates with the state exclusively through these functions.
Use NewState to construct it.
All Starlark code is executed sequentially in a single goroutine, thus the state is not protected by any mutexes.
type ValidationMessage ¶
type ValidationMessage struct {
*config.ValidationResult_Message
}
ValidationMessage is one validation message from the LUCI Config.
It just wraps a proto, serializing it into JSON using JSONPB format using proto_names for fields. The result looks almost like the default json.Marshal serialization, except enum-valued fields (like Severity) use string enum names as values, not integers. The reason is that existing callers of lucicfg expect to see e.g. "WARNING" in the JSON, not "30".
func (ValidationMessage) MarshalJSON ¶
func (m ValidationMessage) MarshalJSON() ([]byte, error)
MarshalJSON implements json.Marshaler.
type ValidationResult ¶
type ValidationResult struct { ConfigSet string `json:"config_set"` // a config set being validated Failed bool `json:"failed"` // true if the config is bad Messages []ValidationMessage `json:"messages"` // errors, warnings, infos, etc. RPCError string `json:"rpc_error,omitempty"` // set if the RPC itself failed }
ValidationResult is what we get after validating a config set.
func (*ValidationResult) Format ¶
func (vr *ValidationResult) Format() string
Format formats the validation result as a multi-line string
func (*ValidationResult) OverallError ¶
func (vr *ValidationResult) OverallError(failOnWarnings bool) error
OverallError is nil if the validation succeeded or non-nil if failed.
Beware: mutates Failed field accordingly.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package buildifier implements processing of Starlark files via buildifier.
|
Package buildifier implements processing of Starlark files via buildifier. |
Package cli contains command line interface for lucicfg tool.
|
Package cli contains command line interface for lucicfg tool. |
base
Package base contains code shared by other CLI subpackages.
|
Package base contains code shared by other CLI subpackages. |
cmds/fmt
Package fmt implements 'fmt' subcommand.
|
Package fmt implements 'fmt' subcommand. |
cmds/generate
Package generate implements 'generate' subcommand.
|
Package generate implements 'generate' subcommand. |
cmds/lint
Package lint implements 'lint' subcommand.
|
Package lint implements 'lint' subcommand. |
cmds/validate
Package validate implements 'validate' subcommand.
|
Package validate implements 'validate' subcommand. |
cmd
|
|
docgen
Command docgen is the documentation generator.
|
Command docgen is the documentation generator. |
lucicfg
Command lucicfg is CLI for LUCI config generator.
|
Command lucicfg is CLI for LUCI config generator. |
Package doc generates starlark documentation.
|
Package doc generates starlark documentation. |
Package graph implements a DAG used internally to represent config objects.
|
Package graph implements a DAG used internally to represent config objects. |
Package starlark contains Starlark code embedded into lucicfg.
|
Package starlark contains Starlark code embedded into lucicfg. |
Package vars implement lucicfg.var() support.
|
Package vars implement lucicfg.var() support. |