lucicfg is a tool for generating low-level LUCI configuration files based on a high-level configuration given as a Starlark script that uses APIs exposed by
lucicfg. In other words, it takes a *.star file (or files) as input and spits out a bunch of *.cfg files (such us
luci-scheduler.cfg) as outputs.
lucicfg follows a “microkernel” architecture. The kernel is implemented in Go. It provides a private interface (used internally by the lucicfg's Starlark standard library) by registering a bunch of builtins. The functionality provided by the kernel is pretty generic:
The builtins are wrapped in two layers of Starlark code:
.../luci: generic (not LUCI specific) Starlark standard library that documents and exposes the builtins via a nicer API. It can be used to build all kinds of config generators (1, 2, 3), as well as extend the LUCI config generator. This API surface is currently marked as “internal” (meaning there‘s no backward compatibility guarantees for it), but it will some day become a part of lucicfg’s public interface, so it should be treated as such (no hacks, no undocumented functions, adequate test coverage, etc).
The standard library and LUCI configs library are bundled with
lucicfg binary via starlark/assets.gen.go file generated from the contents of starlark/stdlib/internal by
lucicfgdirectory (where this README.md file is) run
go generate ./...to regenerate starlark/assets.gen.go, examples/.../generated and doc/README.md.
go generate ./...again: it might have used a stale assets.gen.go the first time.
go test ./...to verify existing tests pass. If your change modifies the format of emitted files you need to update the expected output in test case files. It will most likely happen for testdata/full_example.star. Update
Expect configs:section there.
Expect errors like:sections at the bottom. The test runner will execute the Starlark code and compare the produced output (or errors) to the expectations.
assert.eq(...)etc. See testdata/misc/version.star for an example.
lucicfg uses a variant of semantic versioning to identify its own version and a version of the bundled Starlark libraries. The version string is set in version.go and looks like
If a user script is relying on a feature which is available only in some recent lucicfg version, it can perform a check, like so:
lucicfg.check_version('1.7.8', 'Please update depot_tools')
That way if the script is executed by an older lucicfg version, the user will get a nice actionable error message instead of some obscure stack trace.
Thus it is very important to update version.go before releasing changes:
PATCHversion when you make backward compatible changes. A change is backward compatible if it doesn‘t reduce Starlark API surface and doesn’t affect lucicfg‘s emitted output (assuming inputs do not change). In particular, adding a new feature that doesn’t affect existing features is backward compatible.
MINORversion when you make backward incompatible changes. Releasing such changes may require modifying user scripts or asking users to regenerate their configs to get an updated output. Note that both these things are painful, since there are dozens of repositories with lucicfg scripts and, strictly speaking, all of them should be eventually updated.
MAJORversion is reserved for major architecture changes or rewrites.
If your new feature is experimental and you don't want to commit to any backward compatibility promises, hide it behind an experiment. Users will need to opt-in to use it. See starlark/stdlib/internal/experiments.star for more info.
git_revision:86afde8bddae.... Like this one.
Steps 2 and 3 usually take about 30 min total, and steps 4 and 5 verify CIPD packages actually exist. So in practice it is OK to just land a lucicfg CL, go do other things, then come back >30 min later, look up the revision of the necessary infra.git DEPS roll (or just use the latest one) and proceed to steps 4 and 5.