lucicfg is a tool for generating low-level LUCI configuration files based on a high-level configuration given as a Starlark script that uses APIs exposed by lucicfg. In other words, it takes a *.star file (or files) as input and spits out a bunch of *.cfg files (such us cr-buildbucket.cfg and luci-scheduler.cfg) as outputs. A single entity (such as a luci.builder(...) definition) in the input is translated into multiple entities (such as Buildbucket‘s builder{...} and Scheduler’s job{...}) in the output. This ensures internal consistency of all low-level configs.
Using Starlark allows further reducing duplication and enforcing invariants in the configs. A common pattern is to use Starlark functions that wrap one or more basic rules (e.g. luci.builder(...) and luci.console_view_entry(...)) to define more “concrete” entities (for example “a CI builder” or “a Try builder”). The rest of the config script then uses such functions to build up the actual configuration.
lucicfg is distributed as a single self-contained binary as part of depot_tools, so if you use them, you already have it. Additionally it is available in PATH on all LUCI builders. The rest of this doc also assumes that lucicfg is in PATH.
If you don't use depot_tools, lucicfg can be installed through CIPD. The package is infra/tools/luci/lucicfg/${platform}, and the canonical stable version can be looked up in the depot_tools CIPD manifest.
Finally, you can always try to build lucicfg from the source code. However, the only officially supported distribution mechanism is CIPD packages.
lucicfg can be found here.Create main.star file with the following content:
#!/usr/bin/env lucicfg luci.project( name = "hello-world", buildbucket = "cr-buildbucket.appspot.com", swarming = "chromium-swarm.appspot.com", ) luci.bucket(name = "my-bucket") luci.builder( name = "my-builder", bucket = "my-bucket", executable = luci.recipe( name = "my-recipe", cipd_package = "recipe/bundle/package", ), )
Now run lucicfg generate main.star. It will create a new directory generated side-by-side with main.star file. This directory contains the project.cfg and cr-buildbucket.cfg files, generated based on the script above.
Equivalently, make the script executable (chmod a+x main.star) and then just execute it (./main.star). This is the exact same thing as running generate subcommand.
Now make some change in main.star (for example, rename the builder), but do not regenerate the configs yet. Instead run lucicfg validate main.star. It will produce an error, telling you that files on disk (in generated/*) are stale. Regenerate them (./main.star), and run the validation again.
If you have never done this before or haven't used any other LUCI tools, you are now asked to authenticate by running lucicfg auth-login. This is because lucicfg validate in addition to checking configs locally also sends them for a more thorough validation to the LUCI Config service, and this requires you to be authenticated. Do lucicfg auth-login and re-run lucicfg validate main.star. It should succeed now. If it still fails with permissions issues, you are probably not in config-validation group (this should be rare, please contact luci-eng@google.com if this is happening).
lucicfg validate is meant to be used from presubmit tests. If you use depot_tools' PRESUBMIT.py, there's a canned check that wraps lucicfg validate.
This is it, your first generated config! It is not very functional yet (e.g. builders without Swarming dimensions are useless), but a good place to start. Keep iterating on it, modifying the script, regenerating configs, and examining the output in generated directory. Once you are satisfied with the result, commit both Starlark scripts and generated configs into the repository, and then configure LUCI Config service to pull configuration from generated directory (how to do it is outside the scope of this doc).
lucicfg, not a generic Starlark interpreter. Also this is advanced stuff. Its full understanding is not required to use lucicfg effectively.Each individual Starlark file is called a module. Several modules under the same root directory form a package. Modules within a single package can refer to each other (in load(...) and exec(...)) using their relative or absolute (if start with //) paths. The root of the main package is taken to be a directory that contains the entry point script (usually main.star) passed to lucicfg, i.e. main.star itself can be referred to as //main.star.
Modules can either be “library-like” (executed via load(...) statement) or “script-like” (executed via exec(...) function). Library-like modules can load other library-like modules via load(...), but may not call exec(...). Script-like modules may use both load(...) and exec(...).
Dicts of modules loaded via load(...) are reused, e.g. if two different scripts load the exact same module, they'll get the exact same symbols as a result. The loaded code always executes only once. The interpreter may load modules in parallel in the future, libraries must not rely on their loading order and must not have side effects.
On the other hand, modules executed via exec(...) are guaranteed to be processed sequentially, and only once. Thus ‘exec’-ed scripts essentially form a tree, traversed exactly once in the depth first order.
All entities manipulated by lucicfg are represented by nodes in a directed acyclic graph. One entity (such as a builder) can internally be represented by multiple nodes. A function that adds nodes and edges to the graph is called a rule (e.g. luci.builder(...) is a rule).
Each node has a unique hierarchical key, usually constructed from entity's properties. For example, a builder name and its bucket name are used to construct a unique key for this builder (roughly <bucket>/<builder>). These keys are used internally by rules when adding edges to the graph.
To refer to entities from public API, one just usually uses strings (e.g. a builder name to refer to the builder). Rules' implementation usually have enough context to construct correct node keys from such strings. Sometimes they need some help, see Resolving naming ambiguities. Other times entities have no meaningful global names at all (for example, luci.console_view_entry(...)). For such cases, one uses a return value of the corresponding rule: rules return opaque pointer-like objects that can be passed to other rules as an input in place of a string identifiers. This allows to “chain” definitions, e.g.
luci.console_view( ... entries = [ luci.console_view_entry(...), luci.console_view_entry(...), ... ], )
It is strongly preferred to either use string names to refer to entities or define them inline where they are needed. Please avoid storing return values of rules in variables to refer to them later. Using string names is as powerful (lucicfg verifies referential integrity), and it offers additional advantages (like referring to entities across file boundaries).
To aid in using inline definitions where makes sense, many rules allow entities to be defines multiple times as long as all definitions are identical (this is internally referred to as “idempotent nodes”). It allows following usage style:
def my_recipe(name): return luci.recipe( name = name, cipd_package = 'my/recipe/bundle', ) luci.builder( name = 'builder 1', executable = my_recipe('some-recipe'), ... ) luci.builder( name = 'builder 2', executable = my_recipe('some-recipe'), ... )
Here some-recipe is formally defined twice, but both definitions are identical, so it doesn't cause ambiguities. See the documentation of individual rules to see whether they allow such redefinitions.
There are 3 stages of lucicfg gen execution:
main.star code and all modules it exec's. This builds a graph in memory (via calls to rules), and registers a bunch of generator callbacks (via lucicfg.generator(...)) that will traverse this graph in the stage 3.main.star code. At this point we have a (potentially incomplete) graph and a list of registered generator callbacks.luci.builder("name") refers to undefined luci.bucket("bucket") at <stack trace of the corresponding luci.builder(...) definition>.lucicfg core code, not touching Starlark at all. It doesn't need to understand the semantics of graph nodes, and thus used for all sorts of configs (LUCI configs are just one specific application).Presently all this machinery is mostly hidden from the end user. It will become available in future versions of lucicfg as an API for extending lucicfg, e.g. for adding new entity types that have relation to LUCI, or for repurposing lucicfg for generating non-LUCI conifgs.
Builder names are scoped to buckets. For example, it is possible to have the following definition:
# Runs pre-submit tests on Linux. luci.builder( name = 'Linux', bucket = 'try', ... ) # Runs post-submit tests on Linux. luci.builder( name = 'Linux', bucket = 'ci', ... )
Here Linux name by itself is ambiguous and can't be used to refer to the builder. E.g. the following chunk of code will cause an error:
luci.list_view_entry( builder = 'Linux', # but which one?... ... )
The fix is to prepend the bucket name:
luci.list_view_entry( builder = 'ci/Linux', # ah, the CI one ... )
It is always correct to use “full” name like this. But in practice the vast majority of real world configs do not have such ambiguities and requiring full names everywhere is a chore. For that reason lucicfg allows to omit the bucket name if the resulting reference is non-ambiguous. In the example above, if we remove one of the builders, builder = 'Linux' reference becomes valid.
Some LUCI Services allow one project to refer to resources in another project. For example, a luci.console_view(...) can display builders that belong to another LUCI project, side-by-side with the builders from the project the console belongs to.
Such external builders can be referred to via their fully qualified name in the format <project>:<bucket>/<name>. Note that <bucket> part can't be omitted.
For example:
luci.console_view_entry( builder = 'chromium:ci/Linux Builder', ... )
luci.builder(...) and luci.gitiles_poller(...) rules have schedule field that defines how often the builder or poller should run. Schedules are given as strings. Supported kinds of schedules (illustrated via examples):
* 0 * * * *: a crontab expression, in a syntax supported by https://github.com/gorhill/cronexpr (see its docs for full reference). LUCI will attempt to start the job at specified moments in time (based on UTC clock). Some examples:
0 */3 * * * * - every 3 hours: at 12:00 AM UTC, 3:00 AM UTC, ...0 */3 * * * - the exact same thing (the last field is optional).0 1/3 * * * - every 3 hours but starting 1:00 AM UTC.0 2,10,18 * * * - at 2 AM UTC, 10 AM UTC, 6 PM UTC.0 7 * * * - at 7 AM UTC, once a day.If a previous invocation is still running when triggering a new one, an overrun is recorded and the new scheduled invocation is skipped. The next attempt to start the job happens based on the schedule (not when the currently running invocation finishes).
with 10s interval: run the job in a loop, waiting 10s after finishing an invocation before starting a new one. Moments when the job starts aren't synchronized with the wall clock at all.
with 1m interval, with 1h interval, with 1h10m interval: same format, just using minutes and hours instead of seconds.
continuously is alias for with 0s interval, meaning to run the job in a loop without any pauses at all.
triggered schedule indicates that the job is only started via some external triggering event (e.g. via LUCI Scheduler API), not periodically.
lucicfg uses buildifier internally to format and lint Starlark code. Buildifier is primarily intended for Bazel BUILD and *.bzl files, but it works with lucicfg's *.star files reasonably well too.
To format a single Starlark file use lucicfg fmt path.star. To format all *.star files in a directory (recursively) use lucicfg fmt <dir>.
There are two ways to run lint checks:
lucicfg lint <path>. What set of checks to perform can be specified via -check <set> argument, where <set> is a special comma-delimited string that identifies what checks to apply. See below for how to construct it.lucicfg validate <entry point>.star. It will check only files loaded while executing the entry point script. This is the recommended way. The set of checks to apply can be specified via lint_checks argument in lucicfg.config(...), see below for examples. Note that all checks (including formatting checks) are disabled by default for now. This will change in the future.Checking that files are properly formatted is a special kind of a lint check called formatting.
Both lucicfg lint -check ... CLI argument and lint_checks in lucicfg.config(...) accept a list of strings that looks like [<initial set>], +warn1, +warn2, -warn3, -warn4, ... , where
<initial set> can be one of default, none or all and it identifies a set of linter checks to use as a base:default is a set of checks that are known to work well with lucicfg Starlark code. If <initial set> is omitted, default is used.none is an empty set.all is all checks known to buildifier. Note that some of them may be incompatible with lucicfg Starlark code.+warn adds some specific check to the set of checks to apply.-warn removes some specific check from the set of checks to apply.See buildifier warnings list for identifiers and meanings of all possible checks. Note that many of them are specific to Bazel not applicable to lucicfg Starlark code.
Additionally a check called formatting can be used to instruct lucicfg to verify formatting of Starlark files. It is part of the default set. Note that it is not a built-in buildifier check and thus it's not listed in the buildifier docs nor can it be disabled via buildifier: disable=....
To apply all default checks when running lucicfg validate use:
lucicfg.config( ... lint_checks = ["default"], )
This is equivalent to running lucicfg lint -checks default or just lucicfg lint.
To check formatting only:
lucicfg.config( ... lint_checks = ["none", "+formatting"], )
This is equivalent to running lucicfg lint -checks "none,+formatting".
To disable some single default check (e.g. function-docstring) globally:
lucicfg.config( ... lint_checks = ["-function-docstring"], )
This is equivalent to running lucicfg lint -checks "-function-docstring".
To suppress a specific occurrence of a linter warning add a special comment # buildifier: disable=<check-name> to the expression that causes the warning:
# buildifier: disable=function-docstring def update_submodules_mirror( name, short_name, source_repo, target_repo, extra_submodules = None, triggered_by = None, refs = None): properties = { "source_repo": source_repo, "target_repo": target_repo, } ...
To suppress formatting changes (and thus formatting check) use # buildifier: leave-alone.
lucicfg.version()
Returns a triple with lucicfg version: (major, minor, revision).
lucicfg.check_version(min, message = None)
Fails if lucicfg version is below the requested minimal one.
Useful when a script depends on some lucicfg feature that may not be available in earlier versions. lucicfg.check_version(...) can be used at the start of the script to fail right away with a clean error message:
lucicfg.check_version( min = "1.30.14", message = "Update depot_tools", )
Or even
lucicfg.check_version("1.30.14")
Additionally implicitly auto-enables not-yet-default lucicfg functionality released with the given version. That way lucicfg changes can be gradually rolled out project-by-project by bumping the version string passed to lucicfg.check_version(...) in project configs.
major.minor.revision with minimally accepted version. Required.lucicfg.config( # Optional arguments. config_service_host = None, config_dir = None, tracked_files = None, fail_on_warnings = None, lint_checks = None, )
Sets one or more parameters for the lucicfg itself.
These parameters do not affect semantic meaning of generated configs, but influence how they are generated and validated.
Each parameter has a corresponding command line flag. If the flag is present, it overrides the value set via lucicfg.config (if any). For example, the flag -config-service-host <value> overrides whatever was set via lucicfg.config(config_service_host=...).
lucicfg.config is allowed to be called multiple times. The most recently set value is used in the end, so think of lucicfg.config(var=...) just as assigning to a variable.
lucicfg binary, usually config.luci.app... is allowed. If set via -config-dir command line flag, it is relative to the current working directory. Will be created if absent. If -, the configs are just printed to stdout in a format useful for debugging. Default is “generated”.config_dir that are considered generated. Each entry is either <glob pattern> (a “positive” glob) or !<glob pattern> (a “negative” glob). A file under config_dir is considered tracked if its slash-separated path matches any of the positive globs and none of the negative globs. If a pattern starts with **/, the rest of it is applied to the base name of the file (not the whole path). If only negative globs are given, single positive **/* glob is implied as well. tracked_files can be used to limit what files are actually emitted: if this set is not empty, only files that are in this set will be actually written to the disk (and all other files are discarded). This is beneficial when lucicfg is used to generate only a subset of config files, e.g. during the migration from handcrafted to generated configs. Knowing the tracked files set is also important when some generated file disappears from lucicfg output: it must be deleted from the disk as well. To do this, lucicfg needs to know what files are safe to delete. If tracked_files is empty (default), lucicfg will save all generated files and will never delete any file in this case it is responsibility of the caller to make sure no stale output remains).lucicfg.config and you want to override it to False via command line flags use -fail-on-warnings=false.lucicfg validate. The first entry defines what group of checks to use as a base and it can be one of none, default or all. The following entries either add checks to the set (+<name>) or remove them (-<name>). See Formatting and linting Starlark code for more info. Default is ['none'] for now.lucicfg.enable_experiment(experiment)
Enables an experimental feature.
Can be used to experiment with non-default features that may later change in a non-backwards compatible way or even be removed completely. Primarily intended for lucicfg developers to test their features before they are “frozen” to be backward compatible.
Enabling an experiment that doesn‘t exist logs a warning, but doesn’t fail the execution. Refer to the documentation and the source code for the list of available experiments.
lucicfg.generator(impl = None)
Registers a generator callback.
Such callback is called at the end of the config generation stage to modify/append/delete generated configs in an arbitrary way.
The callback accepts single argument ctx which is a struct with the following fields and methods:
output: a dict {config file name -> (str | proto)}. The callback is free to modify ctx.output in whatever way it wants, e.g. by adding new values there or mutating/deleting existing ones.
declare_config_set(name, root): proclaims that generated configs under the given root (relative to config_dir) belong to the given config set. Safe to call multiple times with exact same arguments, but changing an existing root to something else is an error.
func(ctx) -> None.lucicfg.emit(dest, data)
Tells lucicfg to write given data to some output file.
In particular useful in conjunction with io.read_file(...) to copy files into the generated output:
lucicfg.emit( dest = "foo.cfg", data = io.read_file("//foo.cfg"), )
Note that lucicfg.emit(...) cannot be used to override generated files. dest must refer to a path not generated or emitted by anything else.
config_dir (see lucicfg.config(...)). Must not start with ../. Required.dest. Proto messages are serialized using text protobuf encoding. Required.lucicfg.current_module()
Returns the location of a module being currently executed.
This is the module being processed by a current load(...) or exec(...) statement. It has no relation to the module that holds the top-level stack frame. For example, if a currently loading module A calls a function in a module B and this function calls lucicfg.current_module(...), the result would be the module A, even though the call goes through code in the module B (i.e. lucicfg.current_module(...) invocation itself resided in a function in module B).
Fails if called from inside a generator callback. Threads executing such callbacks are not running any load(...) or exec(...).
A struct(package='...', path='...') with the location of the module.
lucicfg.var(default = None, validator = None, expose_as = None)
Declares a variable.
A variable is a slot that can hold some frozen value. Initially this slot is usually empty. lucicfg.var(...) returns a struct with methods to manipulate it:
set(value): sets the variable‘s value if it’s unset, fails otherwise.get(): returns the current value, auto-setting it to default if it was unset.Note the auto-setting the value in get() means once get() is called on an unset variable, this variable can't be changed anymore, since it becomes initialized and initialized variables are immutable. In effect, all callers of get() within a scope always observe the exact same value (either an explicitly set one, or a default one).
Any module (loaded or exec'ed) can declare variables via lucicfg.var(...). But only modules running through exec(...) can read and write them. Modules being loaded via load(...) must not depend on the state of the world while they are loading, since they may be loaded at unpredictable moments. Thus an attempt to use get or set from a loading module causes an error.
Note that functions exported by loaded modules still can do anything they want with variables, as long as they are called from an exec-ing module. Only code that executes while the module is loading is forbidden to rely on state of variables.
Assignments performed by an exec-ing module are visible only while this module and all modules it execs are running. As soon as it finishes, all changes made to variable values are “forgotten”. Thus variables can be used to implicitly propagate information down the exec call stack, but not up (use exec's return value for that).
Generator callbacks registered via lucicfg.generator(...) are forbidden to read or write variables, since they execute outside of context of any exec(...). Generators must operate exclusively over state stored in the node graph. Note that variables still can be used by functions that build the graph, they can transfer information from variables into the graph, if necessary.
The most common application for lucicfg.var(...) is to “configure” library modules with default values pertaining to some concrete executing script:
This is more magical but less wordy alternative to either passing specific default values in every call to library functions, or wrapping all library functions with wrappers that supply such defaults. These more explicit approaches can become pretty convoluted when there are multiple scripts and libraries involved.
Another use case is to allow parameterizing configs with values passed via CLI flags. A string-typed var can be declared with expose_as=<name> argument, making it settable via -var <name>=<value> CLI flag. This is primarily useful in conjunction with -emit-to-stdout CLI flag to use lucicfg as a “function call” that accepts arguments via CLI flags and returns the result via stdout to pipe somewhere else, e.g.
lucicfg generate main.star -var environ=dev -emit-to-stdout all.json | ...
Danger: Using -var without -emit-to-stdout is generally wrong, since configs generated on disk (and presumably committed into a repository) must not depend on undetermined values passed via CLI flags.
get() if it was unset.validator(value) from set(value) and inside lucicfg.var(...) declaration itself (to validate default or a value passed via CLI flags). Must be a side-effect free idempotent function that returns the value to be assigned to the variable (usually just value itself, but conversions are allowed, including type changes).-var <expose_as>=<value>. If there's no such flag, the variable is auto-initialized to its default value (which must be string or None). Variables declared with expose_as are not settable via set() at all, they appear as “set” already the moment they are declared. If multiple vars use the same expose_as identifier, they will all be initialized to the same value.A struct with two methods: set(value) and get(): value.
lucicfg.rule(impl, defaults = None)
Declares a new rule.
A rule is a callable that adds nodes and edges to an entity graph. It wraps the given impl callback by passing one additional argument ctx to it (as the first positional argument).
ctx is a struct with the following fields:
defaults: a struct with module-scoped defaults for the rule.The callback is expected to return a graph.keyset(...) with the set of graph keys that represent the added node (or nodes). Other rules use such keysets as inputs.
ctx. The rest of the arguments define the API of the rule. Required.rule.defaults.<name>.set(...). impl callback can get them via ctx.defaults.<name>.get(). It is up to the rule's author to define vars for fields that can have defaults, document them in the rule doc, and finally use them from impl callback.A special callable.
Time module provides a simple API for defining durations in a readable way, resembling golang's time.Duration.
Durations are represented by integer-like values of time.duration(...) type, which internally hold a number of milliseconds.
Durations can be added and subtracted from each other and multiplied by integers to get durations. They are also comparable to each other (but not to integers). Durations can also be divided by each other to get an integer, e.g. time.hour / time.second produces 3600.
The best way to define a duration is to multiply an integer by a corresponding “unit” constant, for example 10 * time.second.
Following time constants are exposed:
| Constant | Value (obviously) |
|---|---|
time.zero | 0 milliseconds |
time.millisecond | 1 millisecond |
time.second | 1000 * time.millisecond |
time.minute | 60 * time.second |
time.hour | 60 * time.minute |
time.day | 24 * time.hour |
time.week | 7 * time.day |
time.duration(milliseconds)
Returns a duration that represents the integer number of milliseconds.
time.duration value.
time.epoch(layout, value, location)
Returns epoch seconds for value interpreted as a time per layout in location.
int epoch seconds for value.
time.truncate(duration, precision)
Truncates the precision of the duration to the given value.
For example time.truncate(time.hour+10*time.minute, time.hour) is time.hour.
Truncated time.duration value.
time.days_of_week(spec)
Parses e.g. Tue,Fri-Sun into a list of day indexes, e.g. [2, 5, 6, 7].
Monday is 1, Sunday is 7. The returned list is sorted and has no duplicates. An empty string results in the empty list.
Tue), or a range (e.g. Wed-Sun). Required.A list of 1-based day indexes. Monday is 1.
luci.project( # Required arguments. name, # Optional arguments. config_dir = None, dev = None, buildbucket = None, logdog = None, milo = None, notify = None, scheduler = None, swarming = None, tricium = None, acls = None, bindings = None, enforce_realms_in = None, omit_lucicfg_metadata = None, )
Defines a LUCI project.
There should be exactly one such definition in the top-level config file.
This rule also implicitly defines the @root realm of the project. It can be used to setup permissions that apply to all resources in the project. See luci.realm(...).
config_dir in lucicfg.config(...)) to place generated LUCI configs under. Default is .. A custom value is useful when using lucicfg to generate LUCI and non-LUCI configs at the same time.bindings.acls.enforce_in in luci.realm(...).lucicfg {...} block with lucicfg invocation details in project.cfg. This may be useful if you pass frequently changing -var ... when generating configs and the resulting generated lucicfg { vars {...} } metadata causes frequent merge conflicts. This option is strongly discouraged as it makes it impossible to reproducibly regenerate project configs in the LUCI automation (it doesn‘t know what var values to use). If you use this option, your project may be left out of automatic config migrations. If this happens, you’ll need to manually complete the migration on-schedule in order to have your LUCI project continue to function.luci.realm( # Required arguments. name, # Optional arguments. extends = None, bindings = None, enforce_in = None, )
Defines a realm.
Realm is a named collection of (<principal>, <permission>) pairs.
A LUCI resource can point to exactly one realm by referring to its full name (<project>:<realm>). We say that such resource “belongs to the realm” or “lives in the realm” or is just “in the realm”. We also say that such resource belongs to the project <project>. The corresponding luci.realm(...) definition then describes who can do what to the resource.
The logic of how resources get assigned to realms is a part of the public API of the service that owns resources. Some services may use a static realm assignment via project configuration files, others may do it dynamically by accepting a realm when a resource is created via an RPC.
A realm can “extend” one or more other realms. If a realm A extends B, then all permissions defined in B are also in A. Remembering that a realm is just a set of (<principal>, <permission>) pairs, the “extends” relation is just a set inclusion.
There are three special realms that a project can have: “@root”, “@legacy” and “@project”.
The root realm is implicitly included into all other realms (including “@legacy”), and it is also used as a fallback when a resource points to a realm that no longer exists. Without the root realm, such resources become effectively inaccessible and this may be undesirable. Permissions in the root realm apply to all realms in the project (current, past and future), and thus the root realm should contain only administrative-level bindings. If you are not sure whether you should use the root realm or not, err on the side of not using it.
The legacy realm is used for existing resources created before the realms mechanism was introduced. Such resources usually are not associated with any realm at all. They are implicitly placed into the legacy realm to allow reusing realms' machinery for them.
Note that the details of how resources are placed in the legacy realm are up to a particular service implementation. Some services may be able to figure out an appropriate realm for a legacy resource based on resource's existing attributes. Some services may not have legacy resources at all. The legacy realm is not used in these case. Refer to the service documentation.
The project realm should be used as the realm for ‘project global’ resources, for example, the project configuration itself, or derivations thereof. Some LUCI services may use bindings in this realm to allow federation of administration responsibilities to the project (rather than relying on exclusively LUCI service administrators).
The primary way of populating the permission set of a realm is via bindings. Each binding assigns a role to a set of principals (individuals, groups or LUCI projects). A role is just a set of permissions. A binding grants these permissions to all principals listed in it.
Binding can be specific either right here:
luci.realm(
name = 'try',
bindings = [
luci.binding(
roles = 'role/a',
groups = ['group-a'],
),
luci.binding(
roles = 'role/b',
groups = ['group-b'],
),
],
)
Or separately one by one via luci.binding(...) declarations:
luci.binding(
realm = 'try',
roles = 'role/a',
groups = ['group-a'],
)
luci.binding(
realm = 'try',
roles = 'role/b',
groups = ['group-b'],
)
[a-z0-9_\.\-/]{1,400} or be @root or @legacy. Required.@root.luci.binding( # Required arguments. roles, # Optional arguments. realm = None, groups = None, users = None, projects = None, conditions = None, )
Binding assigns roles in a realm to individuals, groups or LUCI projects.
A role can either be predefined (if its name starts with role/) or custom (if its name starts with customRole/).
Predefined roles are declared in the LUCI deployment configs, see TODO for the up-to-date list of available predefined roles and their meaning.
Custom roles are defined in the project configs via luci.custom_role(...). They can be used if none of the predefined roles represent the desired set of permissions.
luci.restrict_attribute(attribute = None, values = None)
A condition for luci.binding(...) to restrict allowed attribute values.
When a service checks a permission, it passes to the authorization library a string-valued dictionary of attributes that describes the context of the permission check. It contains things like the name of the resource being accessed, or parameters of the incoming RPC request that triggered the check.
luci.restrict_attribute(...) condition makes the binding active only if the value of the given attribute is in the given set of allowed values.
A list of available attributes and meaning of their values depends on the permission being checked and is documented in the corresponding service documentation.
An opaque struct that can be passed to luci.binding(...) as a condition.
luci.custom_role(name, extends = None, permissions = None)
Defines a custom role.
It can be used in luci.binding(...) if predefined roles are too broad or do not map well to the desired set of permissions.
Custom roles are scoped to the project (i.e. different projects may have identically named, but semantically different custom roles).
customRole/. Required.role/) or another custom role (if it is a string that starts with customRole/ or a luci.custom_role(...) key).<service>.<subject>.<verb>, which describes some elementary action (<verb>) that can be done to some category of resources (<subject>), managed by some particular kind of LUCI service (<service>). See TODO for the up-to-date list of available permissions and their meaning.luci.logdog(gs_bucket = None, cloud_logging_project = None)
Defines configuration of the LogDog service for this project.
Usually required for any non-trivial project.
luci.bucket( # Required arguments. name, # Optional arguments. acls = None, extends = None, bindings = None, shadows = None, constraints = None, dynamic = None, )
Defines a bucket: a container for LUCI builds.
This rule also implicitly defines the realm to use for the builds in this bucket. It can be used to specify permissions that apply to all builds in this bucket and all resources these builds produce. See luci.realm(...).
ci or try. Required.bindings.@root.acls.luci.executable( # Required arguments. name, # Optional arguments. cipd_package = None, cipd_version = None, cmd = None, wrapper = None, )
Defines an executable.
Builders refer to such executables in their executable field, see luci.builder(...). Multiple builders can execute the same executable (perhaps passing different properties to it).
Executables must be available as cipd packages.
The cipd version to fetch is usually a lower-cased git ref (like refs/heads/main), or it can be a cipd tag (like git_revision:abc...).
A luci.executable(...) with some particular name can be redeclared many times as long as all fields in all declaration are identical. This is helpful when luci.executable(...) is used inside a helper function that at once declares a builder and an executable needed for this builder.
refs/heads/main. Supports the module-scoped default.('recipes',) or ('luciexe',) will be used by Buildbucket, according to its global configuration. The special value of ('recipes',) indicates that this executable should be run under the legacy kitchen runtime. All other values will be executed under the go.chromium.org/luci/luciexe protocol.cmd. If set, the builder will run <wrapper> -- <cmd>. The 0th argument of the wrapper may be an absolute path. It is up to the owner of the builder to ensure that the wrapper executable is distributed to whatever machines this executable may run on.luci.recipe( # Required arguments. name, # Optional arguments. cipd_package = None, cipd_version = None, recipe = None, use_bbagent = None, use_python3 = None, wrapper = None, )
Defines an executable that runs a particular recipe.
Recipes are python-based DSL for defining what a builder should do, see recipes-py.
Builders refer to such executable recipes in their executable field, see luci.builder(...). Multiple builders can execute the same recipe (perhaps passing different properties to it).
Recipes are located inside cipd packages called “recipe bundles”. Typically the cipd package name with the recipe bundle will look like:
infra/recipe_bundles/chromium.googlesource.com/chromium/tools/build
Recipes bundled from internal repositories are typically under
infra_internal/recipe_bundles/...
But if you're building your own recipe bundles, they could be located elsewhere.
The cipd version to fetch is usually a lower-cased git ref (like refs/heads/main), or it can be a cipd tag (like git_revision:abc...).
A luci.recipe(...) with some particular name can be redeclared many times as long as all fields in all declaration are identical. This is helpful when luci.recipe(...) is used inside a helper function that at once declares a builder and a recipe needed for this builder.
recipe is None, also specifies the recipe name within the bundle. Required.refs/heads/main. Supports the module-scoped default.name. Useful if recipe names clash between different recipe bundles. When this happens, name can be used as a non-ambiguous alias, and recipe can provide the actual recipe name. Defaults to name.<wrapper> -- <luciexe>. The 0th argument of the wrapper may be an absolute path. It is up to the owner of the builder to ensure that the wrapper executable is distributed to whatever machines this executable may run on.luci.builder( # Required arguments. name, bucket, executable, # Optional arguments. description_html = None, properties = None, allowed_property_overrides = None, service_account = None, caches = None, execution_timeout = None, grace_period = None, heartbeat_timeout = None, dimensions = None, priority = None, swarming_host = None, swarming_tags = None, expiration_timeout = None, wait_for_capacity = None, retriable = None, schedule = None, triggering_policy = None, build_numbers = None, experimental = None, experiments = None, task_template_canary_percentage = None, repo = None, resultdb_settings = None, test_presentation = None, backend = None, backend_alt = None, shadow_service_account = None, shadow_pool = None, shadow_properties = None, shadow_dimensions = None, triggers = None, triggered_by = None, notifies = None, contact_team_email = None, custom_metrics = None, )
Defines a generic builder.
It runs some executable (usually a recipe) in some requested environment, passing it a struct with given properties. It is launched whenever something triggers it (a poller or some other builder, or maybe some external actor via Buildbucket or LUCI Scheduler APIs).
The full unique builder name (as expected by Buildbucket RPC interface) is a pair (<project>, <bucket>/<name>), but within a single project config this builder can be referred to either via its bucket-scoped name (i.e. <bucket>/<name>) or just via it‘s name alone (i.e. <name>), if this doesn’t introduce ambiguities.
The definition of what can potentially trigger what is defined through triggers and triggered_by fields. They specify how to prepare ACLs and other configuration of services that execute builds. If builder A is defined as “triggers builder B”, it means all services should expect A builds to trigger B builds via LUCI Scheduler‘s EmitTriggers RPC or via Buildbucket’s ScheduleBuild RPC, but the actual triggering is still the responsibility of A's executable.
There‘s a caveat though: only Scheduler ACLs are auto-generated by the config generator when one builder triggers another, because each Scheduler job has its own ACL and we can precisely configure who’s allowed to trigger this job. Buildbucket ACLs are left unchanged, since they apply to an entire bucket, and making a large scale change like that (without really knowing whether Buildbucket API will be used) is dangerous. If the executable triggers other builds directly through Buildbucket, it is the responsibility of the config author (you) to correctly specify Buildbucket ACLs, for example by adding the corresponding service account to the bucket ACLs:
luci.bucket( ... acls = [ ... acl.entry(acl.BUILDBUCKET_TRIGGERER, <builder service account>), ... ], )
This is not necessary if the executable uses Scheduler API instead of Buildbucket.
properties parameter here (or via the module-scoped defaults) are allowed.execution_timeout or after a Cancel event, before the build is forcefully shut down. Your build can use this time as a ‘last gasp’ to do quick actions like killing child processes, cleaning resources, etc. Supports the module-scoped default.INFRA_FAILURE. If None, Buildbucket won‘t check the heartbeat timeout. This field only takes effect for builds that don’t have Buildbucket managing their underlying backend tasks, namely the ones on TaskBackendLite. E.g. builds running on Swarming don't need to set this.os), and values are either strings (e.g. Linux), swarming.dimension(...) objects (for defining expiring dimensions) or lists of thereof. Supports the module-scoped defaults. They are merged (non-recursively) with the explicitly passed dimensions.k:v strings).dimensions) before canceling the build and marking it as expired. If None, defer the decision to Buildbucket service. Supports the module-scoped default.expiration_timeout even if it has never seen a bot whose dimensions are a superset of the requested dimensions. This is useful if this builder has bots whose dimensions are mutated dynamically. Supports the module-scoped default.https://) associated with the builder, if known. It is in particular important when using luci.notifier(...) to let LUCI know what git history it should use to chronologically order builds on this builder. If unknown, builds will be ordered by creation time. If unset, will be taken from the configuration of luci.gitiles_poller(...) that trigger this builder if they all poll the same repo.role/buildbucket.builderServiceAccount role in the shadow bucket realm.notified_by field in luci.notifier(...) or luci.tree_closer(...).luci.gitiles_poller( # Required arguments. name, bucket, repo, # Optional arguments. refs = None, path_regexps = None, path_regexps_exclude = None, schedule = None, triggers = None, )
Defines a gitiles poller which can trigger builders on git commits.
It periodically examines the state of watched refs in the git repository. On each iteration it triggers builders if either:
A watched ref‘s tip has changed since the last iteration (e.g. a new commit landed on a ref). Each new detected commit results in a separate triggering request, so if for example 10 new commits landed on a ref since the last poll, 10 new triggering requests will be submitted to the builders triggered by this poller. How they are converted to actual builds depends on triggering_policy of a builder. For example, some builders may want to have one build per commit, others don’t care and just want to test the latest commit. See luci.builder(...) and scheduler.policy(...) for more details.
A ref belonging to the watched set has just been created. This produces a single triggering request for the commit at the ref's tip. This also applies right after a configuration change which instructs the scheduler to watch a new ref.
Commits that trigger builders can also optionally be filtered by file paths they touch. These conditions are specified via path_regexps and path_regexps_exclude fields, each is a list of regular expressions against Unix file paths relative to the repository root. A file is considered “touched” if it is either added, modified, removed, moved (both old and new paths are considered “touched”), or its metadata has changed (e.g. chmod +x).
A triggering request is emitted for a commit if only if at least one touched file is not matched by any path_regexps_exclude and simultaneously matched by some path_regexps, subject to following caveats:
path_regexps = [".+"] will not match commits which modify no files (aka empty commits) and as such this situation differs from the default case of not specifying any path_regexps.A luci.gitiles_poller(...) with some particular name can be redeclared many times as long as all fields in all declaration are identical. This is helpful when luci.gitiles_poller(...) is used inside a helper function that at once declares a builder and a poller that triggers this builder.
https://. Required.refs/heads/[^/]+ or refs/branch-heads/\d+\.\d+. The regular expression should have a literal prefix with at least two slashes present, e.g. refs/release-\d+/foobar is not allowed, because the literal prefix refs/release- contains only one slash. The regexp should not start with ^ or end with $ as they will be added automatically. Each supplied regexp must match at least one ref in the gitiles output, e.g. specifying refs/tags/v.+ for a repo that doesn't have tags starting with v causes a runtime error. If empty, defaults to ['refs/heads/main'].^ and $ are implied and should not be specified manually. See the explanation above for all details.^ and $ are implied and should not be specified manually. See the explanation above for all details.luci.milo(logo = None, favicon = None, bug_url_template = None)
Defines optional configuration of the Milo service for this project.
Milo service is a public user interface for displaying (among other things) builds, builders, builder lists (see luci.list_view(...)) and consoles (see luci.console_view(...)).
Can optionally be configured with a bug_url_template for filing bugs via custom bug links on build pages. The protocol must be https and the domain name must be one of the allowed domains (see Project.bug_url_template for details).
The template is interpreted as a mustache template and the following variables are available:
All variables are URL component encoded. Additionally, use {{{ ... }}} to disable HTML escaping. If the template does not satisfy the requirements above, the link is not displayed.
storage.googleapis.com.storage.googleapis.com.luci.list_view( # Required arguments. name, # Optional arguments. title = None, favicon = None, entries = None, )
A Milo UI view that displays a list of builders.
Builders that belong to this view can be specified either right here:
luci.list_view(
name = 'Try builders',
entries = [
'win',
'linux',
luci.list_view_entry('osx'),
],
)
Or separately one by one via luci.list_view_entry(...) declarations:
luci.list_view(name = 'Try builders')
luci.list_view_entry(
builder = 'win',
list_view = 'Try builders',
)
luci.list_view_entry(
builder = 'linux',
list_view = 'Try builders',
)
Note that list views support builders defined in other projects. See Referring to builders in other projects for more details.
name.storage.googleapis.com. Defaults to favicon in luci.milo(...).luci.list_view_entry(builder = None, list_view = None)
A builder entry in some luci.list_view(...).
Can be used to declare that a builder belongs to a list view outside of the list view declaration. In particular useful in functions. For example:
luci.list_view(name = 'Try builders')
def try_builder(name, ...):
luci.builder(name = name, ...)
luci.list_view_entry(list_view = 'Try builders', builder = name)
Can also be used inline in luci.list_view(...) declarations, for consistency with corresponding luci.console_view_entry(...) usage. list_view argument can be omitted in this case:
luci.list_view(
name = 'Try builders',
entries = [
luci.list_view_entry(builder = 'Win'),
...
],
)
list_view_entry is used inline inside some luci.list_view(...) declaration.luci.console_view( # Required arguments. name, repo, # Optional arguments. title = None, refs = None, exclude_ref = None, header = None, include_experimental_builds = None, favicon = None, default_commit_limit = None, default_expand = None, entries = None, )
A Milo UI view that displays a table-like console.
In this view columns are builders and rows are git commits on which builders are triggered.
A console is associated with a single git repository it uses as a source of commits to display as rows. The watched ref set is defined via refs and optional exclude_ref fields. If refs are empty, the console defaults to watching refs/heads/main.
exclude_ref is useful when watching for commits that landed specifically on a branch. For example, the config below allows to track commits from all release branches, but ignore the commits from the main branch, from which these release branches are branched off:
luci.console_view(
...
refs = ['refs/branch-heads/\d+\.\d+'],
exclude_ref = 'refs/heads/main',
...
)
For best results, ensure commits on each watched ref have committer timestamps monotonically non-decreasing. Gerrit will take care of this if you require each commit to go through Gerrit by prohibiting “git push” on these refs.
Builders that belong to the console can be specified either right here:
luci.console_view(
name = 'CI builders',
...
entries = [
luci.console_view_entry(
builder = 'Windows Builder',
short_name = 'win',
category = 'ci',
),
# Can also pass a dict, this is equivalent to passing
# luci.console_view_entry(**dict).
{
'builder': 'Linux Builder',
'short_name': 'lnx',
'category': 'ci',
},
...
],
)
Or separately one by one via luci.console_view_entry(...) declarations:
luci.console_view(name = 'CI builders')
luci.console_view_entry(
builder = 'Windows Builder',
console_view = 'CI builders',
short_name = 'win',
category = 'ci',
)
Note that consoles support builders defined in other projects. See Referring to builders in other projects for more details.
Consoles can have headers which are collections of links, oncall rotation information, and console summaries that are displayed at the top of a console, below the tree status information. Links and oncall information is always laid out to the left, while console groups are laid out to the right. Each oncall and links group take up a row.
Header definitions are based on Header message in Milo's project.proto. There are two way to supply this message via header field:
Pass an appropriately structured dict. Useful for defining small headers inline:
luci.console_view(
...
header = {
'links': [
{'name': '...', 'links': [...]},
...
],
},
...
)
Pass a string. It is treated as a path to a file with serialized Header message. Depending on its extension, it is loaded ether as JSONPB-encoded message (*.json and *.jsonpb paths), or as TextPB-encoded message (everything else):
luci.console_view(
...
header = '//consoles/main_header.textpb',
...
)
name.https://. Required.refs/heads/[^/]+ or refs/branch-heads/\d+\.\d+. The regular expression should have a literal prefix with at least two slashes present, e.g. refs/release-\d+/foobar is not allowed, because the literal prefix refs/release- contains only one slash. The regexp should not start with ^ or end with $ as they will be added automatically. If empty, defaults to ['refs/heads/main'].refs and refs_regexps. Note that force pushes to this ref are not supported. Milo uses caching assuming set of commits reachable from this ref may only grow, never lose some commits.storage.googleapis.com. Defaults to favicon in luci.milo(...).luci.console_view_entry( # Optional arguments. builder = None, short_name = None, category = None, console_view = None, )
A builder entry in some luci.console_view(...).
Used inline in luci.console_view(...) declarations to provide category and short_name for a builder. console_view argument can be omitted in this case:
luci.console_view(
name = 'CI builders',
...
entries = [
luci.console_view_entry(
builder = 'Windows Builder',
short_name = 'win',
category = 'ci',
),
...
],
)
Can also be used to declare that a builder belongs to a console outside of the console declaration. In particular useful in functions. For example:
luci.console_view(name = 'CI builders') def ci_builder(name, ...): luci.builder(name = name, ...) luci.console_view_entry(console_view = 'CI builders', builder = name)
term1|term2|... that describes the hierarchy of the builder columns. Neighboring builders with common ancestors will have their column headers merged. In expanded view, each leaf category or builder under a non-leaf category will have it's own column. The recommendation for maximum density is not to mix subcategories and builders for children of each category.console_view_entry is used inline inside some luci.console_view(...) declaration.luci.external_console_view(name, source, title = None)
Includes a Milo console view from another project.
This console will be listed in the Milo UI on the project page, alongside the consoles native to this project.
In the following example, we include a console from the ‘chromium’ project called ‘main’, and we give it a local name of ‘cr-main’ and title of ‘Chromium Main Console’.
luci.external_console_view(
name = 'cr-main',
title = 'Chromium Main Console',
source = 'chromium:main'
)
name.project:console_id. Required.luci.notify(tree_closing_enabled = None)
Defines configuration of the LUCI-Notify service for this project.
luci.notifier( # Required arguments. name, # Optional arguments. on_occurrence = None, on_new_status = None, on_failure = None, on_new_failure = None, on_status_change = None, on_success = None, failed_step_regexp = None, failed_step_regexp_exclude = None, notify_emails = None, notify_rotation_urls = None, notify_blamelist = None, blamelist_repos_whitelist = None, template = None, notified_by = None, )
Defines a notifier that sends notifications on events from builders.
A notifier contains a set of conditions specifying what events are considered interesting (e.g. a previously green builder has failed), and a set of recipients to notify when an interesting event happens. The conditions are specified via on_* fields, and recipients are specified via notify_* fields.
The set of builders that are being observed is defined through notified_by field here or notifies field in luci.builder(...). Whenever a build finishes, the builder “notifies” all luci.notifier(...) objects subscribed to it, and in turn each notifier filters and forwards this event to corresponding recipients.
Note that luci.notifier(...) and luci.tree_closer(...) are both flavors of a luci.notifiable object, i.e. both are something that “can be notified” when a build finishes. They both are valid targets for notifies field in luci.builder(...). For that reason they share the same namespace, i.e. it is not allowed to have a luci.notifier(...) and a luci.tree_closer(...) with the same name.
SUCCESS, FAILURE, and INFRA_FAILURE. Default is None.SUCCESS, FAILURE, and INFRA_FAILURE. Default is None.on_new_status or on_occurrence instead. If True, notify on each build failure. Ignores transient (aka “infra”) failures. Default is False.on_new_status or on_occurrence instead. If True, notify on a build failure unless the previous build was a failure too. Ignores transient (aka “infra”) failures. Default is False.on_new_status or on_occurrence instead. If True, notify on each change to a build status (e.g. a green build becoming red and vice versa). Default is False.on_new_status or on_occurrence instead. If True, notify on each build success. Default is False.on_new_status.failed_step_regexp, but negated - this regex must not match any failed steps for a notification to be sent. Mutually exclusive with on_new_status.repo field in luci.builder(...). Default is False.https://host/repo) to restrict the blamelist calculation to. If empty (default), only the primary repository associated with the builder is considered, see repo field in luci.builder(...).default is defined in the project somewhere, it is used implicitly by the notifier.notifies field in luci.builder(...).luci.tree_closer( # Required arguments. name, tree_name, # Optional arguments. tree_status_host = None, failed_step_regexp = None, failed_step_regexp_exclude = None, template = None, notified_by = None, )
Defines a rule for closing or opening a tree via a tree status app.
The set of builders that are being observed is defined through notified_by field here or notifies field in luci.builder(...). Whenever a build finishes, the builder “notifies” all (but usually none or just one) luci.tree_closer(...) objects subscribed to it, so they can decide whether to close or open the tree in reaction to the new builder state.
Note that luci.notifier(...) and luci.tree_closer(...) are both flavors of a luci.notifiable object, i.e. both are something that “can be notified” when a build finishes. They both are valid targets for notifies field in luci.builder(...). For that reason they share the same namespace, i.e. it is not allowed to have a luci.notifier(...) and a luci.tree_closer(...) with the same name.
tree_status_name in luci.cq_group(...). Required.tree_status_host in luci.cq_group(...).failed_step_regexp, in which case it must also have a failed step matching that regular expression.default_tree_status is defined in the project somewhere, it is used implicitly by the tree closer.notifies field in luci.builder(...).luci.notifier_template(name, body)
Defines a template to use for notifications from LUCI.
Such template can be referenced by luci.notifier(...) and luci.tree_closer(...) rules.
The main template body should have format <subject>\n\n<body> where subject is one line of text/template and body is an html/template. The body can either be specified inline right in the starlark script or loaded from an external file via io.read_file(...).
The input to both templates is a TemplateInput Go struct derived from TemplateInput proto message.
The following functions are available to templates in addition to the standard ones.
A {{.Build.Builder.Builder}} build completed <a href="https://ci.chromium.org/b/{{.Build.Id}}">Build {{.Build.Number}}</a> has completed with status {{.Build.Status}} on `{{.Build.EndTime | time}}`
A template can “import” subtemplates defined in all other luci.notifier_template(...). When rendering, all templates defined in the project are merged into one. Example:
# The actual email template which uses subtemplates defined below. In the # real life it might be better to load such large template from an external # file using io.read_file. luci.notifier_template( name = 'default', body = '\n'.join([ 'A {{.Build.Builder.Builder}} completed', '', 'A <a href="https://ci.chromium.org/b/{{.Build.Id}}">build</a> has completed.', '', 'Steps: {{template "steps" .}}', '', '{{template "footer"}}', ]), ) # This template renders only steps. It is "executed" by other templates. luci.notifier_template( name = 'steps', body = '{{range $step := .Build.Steps}}<li>{{$step.name}}</li>{{end}', ) # This template defines subtemplates used by other templates. luci.notifier_template( name = 'common', body = '{{define "footer"}}Have a nice day!{{end}}', )
preview_email command can render a template file to stdout.
bb get -json -A 8914184822697034512 | preview_email ./default.template
This example uses bb tool, available in depot_tools.
Command preview_email is available in infra Go env and as a CIPD package.
If a user-defined template fails to render, a built-in template is used to generate a very short email with a link to the build and details about the failure.
^[a-z][a-z0-9\_]*$. Required.luci.cq( # Optional arguments. submit_max_burst = None, submit_burst_delay = None, draining_start_time = None, status_host = None, honor_gerrit_linked_accounts = None, )
Defines optional configuration of the CQ service for this project.
CQ is a service that monitors Gerrit CLs in a configured set of Gerrit projects, and launches tryjobs (which run pre-submit tests etc.) whenever a CL is marked as ready for CQ, and submits the CL if it passes all checks.
NOTE: before adding a new luci.cq(...), visit and follow instructions at http://go/luci/cv/gerrit-pubsub to ensure that pub/sub integration is enabled for all the Gerrit projects.
This optional rule can be used to set global CQ parameters that apply to all luci.cq_group(...) defined in the project.
submit_burst_delay. This feature today applies to all attempts processed by CQ, across all luci.cq_group(...) instances. Optional, by default there's no limit. If used, requires submit_burst_delay to be set too.submit_max_burst is used.2017-12-23T15:47:58Z and Z is mandatory.luci.cq_group( # Required arguments. watch, # Optional arguments. name = None, acls = None, allow_submit_with_open_deps = None, allow_owner_if_submittable = None, trust_dry_runner_deps = None, allow_non_owner_dry_runner = None, tree_status_host = None, tree_status_name = None, retry_config = None, cancel_stale_tryjobs = None, verifiers = None, additional_modes = None, user_limits = None, user_limit_default = None, post_actions = None, tryjob_experiments = None, )
Defines a set of refs to watch and a set of verifier to run.
The CQ will run given verifiers whenever there's a pending approved CL for a ref in the watched set.
Pro-tip: a command line tool exists to validate a locally generated .cfg file and verify that it matches arbitrary given CLs as expected. See https://chromium.googlesource.com/infra/luci/luci-go/+/refs/heads/main/cv/#luci-cv-command-line-utils
NOTE: if you are configuring a luci.cq_group for a new Gerrit host, follow instructions at http://go/luci/cv/gerrit-pubsub to ensure that pub/sub integration is enabled for the Gerrit host.
^[a-zA-Z][a-zA-Z0-9_-]*$.acl.CQ_* roles are allowed here. By default ACLs are inherited from luci.project(...) definition. At least one acl.CQ_COMMITTER entry should be provided somewhere (either here or in luci.project(...)).Code-Review and other approvals regardless of acl.CQ_COMMITTER or acl.CQ_DRY_RUNNER roles. Only cq.ACTION_* are allowed here. Default is cq.ACTION_NONE which grants no additional permissions. CL owner is user owning a CL, i.e. its first patchset uploader, not to be confused with OWNERS files. WARNING: using this option is not recommended if you have sticky Code-Review label because this allows a malicious developer to upload a good looking patchset at first, get code review approval, and then upload a bad patchset and CQ it right away.acl.CQ_DRY_RUNNER role as trusted, even if they are not approved. By default, unapproved dependencies are only trusted if they are owned by members of the acl.CQ_COMMITER role. This allows CQ dry run on CLs with unapproved dependencies owned by members of acl.CQ_DRY_RUNNER role.acl.CQ_DRY_RUNNER role to trigger DRY_RUN CQ on CLs that are owned by someone else, if all the CL dependencies are trusted.cq.RETRY_* constants that define how CQ should retry failed builds. See CQ for more info. Default is cq.RETRY_TRANSIENT_FAILURES.luci.cq_tryjob_verifier(**entry) and a string is an alias for luci.cq_tryjob_verifier(builder = entry).user_limit_default will be applied instead. Each cq.user_limit(...) must specify at least one user or group.user_limits are applicable and user_limit_default is not specified, the user is granted unlimited runs. user_limit_default must not specify users and groups.luci.cq_tryjob_verifier( # Required arguments. builder, # Optional arguments. cq_group = None, result_visibility = None, cancel_stale = None, includable_only = None, disable_reuse = None, experiment_percentage = None, location_filters = None, owner_whitelist = None, equivalent_builder = None, equivalent_builder_percentage = None, equivalent_builder_whitelist = None, mode_allowlist = None, )
A verifier in a luci.cq_group(...) that triggers tryjobs to verify CLs.
When processing a CL, the CQ examines a list of registered verifiers and launches new corresponding builds (called “tryjobs”) if it decides this is necessary (per the configuration of the verifier and the previous history of this CL).
The CQ automatically retries failed tryjobs (per configured retry_config in luci.cq_group(...)) and only allows CL to land if each builder has succeeded in the latest retry. If a given tryjob result is too old (>1 day) it is ignored.
The CQ can examine a set of files touched by the CL and decide to skip this verifier. Touching a file means either adding, modifying or removing it.
This is controlled by the location_filters field.
location_filters is a list of filters, each of which includes regular expressions for matching Gerrit host, project, and path. The Gerrit host, Gerrit project and file path for each file in each CL are matched against the filters; The last filter that matches all paterns determines whether the file is considered included (not skipped) or excluded (skipped); if the last matching LocationFilter has exclude set to true, then the builder is skipped. If none of the LocationFilters match, then the file is considered included if the first rule is an exclude rule; else the file is excluded.
The comparison is a full match. The pattern is implicitly anchored with ^ and $, so there is no need add them. The pattern must use Google Re2 library syntax, documented here.
This filtering currently cannot be used in any of the following cases:
allow_submit_with_open_deps = True.Please talk to CQ owners if these restrictions are limiting you.
Enable the verifier only for all CLs touching any file in third_party/blink directory of the main branch of chromium/src repo.
luci.cq_tryjob_verifier(
location_filters = [
cq.location_filter(
gerrit_host_regexp = 'chromium-review.googlesource.com',
gerrit_project_regexp = 'chromium/src'
gerrit_ref_regexp = 'refs/heads/main'
path_regexp = 'third_party/blink/.+')
],
)
Enable the verifier for CLs that touch files in “foo/”, on any host and repo.
luci.cq_tryjob_verifier(
location_filters = [
cq.location_filter(path_regexp = 'foo/.+')
],
)
Disable the verifier for CLs that only touches the “all/one.txt” file in “repo” of “example.com”. If the CL touches anything else in the same host and repo, or touches any file in a different repo and/or host, the verifier will be enabled.
luci.cq_tryjob_verifier(
location_filters = [
cq.location_filter(
gerrit_host_regexp = 'example.com',
gerrit_project_regexp = 'repo',
path_regexp = 'all/one.txt',
exclude = True),
],
)
Match a CL which touches at least one file other than one.txt inside all/ directory of the Gerrit project repo:
luci.cq_tryjob_verifier(
location_filters = [
cq.location_filter(
gerrit_host_regexp = 'example.com',
gerrit_project_regexp = 'repo',
path_regexp = 'all/.+'),
cq.location_filter(
gerrit_host_regexp = 'example.com',
gerrit_project_regexp = 'repo',
path_regexp = 'all/one.txt',
exclude = True),
],
)
For builders which may be useful only for some CLs, predeclare them using includable_only=True flag. Such builders will be triggered by CQ if and only if a CL opts in via CQ-Include-Trybots: <builder> in its description.
For example, default verifiers may include only fast builders which skip low level assertions, but for coverage of such assertions one may add slower “debug” level builders into which CL authors opt-in as needed:
# triggered & required for all CLs. luci.cq_tryjob_verifier(builder="win") # triggered & required if only if CL opts in via # `CQ-Include-Trybots: project/try/win-debug`. luci.cq_tryjob_verifier(builder="win-debug", includable_only=True)
cq_tryjob_verifier is used inline in luci.cq_group(...) declarations to provide per-builder verifier parameters. cq_group argument can be omitted in this case:
luci.cq_group(
name = 'Main CQ',
...
verifiers = [
luci.cq_tryjob_verifier(
builder = 'Presubmit',
disable_reuse = True,
),
...
],
)
It can also be associated with a luci.cq_group(...) outside of luci.cq_group(...) declaration. This is in particular useful in functions. For example:
luci.cq_group(name = 'Main CQ')
def try_builder(name, ...):
luci.builder(name = name, ...)
luci.cq_tryjob_verifier(builder = name, cq_group = 'Main CQ')
cq_tryjob_verifier can be used to declare a Tricium analyzer by providing the builder and mode_allowlist=[cq.MODE_ANALYZER_RUN]. It will generate the Tricium config as well as CQ config, so that no additional changes should be required as Tricium is merged into CV.
However, the following restrictions apply until CV takes on Tricium:
location_filters and owner_whitelist. If provided, they must meet the following conditions:location_filters must specify either both host_regexp and project_regexp or neither. For path_regexp, it must match file extension only (e.g. .+\.py) or everything. Note that, the exact same set of Gerrit repos should be specified across all analyzers in this cq_group and across each unique file extension.owner_whitelist must be the same for all analyzers declared in this cq_group.For example:
luci.project(tricium="tricium-prod.appspot.com")
luci.cq_group(
name = 'Main CQ',
...
verifiers = [
luci.cq_tryjob_verifier(
builder = "spell-checker",
owner_whitelist = ["project-committer"],
mode_allowlist = [cq.MODE_ANALYZER_RUN],
),
luci.cq_tryjob_verifier(
builder = "go-linter",
location_filters = [cq.location_filter(path_regexp = ".+\.go")]
owner_whitelist = ["project-committer"],
mode_allowlist = [cq.MODE_ANALYZER_RUN],
),
luci.cq_tryjob_verifier(builder = "Presubmit"),
...
],
)
Note for migrating to lucicfg for LUCI Projects whose sole purpose is to host a single Tricium config today (Example):
Due to the restrictions mentioned above, it is not possible to merge those auxiliary Projects back to the main LUCI Project. It will be unblocked after Tricium is folded into CV. To migrate, users can declare new luci.cq_group(...)s in those Projects to host Tricium analyzers. However, CQ config should not be generated because the config groups will overlap with the config group in the main LUCI Project (i.e. watch same refs) and break CQ. This can be done by asking lucicfg to track only Tricium config: lucicfg.config(tracked_files=["tricium-prod.cfg"]).
cq_tryjob_verifier is used inline inside some luci.cq_group(...) declaration.cq.COMMENT_LEVEL_FULL and cq.COMMENT_LEVEL_RESTRICTED constants. Default is to give full visibility: builder name and full summary markdown are included in the Gerrit comment.CQ-Include-Trybots: on CL description. Default is False. See the explanation above for all details. For builders with experiment_percentage or location_filters, don't specify includable_only. Such builders can already be forcefully added via CQ-Include-Trybots: in the CL description.equivalent_builder instead of builder. A choice itself is made deterministically based on CL alone, hereby all CQ attempts on all patchsets of a given CL will trigger the same builder, assuming CQ config doesn't change in the mean time. Note that if equivalent_builder_whitelist is also specified, the choice over which of the two builders to trigger will be made only for CLs owned by the accounts in the whitelisted group. Defaults to 0, meaning the equivalent builder is never triggered by the CQ, but an existing build can be re-used.cq.MODE_DRY_RUN and cq.MODE_FULL_RUN, and cq.MODE_NEW_PATCHSET_RUN out of the box. Additional Run modes can be defined via luci.cq_group(additional_modes=...).luci.bucket_constraints(bucket = None, pools = None, service_accounts = None)
Adds constraints to a bucket.
service_accounts added as the bucket's constraints will also be granted role/buildbucket.builderServiceAccount role.
Used inline in luci.bucket(...) declarations to provide pools and service_accounts constraints for a bucket. bucket argument can be omitted in this case:
luci.bucket(
name = 'try.shadow',
shadows ='try',
...
constraints = luci.bucket_constraints(
pools = ['luci.project.shadow'],
service_accounts = [`shadow@chops-service-account.com`],
),
)
luci.builder function implicitly populates the constraints to the builder’s bucket. I.e.
luci.builder(
'builder',
bucket = 'ci',
service_account = 'ci-sa@service-account.com',
)
adds ‘ci-sa@service-account.com’ to bucket ci’s constraints.
Can also be used to add constraints to a bucket outside of the bucket declaration. For example:
luci.bucket(name = 'ci') luci.bucket(name = 'ci.shadow', shadows = 'ci') luci.bucket_constraints(bucket = 'ci.shadow', pools = [shadow_pool])
luci.buildbucket_notification_topic(name, compression = None)
Define a buildbucket notification topic.
Buildbucket will publish build notifications (using the luci project scoped service account) to this topic every time build status changes. For details, see BuildbucketCfg.builds_notification_topics
luci.task_backend(name, target, config = None)
Specifies how Buildbucket should integrate with TaskBackend.
luci.dynamic_builder_template( # Required arguments. bucket, # Optional arguments. executable = None, properties = None, allowed_property_overrides = None, service_account = None, caches = None, execution_timeout = None, grace_period = None, heartbeat_timeout = None, dimensions = None, priority = None, expiration_timeout = None, retriable = None, experiments = None, resultdb_settings = None, test_presentation = None, backend = None, backend_alt = None, contact_team_email = None, custom_metrics = None, )
Defines a dynamic builder template for a dynamic bucket.
properties parameter here are allowed.execution_timeout or after a Cancel event, before the build is forcefully shut down. Your build can use this time as a ‘last gasp’ to do quick actions like killing child processes, cleaning resources, etc.INFRA_FAILURE. If None, Buildbucket won‘t check the heartbeat timeout. This field only takes effect for builds that don’t have Buildbucket managing their underlying backend tasks, namely the ones on TaskBackendLite. E.g. builds running on Swarming don't need to set this.os), and values are either strings (e.g. Linux), swarming.dimension(...) objects (for defining expiring dimensions) or lists of thereof.dimensions) before canceling the build and marking it as expired. If None, defer the decision to Buildbucket service.See permissions.cfg (sorry, internal only) for permissions and roles in LUCI Realms.
Below is the table with role constants that can be passed as roles in acl.entry(...).
Due to some inconsistencies in how LUCI service are currently implemented, some roles can be assigned only in luci.project(...) rule, but some also in individual luci.bucket(...) or luci.cq_group(...) rules.
Similarly some roles can be assigned to individual users, other only to groups.
| Role | Scope | Principals | Allows |
|---|---|---|---|
| acl.PROJECT_CONFIGS_READER | project only | groups, users | Reading contents of project configs through LUCI Config API/UI. |
| acl.LOGDOG_READER | project only | groups | Reading logs under project's logdog prefix. |
| acl.LOGDOG_WRITER | project only | groups | Writing logs under project's logdog prefix. |
| acl.BUILDBUCKET_READER | project, bucket | groups, users | Fetching info about a build, searching for builds in a bucket. |
| acl.BUILDBUCKET_TRIGGERER | project, bucket | groups, users | Same as BUILDBUCKET_READER + scheduling and canceling builds. |
| acl.BUILDBUCKET_OWNER | project, bucket | groups, users | Full access to the bucket (should be used rarely). |
| acl.SCHEDULER_READER | project, bucket | groups, users | Viewing Scheduler jobs, invocations and their debug logs. |
| acl.SCHEDULER_TRIGGERER | project, bucket | groups, users | Same as SCHEDULER_READER + ability to trigger jobs. |
| acl.SCHEDULER_OWNER | project, bucket | groups, users | Full access to Scheduler jobs, including ability to abort them. |
| acl.CQ_COMMITTER | project, cq_group | groups | Committing approved CLs via CQ. |
| acl.CQ_DRY_RUNNER | project, cq_group | groups | Executing presubmit tests for CLs via CQ. |
| acl.CQ_NEW_PATCHSET_RUN_TRIGGERER | project, cq_group | groups | Having CV automatically run certain tryjobs (e.g. static analyzers) when |
| a member uploads a new patchset to a CL monitored by CV and the feature | |||
| is enabled. |
acl.entry( # Required arguments. roles, # Optional arguments. groups = None, users = None, projects = None, )
Returns a new ACL binding.
It assign the given role (or roles) to given individuals, groups or LUCI projects.
Lists of acl.entry structs are passed to acls fields of luci.project(...) and luci.bucket(...) rules.
An empty ACL binding is allowed. It is ignored everywhere. Useful for things like:
luci.project( acls = [ acl.entry(acl.PROJECT_CONFIGS_READER, groups = [ # TODO: members will be added later ]) ] )
acl.entry object, should be treated as opaque.
resultdb.settings(enable = None, bq_exports = None, history_options = None)
Specifies how Buildbucket should integrate with ResultDB.
A populated buildbucket_pb.BuilderConfig.ResultDB() proto.
resultdb.export_test_results(bq_table = None, predicate = None)
Defines a mapping between a test results and a BigQuery table for them.
(project, dataset, table); OR a string of the form <project>.<dataset>.<table> where the parts represent the BigQuery-enabled gcp project, dataset and table to export results.A populated resultdb_pb.BigQueryExport() proto.
resultdb.test_result_predicate( # Optional arguments. test_id_regexp = None, variant = None, variant_contains = None, unexpected_only = None, )
Represents a predicate of test results.
{"test_suite": "not_site_per_process_webkit_layout_tests"}A populated predicate_pb.TestResultPredicate() proto.
resultdb.validate_settings(attr, settings = None)
Validates the type of a ResultDB settings proto.
A validated proto, if it's the correct type.
resultdb.history_options(by_timestamp = None)
Defines a history indexing configuration.
A populated resultdb_pb.HistoryOptions() proto.
resultdb.export_text_artifacts(bq_table = None, predicate = None)
Defines a mapping between text artifacts and a BigQuery table for them.
<project>.<dataset>.<table> where the parts respresent the BigQuery-enabled gcp project, dataset and table to export results.A populated resultdb_pb.BigQueryExport() proto.
resultdb.artifact_predicate( # Optional arguments. test_result_predicate = None, included_invocations = None, test_results = None, content_type_regexp = None, artifact_id_regexp = None, )
Represents a predicate of text artifacts.
A populated predicate_pb.ArtifactPredicate() proto.
resultdb.test_presentation(column_keys = None, grouping_keys = None)
Specifies how test should be rendered.
test_presentation.config struct with fields column_keys and grouping_keys.
resultdb.validate_test_presentation(attr, config = None, required = None)
Validates a test presentation config.
A validated test_presentation.config.
swarming.cache(path, name = None, wait_for_warm_cache = None)
Represents a request for the bot to mount a named cache to a path.
Each bot has a LRU of named caches: think of them as local named directories in some protected place that survive between builds.
A build can request one or more such caches to be mounted (in read/write mode) at the requested path relative to some known root. In recipes-based builds, the path is relative to api.paths['cache'] dir.
If it's the first time a cache is mounted on this particular bot, it will appear as an empty directory. Otherwise it will contain whatever was left there by the previous build that mounted exact same named cache on this bot, even if that build is completely irrelevant to the current build and just happened to use the same named cache (sometimes this is useful to share state between different builders).
At the end of the build the cache directory is unmounted. If at that time the bot is running out of space, caches (in their entirety, the named cache directory and all files inside) are evicted in LRU manner until there's enough free disk space left. Renaming a cache is equivalent to clearing it from the builder perspective. The files will still be there, but eventually will be purged by GC.
Additionally, Buildbucket always implicitly requests to mount a special builder cache to ‘builder’ path:
swarming.cache('builder', name=some_hash('<project>/<bucket>/<builder>'))
This means that any LUCI builder has a “personal disk space” on the bot. Builder cache is often a good start before customizing caching. In recipes, it is available at api.path['cache'].join('builder').
In order to share the builder cache directory among multiple builders, some explicitly named cache can be mounted to builder path on these builders. Buildbucket will not try to override it with its auto-generated builder cache.
For example, if builders A and B both declare they use named cache swarming.cache('builder', name='my_shared_cache'), and an A build ran on a bot and left some files in the builder cache, then when a B build runs on the same bot, the same files will be available in its builder cache.
If the pool of swarming bots is shared among multiple LUCI projects and projects mount same named cache, the cache will be shared across projects. To avoid affecting and being affected by other projects, prefix the cache name with something project-specific, e.g. v8-.
api.path['cache']). Must use POSIX format (forward slashes). In most cases, it does not need slashes at all. Must be unique in the given builder definition (cannot mount multiple caches to the same path). Required.path itself. Must be unique in the given builder definition (cannot mount the same cache to multiple paths).swarming.cache struct with fields path, name and wait_for_warm_cache.
swarming.dimension(value, expiration = None)
A value of some Swarming dimension, annotated with its expiration time.
Intended to be used as a value in dimensions dict of luci.builder(...) when using dimensions that expire:
luci.builder( ... dimensions = { ... 'device': swarming.dimension('preferred', expiration=5*time.minute), ... }, ... )
swarming.dimension struct with fields value and expiration.
swarming.validate_caches(attr, caches)
Validates a list of caches.
Ensures each entry is swarming.cache struct, and no two entries use same name or path.
Validates list of caches (may be an empty list, never None).
swarming.validate_dimensions(attr, dimensions, allow_none = None)
Validates and normalizes a dict with dimensions.
The dict should have string keys and values are swarming.dimension, a string or a list of thereof (for repeated dimensions).
{string: string|swarming.dimension}. Required.Validated and normalized dict in form {string: [swarming.dimension]}.
swarming.validate_tags(attr, tags)
Validates a list of k:v pairs with Swarming tags.
Validated list of tags in same order, with duplicates removed.
scheduler.policy( # Required arguments. kind, # Optional arguments. max_concurrent_invocations = None, max_batch_size = None, log_base = None, pending_timeout = None, )
Policy for how LUCI Scheduler should handle incoming triggering requests.
This policy defines when and how LUCI Scheduler should launch new builds in response to triggering requests from luci.gitiles_poller(...) or from EmitTriggers RPC call.
The following batching strategies are supported:
scheduler.GREEDY_BATCHING_KIND: use a greedy batching function that takes all pending triggering requests (up to max_batch_size limit) and collapses them into one new build. It doesn't wait for a full batch, nor tries to batch evenly.scheduler.LOGARITHMIC_BATCHING_KIND: use a logarithmic batching function that takes floor(log(base,N)) pending triggers (at least 1 and up to max_batch_size limit) and collapses them into one new build, where N is the total number of pending triggers. The base of the logarithm is defined by log_base.scheduler.NEWEST_FIRST: use a function that prioritizes the most recent pending triggering requests. Triggers stay pending until they either become the most recent pending triggering request or expire. The timeout for pending triggers is specified by pending_timeout.*_BATCHING_KIND values above. Required.LOGARITHMIC_BATCHING_KIND, ignored otherwise. Must be larger or equal to 1.0001 for numerical stability reasons.An opaque triggering policy object.
scheduler.greedy_batching(max_concurrent_invocations = None, max_batch_size = None)
Shortcut for scheduler.policy(scheduler.GREEDY_BATCHING_KIND, ...).
See scheduler.policy(...) for all details.
scheduler.logarithmic_batching(log_base, max_concurrent_invocations = None, max_batch_size = None)
Shortcut for scheduler.policy(scheduler.LOGARITHMIC_BATCHING_KIND, ...).
See scheduler.policy(...) for all details.
scheduler.newest_first(max_concurrent_invocations = None, pending_timeout = None)
Shortcut for scheduler.policy(scheduler.NEWEST_FIRST_KIND, ...).
See scheduler.policy(...) for all details.
CQ module exposes structs and enums useful when defining luci.cq_group(...) entities.
cq.ACTION_* constants define possible values for allow_owner_if_submittable field of luci.cq_group(...):
CQ_COMMITTER or CQ_DRY_RUNNER (if any).CQ_DRY_RUNNER role.CQ_COMMITTER role.cq.RETRY_* constants define some commonly used values for retry_config field of luci.cq_group(...):
cq.COMMENT_LEVEL_* constants define possible values for result_visibility field of luci.cq_tryjob_verifier(...):
cq.MODE_* constants define common values for cq run modes.
cq.STATUS_* constants define possible values for cq run statuses.
cq.post_action._* functions construct a post action that performs an action on a Run completion. They are passed to cq_group() via param post_actions. For exmaple, the following param constructs a post action that votes labels when a dry-run completes successfully in the cq group.
luci.cq_group( name = "main", post_actions = [ cq.post_action_gerrit_label_votes( name = "dry-run-verification", labels = {"dry-run-succeeded": 1}, conditions = [cq.post_action_triggering_condition( mode = cq.MODE_DRY_RUN, statuses = [cq.STATUS_SUCCEEDED], )], ), ], ... )
cq.refset(repo, refs = None, refs_exclude = None)
Defines a repository and a subset of its refs.
Used in watch field of luci.cq_group(...) to specify what refs the CQ should be monitoring.
https://. Only repositories hosted on *.googlesource.com are supported currently. Required.refs/heads/.+. If not set, defaults to refs/heads/main.An opaque struct to be passed to watch field of luci.cq_group(...).
cq.retry_config( # Optional arguments. single_quota = None, global_quota = None, failure_weight = None, transient_failure_weight = None, timeout_weight = None, )
Collection of parameters for deciding whether to retry a single build.
All parameters are integers, with default value of 0. The returned struct can be passed as retry_config field to luci.cq_group(...).
Some commonly used presents are available as cq.RETRY_* constants. See CQ for more info.
cq.retry_config struct.
cq.run_mode( # Required arguments. name, cq_label_value, triggering_label, triggering_value, )
Defines a CQ Run mode and how it can be triggered.
triggering_label that MUST be set to when triggering a CQ Run in this mode. Required.cq.run_mode struct.
cq.location_filter( # Optional arguments. gerrit_host_regexp = None, gerrit_project_regexp = None, gerrit_ref_regexp = None, path_regexp = None, exclude = None, )
Defines a location filter for the builder location_filters field.
All regexp fields can be empty, which is treated the same as “.*”, i.e. a wildcard which should match anything. Patterns are implicitly wrapped with “^...$”. They are allowed to contain these anchors, but it's redundant.
cq.location_filter struct.
cq.post_action_triggering_condition(mode, statuses = None)
Constructs cq.post_action_triggering_condition(...).
The condition is met if a Run in the mode terminates with one of the statuses.
cq.post_action_gerrit_label_votes(name, labels, conditions)
Constructs a post action that votes Gerrit labels.
cq.tryjob_experiment(name = None, owner_group_allowlist = None)
Constructs an experiment to enable on the Tryjobs.
The experiment will only be enabled if the owner of the CL is a member of any groups specified in owner_group_allowlist.
experiments field in Builder Configcq.user_limit( # Required arguments. name, # Optional arguments. users = None, groups = None, run = None, )
Construct a user_limit for run and tryjob limits.
At the time of Run creation, CV looks up a user_limit applicable for the Run, and blocks processing the Run or the tryjobs, if the number of ongoing runs and tryjobs reached the limits.
This constructs and return a user_limit, which specifies run and tryjob limits for given users and members of given groups. Find cq_group(...) to find how user_limit(s) are used in cq_group(...).
cq.run_limits(max_active = None, reach_limit_msg = None)
Constructs Run limits.
All limit values must be > 0, or None if no limit.
buildbucket.custom_metric(name, predicates, extra_fields = None)
Defines a custom metric for builds or builders.
NOTE: Before adding a custom metric to your project, you must first register it in buildbucket's service config specifying the metric's name, the standard build metric it bases on and metric extra_fields, if you plan to add any.
Then when adding the metric to your project, you must specify predicates.
If the metric is to report events of all builds under the builder, use the standard build metrics instead.
If the metric is to report events of all builds under the builder, but with extra metric fields, copied from a build metadata, such as tags, then add a predicate for the metadata. e.g., 'build.tags.exists(t, t.key=="os")'.
Elements in the predicates are concatenated with AND during evaluation, meaning the builds must satisfy all predicates to reported to this custom metrics. But you could use “||” inside an element to accept a build that satisfys a portion of that predicate.
Each element must be a boolean expression formatted in https://github.com/google/cel-spec. Current supported use cases are:
'has(build.tags)''has(build.input.properties.out_key)''string(build.output.properties.out_key) == "out_val"''build.status.to_string()=="INFRA_FAILURE"''build.input.experiments.exists(e, e=="luci.buildbucket.exp")''build.steps.exists(s, s.name=="compile")''build.tags.exists(t, t.key=="os")''build.tags.get_value("os")=="Linux"'If you specified extra_fields in Buildbucket's service config for the metric, you must also specify them here. For extra_fields:
'build.status.to_string()''build.experiments.to_string()''string(build.input.properties.out_key)''build.tags.get_value("branch")''"m121"'Additional metric extra_fields that are not listed in the registration in buildbucket‘s service config is allowed, but they will be ignored in metric reporting until being added to buildbucket’s service config.
NOTE: if you have a predicate or an extra_fields that cannot be supported by the CEL expressions we provided, please file a bug.
NOTE: You could test the predicates and extra_fields using the CustomMetricPreview RPC with one example build.
buildbucket.custom_metric struct with fields name, predicates and extra_fields.
buildbucket.validate_custom_metrics(attr, custom_metrics = None)
Validates a list of custom metrics.
Ensure each entry is buildbucket.custom_metric struct.
Validates list of custom metrics (may be an empty list, never None).
Refer to the list of built-in constants and functions exposed in the global namespace by Starlark itself.
In addition, lucicfg exposes the following functions.
__load(module, *args, **kwargs)
Loads a Starlark module as a library (if it hasn't been loaded before).
Extracts one or more values from it, and binds them to names in the current module.
A load statement requires at least two “arguments”. The first must be a literal string, it identifies the module to load. The remaining arguments are a mixture of literal strings, such as 'x', or named literal strings, such as y='x'.
The literal string ('x'), which must denote a valid identifier not starting with _, specifies the name to extract from the loaded module. In effect, names starting with _ are not exported. The name (y) specifies the local name. If no name is given, the local name matches the quoted name.
load('//module.star', 'x', 'y', 'z') # assigns x, y, and z
load('//module.star', 'x', y2='y', 'z') # assigns x, y2, and z
A load statement within a function is a static error.
See also Modules and packages for how load(...) interacts with exec(...).
//path/within/current/package.star or @<pkg>//path/within/pkg.star or ./relative/path.star. Required.exec(module)
Executes another Starlark module for its side effects.
See also Modules and packages for how load(...) interacts with exec(...).
//path/within/current/package.star or @<pkg>//path/within/pkg.star or ./relative/path.star. Required.A struct with all exported symbols of the executed module.
fail(msg, trace = None)
Aborts the execution with an error message.
fail is called.stacktrace(skip = None)
Captures and returns a stack trace of the caller.
A captured stacktrace is an opaque object that can be stringified to get a nice looking trace (e.g. for error messages).
struct(**kwargs)
Returns an immutable struct object with given fields.
Can be used to define namespaces, for example:
def _func1(): ... def _func2(): ... exported = struct( func1 = _func1, func2 = _func2, )
Then _func1 can be called as exported.func1().
to_json(value)
Serializes a value to a compact JSON string.
Doesn't support integers that do not fit int64. Fails if the value has cycles.
json.encode(value)
Encodes a value into a JSON string.
Accepts one required positional argument, which it converts to JSON by cases:
Encoding any other value yields an error.
json.decode(str)
Decodes a JSON string.
Accepts one positional parameter, a JSON string. It returns the Starlark value that the string denotes:
Decoding fails if str is not a valid JSON string.
json.indent(str, prefix = None, indent = None)
Pretty-prints a valid JSON encoding.
The indented form of str.
proto.to_textpb(msg)
Serializes a protobuf message to a string using ASCII proto serialization.
proto.to_jsonpb(msg, use_proto_names = None)
Serializes a protobuf message to a string using JSONPB serialization.
proto.to_wirepb(msg)
Serializes a protobuf message to a string using binary wire encoding.
proto.from_textpb(ctor, text, discard_unknown = None)
Deserializes a protobuf message given its ASCII proto serialization.
The returned message is frozen. Use proto.clone(...) to get a mutable copy if necessary.
Deserialized frozen message constructed via ctor.
proto.from_jsonpb(ctor, text, discard_unknown = None)
Deserializes a protobuf message given its JSONPB serialization.
The returned message is frozen. Use proto.clone(...) to get a mutable copy if necessary.
Deserialized frozen message constructed via ctor.
proto.from_wirepb(ctor, blob, discard_unknown = None)
Deserializes a protobuf message given its wire serialization.
The returned message is frozen. Use proto.clone(...) to get a mutable copy if necessary.
Deserialized frozen message constructed via ctor.
proto.struct_to_textpb(s = None)
Converts a struct to a text proto string.
A str containing a text format protocol buffer message.
proto.clone(msg)
Returns a deep copy of a given proto message.
A deep copy of the message.
proto.has(msg, field)
Checks if a proto message has the given optional field set.
Following rules apply:
*.proto file are always unset.int64), repeated and map fields (even empty ones) are always set. There's no way to distinguish zero values of such fields from unset fields.oneof field (regardless of their type) are initialized only when they are explicitly “picked”.True if the message has the field set.
io.read_file(path)
Reads a file and returns its contents as a string.
Useful for rules that accept large chunks of free form text. By using io.read_file such text can be kept in a separate file.
//) an absolute path within the currently executing package. If it is a relative path, it must point somewhere inside the current package directory. Required.The contents of the file as a string. Fails if there‘s no such file, it can’t be read, or it is outside of the current package directory.
io.read_proto(ctor, path, encoding = None)
Reads a serialized proto message from a file, deserializes and returns it.
//) an absolute path within the currently executing package. If it is a relative path, it must point somewhere inside the current package directory. Required.jsonpb or textpb or auto to detect based on the file extension. Default is auto.Deserialized proto message constructed via ctor.