Buildbot Testing Configuration Files

The files in this directory control how tests are run on the Chromium buildbots. In addition to specifying what tests run on which builders, they also specify special arguments and constraints for the tests.

A tour of the directory

• <master_name>.json -- buildbot configuration json files. These are used to configure what tests are run on what builders, in addition to specifying builder-specific arguments and parameters.
• gn_isolate_map.pyl -- maps Ninja build target names to GN labels. Allows for certain overrides to get certain tests targets to work with GN (and properly run when isolated).
• trybot_analyze_config.json -- used to provide exclusions to the analyze step on trybots.
• filters/ -- filters out tests that shouldn't be run in a particular mode.
• timeouts.py -- calculates acceptable timeouts for tests by analyzing their execution on swarming.
• manage.py -- makes sure the buildbot configuration json is in a standardized format.

How the files are consumed

Buildbot configuration json

Logic in the Chromium recipe looks up each builder for each master and test generators in chromium_tests/steps.py parse the data. For example, as of a6e11220 generate_gtest parses any entry in a builder's ‘gtest_tests’ entry.

How to edit

Making the changes

Buildbot configuration json

After editing any buildbot json, run ./manage.py -w to load and write in the canonical format. Then commit as normal.

Note that trybots mirror regular waterfall bots, with the mapping defined in trybots.py. This means that, as of 81fcc4bc, if you want to edit linux_android_rel_ng, you actually need to edit Android Tests.

Trying the changes on trybots

You should be able to try build changes that affect the trybots directly (for example, adding a test to linux_android_rel_ng should show up immediately in your tryjob). Non-trybot changes have to be landed manually :(.

Capacity considerations when editing the buildbot configuration json

When adding tests or bumping timeouts in the buildbot configuration json, care must be taken to ensure the infrastructure has capacity to handle the extra load. This is especially true for the established Chromium CQ builders, as they operate under strict execution requirements. Make sure to get an infrastructure engineer on the Crossover Team to sign off that there is both buildbot and swarming capacity available.