commit	9324eed93ea33b55756186ef5797e54cd280a5c1	[log] [tgz]
author	Julio Hurtado <juahurta@google.com>	Thu Oct 29 23:30:21 2020
committer	Commit Bot <commit-bot@chromium.org>	Fri Oct 30 15:13:05 2020
tree	5e4a58e18d6f1fbd1652fd37c56f0b068badda60
parent	b8e93b6828b1f732d4ea8d2ebeade0c0090dab72 [diff]

tree: 5e4a58e18d6f1fbd1652fd37c56f0b068badda60

README.md

Chrome OS Recipes

This is a quick-start guide for Chrome OS Recipes. For more in-depth information about the Recipes framework, see the Recipes User Guide.

The recipes and recipe modules in this repo have documentation auto-generated by the ‘test train’ process: README.recipes.md

Getting Started

The recipes.py script in this repo will bootstrap itself. You can verify that it is working by running tests:

./recipes.py test run

After making a change, you will usually need to “re-train” expected output in order for tests to pass:

./recipes.py test train

This will also regenerate the README.recipes.md documentation. That file and the expectation data in *.expected directories should be checked in with your changes.

Support Tools

This repo contains some non-recipes support tooling in the support/ directory. These tools are delivered to CI hosts via CIPD. This process is automated by the support/deploy_cipd.sh script. See support/README.txt for more details.

Testing With `led`

Expectation tests only mock the external services a recipe depends on. This means the expectation tests cannot catch bugs caused by calling external services incorrectly. For example, if a recipe made the call

ls --bad_flag

it would not be caught by ./recipes.py test train. A more realistic example is the recipe makes incorrect assumptions about ACLs, paths, responses from the service, etc.

One way to get more confidence the recipe code will work is by running it with the actual builder definitions and swarming environment, using the led command-line tool. This will call the external services with the same user, timeouts, swarming caches, etc. as the submitted recipes.

As stated above, led means the code will actually run commands on swarming. This means the code being tested can impact production resources, e.g. clobber a Google Storage file, break Skylab DUTs, etc. Do NOT blindly test code with led, and use the staging environment where possible, which may limit the blast radius of bugs.

To use led, first call get-build or get-builder to get a job definition. For example, to test a change to the Annealing builder, we could start by running

led get-builder staging:staging-Annealing > builder_def.json

which will produce a JSON object containing properties, swarming information, etc. Sometimes, builders require properties that are not set in the lucicfg definition. For example, build_target requires the gitiles_commit property. In this case, we could get a definition from a previous build. For example, say the last build of staging-amd64-generic-postsubmit has id 12345. To get a job definition with the same properties (including gitiles_commit), run the command

led get-build 12345 > builder_def.json

Next, the led edit-recipe-bundle command patches your local recipes edits into the job definition, and led launch launches the job on swarming. The job definition is piped from each command to the next, so in practice an example command would be

led get-build 12345 | led edit-recipe-bundle | led launch

The output from led launch should give a link to the swarming task, e.g.

[I 2019-05-06 15:31:43] Launched swarming task: https://chrome-swarming.appspot.com/task?id=44a7e0cb7a0d6310
[I 2019-05-06 15:31:43] LUCI UI: https://ci.chromium.org/swarming/task/44a7e0cb7a0d6310?server=chrome-swarming.appspot.com

Test Recipes Presubmit

Recipe changes run through a presubmit job that verifies them on actual ChromeOS CI builders (essentially automating the led testing process).

The builders to be run are controlled with the builders property on the test_recipes recipe. Builders will not be run if the recipe change does not affect them (as determined by ./recipes.py analyze, which checks if the recipe or any of its dependencies has been modified).

If you are confident that your change does not need testing, you can include the Test-Recipes-Skip-Builder footer in your CL. That this should be done sparingly. If the presubmit is flaky, the root cause should be fixed. Please put a justification in the CL description for why the builder is skipped. The footer syntax is a bit picky; an example commit message would look like

A test commit

- Note: Skipping testers because this change must be submitted now.

Test-Recipes-Skip-Builder: Builder1
Test-Recipes-Skip-Builder: Builder2
Change-Id: 1234

Use depot_tools/git-footers to verify the footer syntax. For example, the above commit returns

Test-Recipes-Skip-Builder: Builder2
Test-Recipes-Skip-Builder: Builder1
Change-Id: 1234

Note that adding a space between footers is not allowed. For example, the commit message

A test commit

- Note: Skipping testers because this change must be submitted now.

Test-Recipes-Skip-Builder: Builder1
Test-Recipes-Skip-Builder: Builder2

Change-Id: 1234

will return

Change-Id: 1234

and thus the builders will not be skipped. Also note that each builder to skip must have its own line; Test-Recipes-Skip-Builder: Builder1, Builder2 will not work.