ANGLE runs hundreds of thousands of tests on every change before it lands in the tree. We scale our pre-commit and post-commit testing to many machines using Chromium Swarming. Our testing setup heavily leverages existing work in Chromium. We also run compile-only Standalone Testing that does not depend on a Chromium checkout.
Also see the documentation on ANGLE Wrangling for more info.
See the pre-commit try waterfall here:
We currently run pre-commit tests on:
Looking at an example build shows how tests are split up between machines. See for example:
This build ran 68 test steps across 3 GPU families. In some cases (e.g.
angle_deqp_gles3_metal_tests) the test is split up between multiple machines to run faster (in this case 2 different machines at once). This build took 23 minutes to complete 72 minutes of real automated testing.
For more details on running and working with our test sets see the docs in Contributing Code.
In addition to the ANGLE try bots using Chrome, and the GPU.FYI bots, ANGLE has standalone testing on the Chrome infrastructure. Currently these tests are compile-only. This page is for maintaining the configurations that don't use Chromium. Also see the main instructions for ANGLE Wrangling.
It‘s the ANGLE team’s responsibility for maintaining this testing infrastructure. The bot configurations live in four different repos and six branches.
Continuous builders for every ANGLE revision are found on the CI console:
Try jobs from pre-commit builds are found on the builders console:
configs/chromium-swarm/starlark/bots/angle.star with either Mac slaves requested in the previous step or increase the amount of Windows or Linux GCEs.
scripts/slave/recipes/angle.py with new the config.
./scripts/slave/recipes.py test train to update checked-in golden files. This might no longer be necessary.
infra/config/global/cr-buildbucket.cfg to add the new builder (to ci and try), and set the new config option.
infra/config/global/luci-milo.cfg to make the builders show up on the ci and try waterfalls.
infra/config/global/luci-scheduler.cfg to make the builders trigger on new commits or try jobs respectively.
infra/config/global/commit-queue.cfg to add the builder to the default CQ jobs (if desired).
There are other places where configuration for ANGLE infra lives. These are files that we shouldn't need to modify very often:
configs/luci-token-server/service_accounts.cfg (service account names)
configs/chromium-swarm/pools.cfg (swarming pools)
recipes/recipe_modules/gclient/config.py (gclient config)