Kokoro is a Google-internal continuous integration service, and is used extensively for building, signing, testing, and releasing container images and guest packages.
Kokoro configs are split into two pieces: jobs and builds. The jobs are held in crostini-guest, and the build configs are in the container-guest-tools repository under
kokoro. Build configs are reused across different jobs, e.g. the
apt_repo build is used in both the
presubmit job flows. However, the
presubmit job flow bypasses the
apt_signer build, and does not publish artifacts to Google Storage buckets.
Both the Termina VM and container-guest-tools .deb packages are branched to match Chrome OS milestones. So for the above
continuous flow, there are job instances for each active (stable, beta, dev, canary) branch. See the detailed configs in crostini-guest. As an example, with 71 on stable and 74 in canary:
|Kokoro job||Milestone||Chrome OS Branch|
The artifacts for each branch will be pushed to a subdirectory on the target Google Storage bucket. Prebuilt containers are pushed to
gs://cros-containers-staging/milestone, and apt repos are pushed to
Push from staging to the live buckets
gs://cros-packages based on the Tast test results.
presubmit flow only runs on the
master branch. Note that there are both a
presubmit-cr jobs - the former runs for
Trybot-Ready +1, and the latter on
Code-Review +2 on Gerrit.
Some tests are run directly in Kokoro by verifying that the guest container starts up as expected. These tests are in lxd/test.py.
Testing in a live environment on a DUT is done with Tast. Most of those tests, including the performance tests and
vm.CrostiniStartEverything, run using the staging version of the container. A staging version of the VM is also used, which is stored in
gs://termina-component-testing. When Stainless shows green test results, the Termina VM, staging container image, and staging apt packages can be rolled out to the production buckets.