Crostini developer guide

If you just want to use Linux (Beta), you should read the Running Custom Containers Under Chrome OS doc. This doc is all about how the crostini is made, not how to use it. :)

What is all this stuff?

Note: this diagram is not exhaustive, but covers the major services and how they interact.

Crostini services diagram

Googlers: update this image at go/termina-rpc

Where does the code live?

NameAffectsRepoebuild
9sHostplatform2/vm_tools/9sdev-rust/9s
chunnelHost, Terminaplatform2/vm_tools/chunnelchromeos-base/chunnel, chromeos-base/termina_container_tools
ciceroneHostplatform2/vm_tools/ciceronechromeos-base/vm_host_tools
conciergeHostplatform2/vm_tools/conciergechromeos-base/vm_host_tools
container .debs, Termina build scriptsContainerplatform/container-guest-toolsN/A
crostini_clientHostplatform2/vm_tools/crostini_clientchromeos-base/crostini_client
crosvmHostplatform/crosvmchromeos-base/crosvm
garconTermina, Containerplatform2/vm_tools/garconchromeos-base/vm_guest_tools, chromeos-base/termina_container_tools
LXDTerminagithub/lxc/lxdapp-emulation/lxd
maitredTerminaplatform2/vm_tools/maitredchromeos-base/vm_guest_tools
seneschalHostplatform2/vm_tools/seneschalchromeos-base/vm_host_tools
sommelierTermina, Containerplatform2/vm_tools/sommelierchromeos-base/sommelier, chromeos-base/termina_container_tools
system_apiHostplatform2/system_apichromeos-base/system_api
tremplinTerminaplatform/tremplinchromeos-base/tremplin
VM protobufsHost, Termina, Containerplatform2/vm_tools/protochromeos-base/vm_protos
vm_syslogHost, Terminaplatform2/vm_tools/syslogchromeos-base/vm_guest_tools, chromeos-base/vm_host_tools
vshHost, Termina, Containerplatform2/vm_tools/vshchromeos-base/vm_host_tools, chromeos-base/vm_guest_tools, chromeos-base/termina_container_tools

How do I build/deploy/test my change?

General prerequisites

  • Follow the Chromium OS Developer Guide for setup.
  • Device with test image in developer mode.
  • TODO: how to deal with cros_debug mismatch?
  • TODO: Emerging tremplin etc. and directly mount it in vm_rootfs.img on the dut.

Ensure you are able to SSH to the device:

(inside) $ export DEVICE_IP=123.45.67.765 # insert your test device IP here
(inside) $ ssh ${DEVICE_IP} echo OK

For the rest of this document, it will be assumed that the BOARD environment variable in cros_sdk is set to the board name of your test device as explained in the Select a board section of the Chromium OS Developer Guide.

Crostini requires a signed-in, non-guest user account to run. You can either use a test Google account, or run /usr/local/autotest/bin/autologin.py -d on a test image to log in with a fake testuser profile.

Host service changes

To begin working on a change to one of the host services (see Where does the code live?), use the cros_workon command:

(inside) $ cros_workon --board=${BOARD} start ${PACKAGE_NAME}

Now that the package(s) are cros_workon start-ed, they will be built from source instead of using binary prebuilts:

(inside) $ emerge-${BOARD} ${PACKAGE_NAME}

Then deploy the package to the device for testing:

(inside) $ cros deploy ${DEVICE_IP} ${PACKAGE_NAME}

Most VM services on the host have upstart conditions start on started vm_concierge and stop on stopped vm_concierge. Since restarting only one daemon could lead to inconsistent state across all services, it's best to shut them all down, then start them again with a clean slate. Stopping the vm_concierge service on the host stops all running VMs, along other VM services on the host. Starting vm_concierge will trigger other VM services to start as well.

(device) # stop vm_concierge && start vm_concierge

Guest service changes

The guest packages that run inside the termina VM are built for two special Chrome OS boards: tatl (for x86 devices) and tael (for arm devices). These VM images are distributed as part of the cros-termina component via the browser’s Component Updater.

To determine the guest board type, run uname -m on the device.

uname -mtermina board
x86_64(inside) $ export GUEST_BOARD=tatl
aarch64(inside) $ export GUEST_BOARD=tael

First, cros_workon --board=${GUEST_BOARD} start each guest package you are modifying (see Where does the code live?):

(inside) $ cros_workon --board=${GUEST_BOARD} start ${PACKAGE_NAME}

Then, to build and deploy the updated packages to a device for testing, run this command:

(inside) $ /mnt/host/source/src/platform/dev/contrib/deploy_termina -p -i -d ${DEVICE_IP}

The deploy_termina command above is recommended for new developers. It automatically determines the correct guest board for the device (tatl or tael) by connecting to the device specified using -d ${DEVICE_IP} via SSH. It then builds the full set of packages for the device (-p), builds the Chrome OS rootfs image for the guest (-i), repacks the guest image into the layout expected for the cros-termina component, copies it to the device, and restarts the vm_concierge and vm_cicerone services on the device (stopping any running VMs in the process).

WARNING: The /run/imageloader directory that holds the Termina image is a tmpfs, so changes are not persisted across reboots. You will need to either run the deploy_termina script again, or manually mount the Termina image into the /run/imageloader/cros-termina/99999.0.0 directory.

After deploy_termina completes, newly-launched VMs will use the testing component with the updated packages.

Container changes

Packages can end up in the container by two mechanisms:

  1. Native Debian packages (.debs) are preinstalled in the container, and upgraded out-of-band from the rest of Chrome OS by APT.

  2. Packages built from Portage in Chrome OS are copied into /opt/google/cros-containers in Termina by the termina_container_tools ebuild. These are updated with the Termina VM image.

When working on Debian packages, the .debs should be copied to the crostini container and installed with apt:

# A leading "./" or other unambiguous path is needed to install a local .deb.
(penguin) $ apt install ./foo.deb

Portage-managed packages should be treated like other [Guest service changes]. However, the termina_container_tools package is not cros_workon, so it must be manually emerged to propagate changes into /opt/google/cros-containers. The following example uses sommelier:

(inside) $ emerge-${GUEST_BOARD} sommelier               # build for Termina
(inside) $ emerge-${GUEST_BOARD} termina_container_tools # copy into /opt

Once termina_container_tools is manually rebuilt, the deploy_termina flow will work as normal.

Running VM executables off of a device

It's possible to run binaries built for the termina VM from the chromium OS chroot, which can be useful for debugging or testing. Assuming you already have a chromium OS chroot set up and have built the tatl board, you can run inside the chroot:

../platform2/common-mk/platform2_test.py --board tatl [--run_as_root] [command to run]

For example, to run LXD, you could run inside the chroot:

../platform2/common-mk/platform2_test.py --board tatl --run_as_root env LXD_DIR=/path/to/lxd/data lxd
../platform2/common-mk/platform2_test.py --board tatl --run_as_root env LXD_DIR=/path/to/lxd/data lxd waitready
../platform2/common-mk/platform2_test.py --board tatl --run_as_root env LXD_DIR=/path/to/lxd/data lxc [lxc subcommands go here]

This is not limited to x86_64 boards either, platform2_test.py will automatically set up QEMU to run ARM binaries if you ask it to run binaries from an ARM board. This is likely to be very slow, however.

Note that this is not equivalent to actually running the VM, since only the command you run will be performed. Depending on the exact set up you need to test, this may not be sufficient.