Tast

Modifying Tast

This document describes how to make changes to the Tast framework itself. For information about writing Tast tests, see Writing Tests.

Components

See the Overview document for a high-level description of the key components and terminology in Tast.

Executables

  • The tast executable is run by developers and by builders. It's used to compile, deploy, and run tests and to collect results.
  • local_test_runner is run on the DUT by the tast process over an SSH connection. It collects system information and initiates running local tests.
  • remote_test_runner is run on the host system by the tast process to initiate running remote tests.

Libraries

Shared library packages are located under the tast directory. Several packages are particularly important:

  • bundle contains code used to implement test bundles, which contain compiled tests and are executed by test runners.
  • protocol defines protocol buffer messages that are used for communication between the tast process, test runners, and test bundles.
  • runner contains code shared between local_test_runner and remote_test_runner.
  • ssh opens SSH connections.
  • testing contains code used to define and run tests.

Making changes

IPC

JSON-marshaled structs are used for communication between processes:

The runner.Args and bundle.Args structs contain other Args-suffixed structs that may or may not be set depending on the type of request that is being made. Note also that runner.RunTestsArgs includes bundle.RunTestsArgs and that runner.ListTestsArgs includes bundle.ListTestsArgs.

When running in the ChromeOS hardware lab or on VM builders, matching versions of tast, test runners, and test bundles are used, so there are no compatibility concerns.

More combinations are possible when running in a developer's chroot:

  • The tast binary in the chroot can be updated by emerging tast-cmd or running fast_build.sh.
  • The remote_test_runner binary in the chroot is typically updated by emerging tast-cmd.
  • The local_test_runner binary on the DUT can either be installed by the tast-local-test-runner package or updated by running tast run -build=true.
  • The default test bundle on the DUT can be installed by the tast-local-tests-cros package or updated by running tast run -build=true.

As a result, developers may end up running the tast executable in their chroot against DUTs that contain either older or newer versions of local_test_runner and test bundles. To maintain compatibility, please ensure that the following conditions hold across both the tast-to-runner and runner-to-bundle boundaries:

  • Older receivers correctly handle structs marshaled by newer senders.
  • Newer receivers correctly handle structs marshaled by older senders.

Go‘s json package does best-effort unmarshaling, ignoring unknown fields by default. When renaming or moving a field, it’s advisable to preserve the old field for at least two months. Add a Deprecated suffix to the old field's name while preserving its original marshaled name in its json tag.

tast and the runner package should set both the old and new fields, and the runner and bundle packages should copy the old fields to the new fields when the former are provided. See change 1474620 for an example.

Compiling changes

fast_build.sh

The quickest way to rebuild the tast executable after modifying its code is by running the fast_build.sh script located at the top of the src/platform/tast repository within the ChromeOS chroot. This script bypasses Portage and runs go build directly, allowing it to take advantage of Go's build cache. Since dependency checks are skipped, there's no guarantee that the resulting executable is correct – before uploading a change, you should verify it that it builds when you run FEATURES=test sudo emerge tast-cmd (after running cros_workon --host start tast-cmd).

  • Without any arguments, fast_build.sh compiles the tast executable to $HOME/go/bin/tast.
  • fast_build.sh -t go.chromium.org/tast/core/testing runs the unit tests for the go.chromium.org/tast/core/testing package.
  • fast_build.sh -T runs all unit tests (including ones in the tast-tests repository).
  • fast_build.sh -c go.chromium.org/tast/core/testing runs go vet against the go.chromium.org/tast/core/testing package.
  • fast_build.sh -C vets all packages.

Run fast_build.sh -h to see all available options.

Testing changes

Unit tests

The different components of the framework are extensively covered by unit tests. Please ensure that any changes that you make are also covered by tests.

The testutil package provides utility functions intended to reduce repetitive code within unit tests.

Meta tests

There are several meta tests. These are remote Tast tests that run a nested instance of the tast executable to perform end-to-end verification of interactions between tast, test runners, and test bundles. They're executed the same way as other Tast tests, i.e. via tast run.