Local tests run directly on devices-under-test (DUTs) and are similar in function to Autotest's “client” tests. They can do anything that any other code running locally as root can do. Most tests should be local tests.
Remote tests run on the machine initiating testing (e.g. a developer‘s workstation, a.k.a. the “host system”) and run shell commands on the DUT remotely, allowing the DUT to be rebooted in the course of a single test. They are similar in function to Autotest’s “server” tests. They run more slowly than local tests, are harder to write, and are more susceptible to infrastructure and hardware issues.
tast commandThe tast executable is built for the host system by the chromeos-base/tast-cmd Portage package from code in the tast repository. The developer runs it within their chroot to:
The Running Tests document contains more information about using the tast command.
Test bundles consist of executables and their associated data files (internal data files + external data link files). Test bundle executables are composed of compiled tests along with code for running them and reporting results back to a test runner. Bundles exist so that test code can be checked into other (probably non-public) repositories.
The default local and remote bundles, consisting of ChromeOS tests, are named cros and are checked into the tast-tests repository.
Local tests can be included in a system image via Portage packages with names of the form chromeos-base/tast-local-tests-<bundle>, while remote tests can be installed onto the host system via packages named chromeos-base/tast-remote-tests-<bundle>. Either type of bundle can also be compiled (and deployed, in the case of local tests) on-the-fly by tast.
Tests don‘t (currently) require anything special from the OS — tast just needs SSH access to the DUT and (when deploying local test bundles that aren’t built into the system image) a writable partition that isn't mounted noexec.
See the Writing Tests document for more information.
Test runners are executables that enumerate and execute test bundles before aggregating results for the tast command. There are two test runners, local_test_runner (which executes local test bundles on-device) and remote_test_runner (which runs remote test bundles on the host system). Runners are installed by the chromeos-base/tast-local-test-runner and chromeos-base/tast-cmd Portage packages, respectively, and are built from code in the tast repository.
Test bundles can be cross-compiled within the chroot by the tast executable using the Go toolchain. Object files are cached and reused where possible — caching happens at the package level, and the tests within each bundle are grouped into packages by category, e.g. ui, power, security, etc.
For local tests, tast establishes a single SSH connection to the DUT for all communication. The connection is used to copy a local test bundle and associated test data files to the DUT (if not built into the system image), execute local_test_runner, and receive the results (including system logs, crash dumps, and output data written by the tests).
For remote tests, tast executes remote_test_runner directly on the host system and receives results provided by it. Each remote test bundle establishes an SSH connection to the DUT and passes it to tests.
The Design Principles document describes the thinking that guided Tast's design.