Chromium OS Unit Testing

Warning: This document is old & has moved. Please update any links:
https://chromium.googlesource.com/chromiumos/docs/+/HEAD/testing/running_unit_tests.md

Resources for writing unit tests

Running unit tests

We run unit tests as part of the build process using chromite's cros_run_unit_tests. In a nutshell, this script uses the portage method of building and running unit tests using FEATURES="test". To run all the Chromium OS unit tests, you can run cros_run_unit_tests. If you want to run just the unit tests for specific packages, you can use:

(chroot) $ cros_run_unit_tests \
           --packages <space-delimited list of portage package names>

See cros_run_unit_tests --help for more information.

As an example, let's say you just want to run the unit tests for the metrics package. To do so, you can run:

(chroot) $ cros_run_unit_tests --board ${BOARD} --packages "metrics"

Running a single test

For platform2 packages, see the platform2 testing section.

For other packages that use gtest:

(chroot) $ GTEST_ARGS="--gtest_filter=FreeDiskSpaceTest.*" cros_run_unit_tests \
           --board ${BOARD} --packages google-breakpad

Adding unit tests

For platform2 packages

For packages using platform2, see the metrics ebuild as a good example of adding your unit test. These steps include:

Modifying the package ebuild.
- Create a platform_pkg_test() stanza with appropriate gtest filters (see below).
Adding a *_test.cc test file (see example file).
- See gtest syntax and the usages from Google Test - Chromium's C++ test harness.

For non-platform2 packages

See example CL for a good example of adding new unit tests. These steps include:

Modifying the package ebuild.
- Create a src_test() stanza with appropriate gtest filters (see below), and add test? ( dev-cpp/gtest:= ) toDEPEND in your ebuild.
Adding a testrunner.
- Look at the canonical example.
Adding a *_test.cc test file (see example file).
- See gtest syntax and the usages from Google Test - Chromium's C++ test harness.

Regarding src_test() stanza, it is fine to have them build in the src_compile() stage as well. See also Portage documentation on src_test().

Non-native architectures (e.g. QEMU)

Platform2 supports running unittests via QEMU for non-native architectures (low level details can be found in this doc). The good news is that the process is the same as described above! Simply use cros_run_unit_tests for your board and specify the packages you want to run.

Caveats

Sometimes QEMU is not able to support your unittest due to using functionality not yet implemented. If you see errors like qemu: unknown syscall: xxx, you will need to filter that test (see below), and you should file a bug so we can update QEMU.

Since QEMU only works when run inside the chroot, only ebuilds that use the platform.eclass are supported currently.

Filtering tests

Sometimes a test is flaky or requires special consideration before it'll work. In general it is not recommended to disable flaky/broken tests in ebuilds. Instead add DISABLED_ to the prefix of the test name in the source, hence eliminating to touch ebuilds at all. Look at this example CL. Otherwise, see below for disabling tests in ebuilds.

For packages using platform.eclass use the existing gtest filtering logic by simply passing it as an argument to platform_test:

platform_pkg_test() {
    local gtest_filter=""
    gtest_filter+="DiskManagerTest.*"

    platform_test "run" "${OUT}/some_testrunner" "0" "${gtest_filter}"
}

For other packages, export the relevant gtest variables directly in src_test() function.

Disabling tests for non-native architectures

For packages using platform.eclass update the global knob in your ebuild:

PLATFORM_NATIVE_TEST="yes"

For other packages, write your src_test() like so:

src_test() {
    if ! use x86 && ! use amd64; then
        ewarn "Skipping unittests for non-native arches"
        return
    fi
    ... existing test logic ...
}

Running unit tests as “root”

For platform2 users, unittests run as non-root by default. If a unittest really needs root access, they can do so. Here is an example CL showing how to. It is recommended to mark such tests with RunAsRoot in their names.

Special Considerations

By default, cros_run_unit_tests will only use FEATURES="test" on packages that:

are in the dependency tree of virtual/target-os (packages only installed in SDK are ignored).
have a known src_test() function; i.e. ebuilds that:
- explicitly declare src_test(), or
- explicitly declare platform_pkg_test() & inherit the platform eclass, or
- inherit the cros-common.mk eclass, or
- inherit the cros-go eclass, or
- inherit the tast-bundle eclass.