The platform.CrosConfig Tast test is a safety net for validating changes to identity, configuration or any CLI did not break the device you are working on or other seemingly unrelated devices.
The test will read in a set of mosys and cros_config commands to execute on the device under test (DUT) and check the results against a golden file containing the expected output.
Setting up a new device involves two steps, setting up the device specific commands and creating a new golden results file.
Each device needs a cros_config_device_commands.yaml file to configure what commands are run on the DUT. For a unibuild configuration this file will contain all the commands for all configurations, but does not require all to be setup at once.
The device specific commands file should inherit the cros_config_test_common.yaml file to pick up all of the common commands, but it isn't required.
During the build the common commands will be merged and a single commands JSON file will be created and installed on dev and test images automatically.
A simple device specific commands YAML file for nautilus and nautiluslte looks like the following:
imports: - "cros_config_test_common.yaml" chromeos: devices: - device-name: "nautilus" command-groups: - *mosys_base_cmds - *cros_config_unibuild_cmds - device-name: "nautiluslte" command-groups: - *mosys_base_cmds - *cros_config_unibuild_cmd
The real nautilus commands file is here
If you only want to test the common commands this is all you need to set up. If you want to extend the commands you will need to create a new custom command group and add it to the command-groups list.
See the the Command Groups section for information on build command groups.
For unibuilds, the device specific commands YAML lives in the private-overlays repo under the overlay-<board>-private/chromeos-base/chromeos-config-bsp directory.
Create a new directory named tast_files and place the new YAML file in this directory. This keeps the testing configuration close to the model.yaml configuration.
An example is the Nautilus test configuration.
NOTE: The testing configuration needed a seperate directory because the unibuild configuration picks up all YAML files in the files directory causing errors during the build.
For non-unibuilds, the device specific commands YAML also lives in the private overlays repo under the overlay-<board>-private/chromeos-base/chromeos-bsp-<board>-private directory in a new directory named tast_files.
An example is the Caroline test configuration.
Updating the ebuild only involves two steps:
cros-config-test.eclass.src_install() step add the install_cros_config_test_files command.Emerge the ebuild and it should copy the device specific commands YAML and any golden results files to the /build/<board>/tmp/chromeos-config/tast directory.
To add or remove commands for an existing devices setup is simply editing the device specific commands YAML and its corresponding golden file. Deploy to the DUT and run the test. See the Test Comparison and Errors section on what will cause test failures and what will pass.
If you add or remove common commands you should update all of the devices golden files with the new results.
All commands run on the DUT need to be included in a YAML Command Group. A Command Group should conform to the following schema:
group-name: &group-anchor name: 'command to run' args: - 'command 1 args' - 'command 2 args'
For examples see cros_config_test_common.yaml The common commands YAML defines two command groups, one for the mosys command and another for the cros_config command.
The common commands are a set of Command Groups that can be added to a device specific commands list. The common commands need to work on all devices.
The cros_config_test_common.yaml is located in the third_party/chromeos-overlay/chromeos-base/cros-config-test/files directory.
When updating the common commands, you will need to update all of the golden results files for all devices that include that command group. See the Creating and Updating Golden Results File section.
To merge the common commands and device specific commands:
emerge-<board> chromeos-base/cros-config-test
This builds the final tarball with the cros_config_test_commands.json file and all of the golden results files.
During the build_image stage for dev and test images this package will be included in the final package. cros-config-test is included in the virtual target target-chromium-os-test.
If you have a new setup or have made modifications to the commands or golden files you will need to deploy the new package to the DUT.
cros deploy --board=<board> --root=/usr/local <DUT ipaddr> chromeos-base/cros-config-test
Run the Go Tast test:
tast run <DUT ipaddr> platform.CrosConfig
For more information on running the tests see the Tast running tests docs.
The test will output commands that it is running and at the bottom will give an overall pass or failure for the platform.CrosConfig test. The results of the test will be copied back to the host you ran the command from to the /tmp/tast/results/latest directory.
The results directory will contain several logs and directories, the most interesting are:
system_logs directory contains the contents of the system logs during the run.crashes directory contains crash results from the run, empty if none.tests/platform.CrosConfig directory contains log.txt of CrosConfig messages.tests/platform.CrosConfig directory contains the new golden file named <board>_output.json.See the following Test Comparison and Errors section on how errors are determined and the Tast interpreting test results docs.
To make life livable for everybody, the only failed comparisons between a test run and an existing golden file are if and only if the exact command exists in the golden file and the values differ.
See details below of the different scenarios:
Crashes of any of the commands will automatically cause the test to FAIL.
To create or update a golden results file you can simply copy the /tmp/tast/results/latest/tests/platform.CrosConfig/<board>_output.json output file of a test run, if correct, to the device's tast_files directory. Build a new cros-config-test package, re-deploy and re-run the test to validate the run passes with no errors.
If you modify a common command group, you will need to update all of the devices that include the common command group. There is currently no easy way to do this except to run the new common config against all devices and add the new golden results file to the device's tast_files directory as documented above.
The platform.CrosConfig test is a Go program that is part of the Tast cros platform bundle of local tests. Local Tast tests are tests that are run on the DUT. The cros_config.go test lives in the src/platform/tast-tests/src/chromiumos/tast/local/bundles/cros/platform directory. The executable will automatically be built and included in all dev and test images.
You shouldn't need to modify this program for setting up and running the platform.CrosConfig test.
We are in trying to get this test to be part of the normal CQ run, at this time it is still flagged as ‘informational’ so it will be excluded on all official CQ runs.