Chrome OS Reference Python Factory Tests

This folder contains all reference factory tests for Chrome OS Factory Software.

For more details of individual test, please look at the SDK document or read the source of individual tests.

Pytest Overview

Take start.py as an example, a pytest should contain 4 sections.

Copyright header
Test documentation
Imports
Implementation

Copyright header

Always add copyright header to the beginning of the file.

# Copyright 2022 The ChromiumOS Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

2022 is year of file created date.

Test Documentation

To help both developers and non-developers understand better how these tests works, how to use it and what's needed to make it run, we have defined a template for documentation. The tests will be processed by Sphinx Documentation, so you just need to write the module doc string in reStructedText format with following sections:

"""Test summary here.

Description
-----------
A section for who wants to know what this test is, and how to use it.

You probably want to write the first section as a general description, and
remaining sections to explain more details, and how to assign the arguments
if the test arguments are pretty complicated (or try using a JSON configuration
via `config_utils` instead).

Test Procedure
--------------
Write "This is an automated test without user interaction." as first line if
the test does not need user interaction.

This is a section as "reference to write SOP for operators". Use simpler words
and abstraction of what will happen.

Dependency
----------
This is a section about "what people will need when they want to port the test
for a new platform, for example Android or even Windows.

Try to list everything needed outside Chrome OS factory repo, especially
kernel and driver dependency.

If you are using Device API (`cros.factory.device.*`), try to list the explicit
API you are using, with few typical dependency that it needs.

Examples
--------
Examples of how to use this test (in test list). Usually we want a "minimal" one
explaining what default behavior it will do, with few more examples to
demonstrate how to use the arguments.
"""

To preview your changes to doc, run make doc in chroot and browse build/doc/pytests/index.html.

For more examples on how to write these docs and how they looks on SDK site, try to look at following tests:

Imports

Import lines are organized in 3 categories:

Standard python libraries, e.g. import os
Third-party modules, e.g. import jsonlibrpc
ChromeOS Factory modules

Implementation

All pytests are a python class inherits test_case.TestCase, which is a subclass of unittest.TestCase from Python unittest module. When a pytest is executed, the following functions will be called (in exact order):

setUp()
runTest()
tearDown()

Writing a new test

All the tests must have implementation file named in lowercase_with_underline style and the Python class named in CamelCase style, following PEP-8 and Chrome OS Factory Coding Style. You can write test as a single Python file like start.py, or implement it as Python package in its own folder like probe/.

To use your test (say mytest.py), define an entry in test list:

  {
    "pytest_name": "mytest"
  }

Writting a test with reboot process

There are two ways to write a test that contains reboot process.

We recommended to use shutdown.py if feasible.
You can add RebootStep as an independent test item in the test group, it will trigger a reboot and continue executing the remain test items after reboot.
For example:
```
{
  "label": "Test Group Label",
  "subtests": [
    "PytestWithReboot",
    "RebootStep",
    "PytestWithReboot"
  ]
}
```
Sometimes the reboot is triggered by 3rd-party commands/operations, which is not supported by shutdown.py.
In this situation, you can set allow_reboot: true for the test item in test_list, and then use AddTask(task, *args, reboot=True, **kwargs) to add tasks in your pytest.
For example:
```
def Task(arg):
  # task
  ...

def RebootTask(arg):
  # task with reboot process
  ...

AddTask(Task, arg)
AddTask(RebootTask, arg, reboot=True)
AddTask(Task, arg)
```

See JSON_TEST_LIST.md and test_case.py for more details.

Using arguments

To read arguments specified from test list, use cros.factory.utils.arg_utils.Arg by adding declarations as class variable. For example:

  ARGS = [Arg('wait_secs', int, 'Wait for N seconds.', default=0)]

Then you can use this by reading self.args.wait_secs.

Since JSON serialization doesn‘t support tuple type, new pytests shouldn’t use tuple in argument type, and should use list instead.

Schema validation

If the argument is complicated, you may want to define a schema rule to validate the structure of the schema. For example:

  Arg(..., schema=schema_object)

The schema object should be one of the schema classes defined in schema.py. Then the argument value would automatically be validated with the schema rule.

You can check tablet_rotation.py as an example usage.

Using user interface

The Chrome OS Factory Software provides a web based user interface. The test UI object will be injected to test instance as self.ui. Read Test UI API for more details.