Shell Offline Test

In station based testing, sometimes we need DUTs run the test by themselves. For example, the run-in test often takes a long time to finish. We don't want a DUT occupying a station so long. Therefore, we provide “Shell Offline Test” to let DUT be able to run a subset of tests by itself.

This framework is mainly implemented in POSIX shell. A shell script will be generated, which contains all tests the DUT is going to run. We will also make the script a start-up application, so the run-in test will begin on next booting.

Example

{
  "pytest_name": "offline_test.shell.deploy",
  "args": {
    "start_up_service": true,
    "shutdown": "poweroff",
    "test_spec": [
      {
        "shtest_name": "stressapptest",
        "dargs": {
          "seconds": 600,
          "memory_ratio": 0.5,
          "disk_thread": true
        }
      },
      {
        "shtest_name": "wait_for",
        "dargs": {
          "wait_seconds": 10
        }
      },
      {
        "shtest_name": "stressapptest",
        "dargs": {
          "seconds": 600,
          "memory_ratio": 0.5,
          "disk_thread": true
        }
      },
      {
        "pytest_name": "verify_components",
        "dargs": {
          # arguments for pytest `verify_components`
        }
      }
    ]
  }
}

How To Write a New Test

Let's say, your are going to add a test foo, which can be implemented in shell script. To add a test, you need to define a function in ScriptBuilder and register it by ShellTestCase.Register. The function need to add a piece of shell script code into self.tasks.

You need to add a script template in this directory. Each test in test spec will be assigned with an unique task id id, your script template should contain a function named task_{%id%}. {%var%} is a place holder for variable var in python, it will be replaced by the value of var when the test script is generated.

Script Template Example

#!/bin/sh

task_{%id%} () {  # id is automatically assigned by ScriptBuilder
  # the implementation of the test
  ... {%var%} ...  # var is a variable in python.
}

helper_{%id%} () {
  # a helper function using placeholders.
  # so that scripts generated with different IDs won't mess up each other.
  ... {%var%} ...
}

helper () {
  # a helper function doesn't use any placeholders.
}

ScriptBuilder Function

@ShellTestCase.Register('foo')
def TestFoo(self, arg1, arg2...):
  # compute variable values for your script template.
  # ...

  # add new task to queue, will fill in variable values.
  # note that `id` is auto generated by `_AddTask`, so you don't need
  # to pass in `id`.
  self._AddTask('your_script.sh', var1=..., var2=..., ...)

Now you can use foo as shtest_name in test_spec:

test_spec = [
  {
    'shtest_name': 'foo',
    'dargs': {
      'arg1': ...,
      'arg2': ...,
      ...
    }
  }
]

Using Factory Pytests

Some tests might be hard to implement in pure shell script, you'd like to reuse the existing pytest. You can add an item in test_spec, which has attribute pytest_name, and attribute dargs for the arguments for that pytest.

{
  'pytest_name': 'bar',
  'dargs': {
    'arg1': ...,
    'arg2': ...,
  }
}

The pytest will be start by factory.par's run_pytest functionality. However, since there is no Goofy on DUT and DUT might not run Chrome OS, the pytest might not work properly.