tast: Only write control messages when running tests.

When the tast command instructs a test runner to run tests,
the results (including fatal errors) are returned via
JSON-encoded control messages streamed to stdout.

When a test runner is performing some other task like
listing tests, the result (e.g. test data) is still written
to stdout, but the runner is supposed to communicate fatal
errors by exiting with a nonzero status and writing a
message to stderr.

I think I broke this with recent refactoring and made
runners always write errors via control messages regardless
of what they're doing. This results in the tast command
printing useless errors like 'json: cannot unmarshal object
into Go value of type []testing.Test' (after trying to
decode a control message as something else) instead of
more-helpful errors like 'No test bundles matched by
"/usr/local/libexec/tast/bundles/*"'.

BUG=chromium:817688
TEST=manually verified that useful errors are printed again
     by the "tast list" command and added a unit test

Change-Id: I51aa40479db70ea910cc4c72d49970118531b99d
Reviewed-on: https://chromium-review.googlesource.com/1011708
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>
2 files changed
tree: 784db1e50d041dc00d77048df2072ad680c117dc
  1. docs/
  2. src/
  3. fast_build.sh
  4. README.md
README.md

Tast

Tast is an integration-testing system for Chrome OS. Its focus is on maintainability, speed, and ease of interpreting and reproducing test results. It supports building, deploying, and running tests. It doesn't implement other functionality like managing labs of devices used for testing, scheduling tests, or storing test results.

To try it out, see the quickstart document. The overview is a good starting point for learning more about how the system is structured.

Directory structure

This repository is organized in accordance with the Go in Chromium OS suggestions.

  • src/chromiumos/
    • cmd/local_test_runner/ - main package for the local_test_runner executable used to execute local test bundles on-device.
    • cmd/remote_test_runner/ - main package for the remote_test_runner executable used to execute remote test bundles on the host system.
    • cmd/tast/ - main package for the tast executable used to build and run tests.
      • ... - Packages used only by the tast executable.
    • tast/... - Common packages shared by multiple executables.

Tests are located in the tast-tests repository.

Documentation

For more details, see the docs subdirectory.

GoDoc