breadcrumbs: Testing Home > page_name: running-smoke-suite-on-a-vm-image title: Running Smoke Suite On a VM Image


Introduction

The VMTest “smoke suite” is a suite of client-side autotests that performs basic sanity checks on a Chromium OS build.

Each of the tests in the smoke test suite is capable of running in a virtual machine.

This allows the smoke suite to be run as part of the automated build process on a ‘builder’.

Smoke Suite tests

The Smoke suite currently consists of the following tests (this list is subject to change):

  • build_RootFilesystemSize
  • desktopui_ChromeFirstRender
  • desktopui_FlashSanityCheck
  • desktopui_KillRestart(‘^chrome$’)
  • desktopui_KillRestart(‘^session_manager$’)
  • logging_CrashSender
  • login_Backdoor(‘$backdoor’)
  • login_BadAuthentication
  • login_CryptohomeIncognitoMounted(‘$backdoor’)
  • login_CryptohomeMounted
  • login_LoginSuccess
  • platform_FilePerms
  • platform_OSLimits

Running the Smoke Test Suite on a Virtual Machine (VM)

Useful Links:

The instructions on this page should be enough to get you going. They install a testable Chromium OS image on a VM and run the smoke suite. However, if you want more details, you can always go to one of the following pages:

Install qemu-kvm

First of all, to run in a vm, you must have vm software installed.

On Ubuntu Lucid, use apt-get to install qemu-kvm:

Build a testable VM image

WARNING: After crbug/710629, ‘betty’ is the only board regularly run through pre-CQ and CQ VMTest and so is the most likely to work at ToT. ‘betty’ is based on ‘amd64-generic’, though, so ‘amd64-generic’ is likely to also work for most (non-ARC) tests.

Next, build the testable VM image. The following steps assume that:

  1. depot_tools/repo have been installed and configured.
  2. Chromium OS source code has been checked out.
  3. A chroot has been created with make_chroot.

The newly created VM image should be in the following path (relative to src/scripts):

../build/images/${BOARD}/latest/chromiumos_qemu_image.bin

Start a VM image

Before launching autotests, its fun to boot up a virtual Chromium OS on your desktop!

Run the autotest tests on the VM image

Autotests can be run against the VM via test_that.

Don't forget to kill the head-less VM when you are done with it!

Run the autotest tests on the VM image using cros_run_vm_test

Another script, cros_run_vm_test, can be used to start the VM, run autotest tests on it, and then shut down the VM, all from a single invocation:

Troubleshooting

No space left on device

If you get an error that looks like this:

Unhandled OSError: [Errno 28] No space left on device: '/home/autotest/tmp/_autotmp_Woz_Qyharness-fifo'

...this may be because you have run out of space on your “stateful partition”. This can happen if you leave the VM running for a long time and errors fill up the /var/log folder. In my case, this folder contained 250M of data! You should go into /var/log and cleanup stuff.

Use df to determine how much space is available in the stateful partition:

stop <daemon_name>.You may be getting lots of errors if you're running a board-specific image with drivers for hardware that is not present on the VM. If this is the case, try stopping the responsible daemon with

Could not set up host forwarding rule ‘tcp::9222-:22’

If you get this error when running cros_vm --start, make sure that you don't have a VM already running. This should kill it for you.

cros_vm --stop

Could not initialize KVM, will disable KVM support

This is probably because you don't have the kvm module loaded. The VM will run.....very.......slowly.

open /dev/kvm: No such file or directory
Could not initialize KVM, will disable KVM support
$ modprobe kvm_intel    # if you have an AMD machine, use 'kvm_amd'

Note that this module interferes with VirtualBox so you may need to remove it (rmmod kvm_intel) if you want to use this.

Updating a client autotest

Autotest tests are usually built and compiled on the autotest server machine.

A ‘client’ test is installed on the client by the autotest infrastructure when the test is run.

The autotest infrastructure is maintained in the chromeos-base/autotest cros_workon ebuild.

Individual tests and test suites are in the chromeos-base/autotest-tests ebuild.

The new way: Use cros_workon with all Python tests

Working on all Python autotests should be simple and fast.

First cros_workon autotest for the board on which you plan to run tests:

Now, test_that will detect that you are working on autotests and automatically copy over Python in the autotest source tree to the sysroot.

Run test_that as usual:

The old way: emerge individual autotest tests

If you have a test with binary dependencies, you must rebuild them explicitly:

This can take a very, very long time since it will rebuild ALL tests.

To speed this up, set the TESTS variable on the command line when runnning emerge to install an individual test:

Note that doing this will deactivate all other tests! This is a problem for test suites.

To rebuild a whole test suite, specify the suite and all individual tests in the suite. For example, to rebuild smoke suite:

Then go grab a coffee...

Debugging

You can use chromium os debugging tips on VM as well.