| Copyright (c) 2012 The Chromium OS Authors. All rights reserved. |
| Use of this source code is governed by a BSD-style license that can be |
| found in the LICENSE file. |
| |
| Intro |
| ----- |
| We test shill using two sets of tests: unit tests, and integration |
| tests. The unit tests are built using Google Test [1] and Google Mock |
| [2]; the integration tests use autotest [3]. |
| |
| Running unit tests |
| ------------------ |
| - build the shill_unittest target |
| - run the resulting shill_unittest binary |
| - if you're using our ebuild scripts, you can do both by running |
| (chroot)$ cros_run_unit_tests -p shill |
| - run the unit tests from your host machine under gdb |
| (chroot)$ FEATURES="test noclean" emerge-x86-generic shill |
| (chroot)$ gdb_x86_local --board x86-generic \ |
| /build/x86-generic/tmp/portage/chromeos-base/shill-9999/work/shill-9999/shill_unittest |
| (Of course if the unit tests segfaulted, you wouldn't need the emerge |
| step since the build directory would have been retained in the course |
| of the test failing.) |
| |
| - for an incremental workflow, you can setup like this: |
| (chroot)$ cros_workon --board x86-generic shill |
| - incrementally build like this (note: this may not work if your |
| host architecture does not match that of the target): |
| (chroot)$ CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| - and then debug tests like this: |
| (chroot)$ gdb_x86_local --board x86-generic /path/to/checkout/src/platform/shill/shill_unittest |
| |
| - you can specify arguments to the unit tests like this: |
| - full workflow: |
| (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| FEATURES="test" emerge-x86-generic shill |
| - incremental workflow: |
| (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| |
| - if you want to set a breakpoint in gdb, make sure to include the shill |
| namespace. e.g., run |
| (cros-gdb) b shill::EthernetService::GetStorageIdentifier |
| Breakpoint 2 at 0x5555563cc270: file ethernet_service.cc, line 63. |
| rather than |
| (cros-gdb) b EthernetService::GetStorageIdentifier |
| Function "EthernetService::GetStorageIdentifier" not defined. |
| Make breakpoint pending on future shared library load? (y or [n]) n |
| |
| - alternate build arguments: |
| - to see the actual compiler commands that are run: |
| (chroot)$ CFLAGS="-print-cmdline" cros_workon_make --reconf \ |
| --board=link shill |
| - to abort compilation on the first error |
| (chroot)$ MAKEFLAGS="--stop" cros_workon_make --test --board=link \ |
| --reconf shill |
| |
| Running integration tests |
| ------------------------- |
| - build a test image, suitable for a VM: |
| (chroot) src/scripts$ ./build_packages --board=x86-generic |
| (chroot) src/scripts$ ./build_image --board=x86-generic \ |
| --noenable_rootfs_verification test |
| (chroot) src/scripts$ ./image_to_vm.sh --board=x86-generic --test_image |
| |
| - start the VM |
| (host)$ sudo kvm -m 2048 -vga std -pidfile /tmp/kvm.pid \ |
| -net nic,model=virtio -net user,hostfwd=tcp::9222-:22 \ |
| -hda <path to chroot>/src/build/images/x86-generic/latest/chromiumos_qemu_image.bin |
| |
| - DO NOT log in on the console. |
| (doing so will load a user profile onto shill's profile stack; this |
| interferes with the test profile that we use in the autotests) |
| |
| - if you've modified the source after building the image, update shill on |
| the image, and then restart shill: |
| (chroot) src/scripts$ ./start_devserver |
| (chroot) src/scripts$ ssh-keygen -R '[127.0.0.1]:9222' |
| (chroot) src/scripts$ ssh -p 9222 root@127.0.0.1 |
| localhost / # FEATURES="nostrip" gmerge shill |
| localhost / # restart shill |
| |
| - run the tests |
| (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| --remote=127.0.0.1 --ssh_port 9222 \ |
| --args="config_file=wifi_vm_config" WiFiManager |
| |
| To run a specific test out of the test suite, use test_pat option to --args. |
| |
| # Example: To just run the 035CheckWEPKeySyntax test: |
| (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| --remote=127.0.0.1 --ssh_port 9222 \ |
| --args="config_file=wifi_vm_config test_pat=035CheckWEPKeySyntax" WiFiManager |
| |
| - configuration note: if you use a different port |
| (e.g. hostfwd=tcp::9223-:22), you'll need to change: |
| - the ssh_port argument to run_remote_tests |
| - the port numbers in |
| <chroot>/third_party/autotest/files/client/config/wifi_vm_config |
| |
| - debugging test failures |
| - "grep shill /var/log/messages" for log messages |
| - "grep wpa_supplicant /var/log/messages" for supplicant log messages |
| - "wpa_debug debug" to increase supplicant log level |
| - try resetting the test infrastructure |
| - rmmod mac80211_hwsim mac80211 cfg80211 |
| - restart wpasupplicant |
| - rm /tmp/hostapd-test.control/* |
| - examine autotest log files |
| - check how far the test got before it failed |
| $ grep -a ': step ' <test output>/<suite name>/<suite name>.<test name>/debug/<suite name>.<test name>.INFO |
| e.g. |
| (chroot) $ grep -a ': step ' /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| - read the log file |
| (chroot) $ LESSOPEN= less /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| |
| (LESSOPEN= prevents less from misinterpreting the logs as binary files, |
| and piping them through hexdump.) |
| |
| - additional test suites: we have a number of other WiFi test suites |
| (in addition to WiFiManager). these are: WiFiMatFunc, WiFiPerf, |
| WiFiRoaming, WiFiSecMat. the WiFiPerf tests are probably not too |
| relevant to shill (just yet), but the rest probably are. |
| |
| [1] http://code.google.com/p/googletest/ |
| [2] http://code.google.com/p/googlemock/ |
| [3] http://autotest.kernel.org/, |
| http://www.chromium.org/chromium-os/testing/testing-faq |
