Google Git
Sign in
chromium / chromium / src / d4bbeec02cbdaf23d6fc04c616b5d7f1c7c9b851 / . / testing / buildbot / filters
tree: 517367ce2998c09a325f19a59bd97b99751b347b [path history] [tgz]
  1. android.asan.breakpad_unittests.filter
  2. android.asan.content_browsertests.filter
  3. android.asan.services_unittests.filter
  4. android.asan.unit_tests.filter
  5. bfcache.browser_tests.filter
  6. bfcache.content_browsertests.filter
  7. bfcache.content_unittests.filter
  8. bfcache.unit_tests.filter
  9. BUILD.gn
  10. cast-linux.content_browsertests.filter
  11. chromeos.base_unittests.filter
  12. chromeos.browser_tests.filter
  13. chromeos.media_unittests.filter
  14. chromeos.ozone_unittests.filter
  15. chromeos.services_unittests.filter
  16. chromeos.unit_tests.filter
  17. code_coverage.browser_tests.filter
  18. fuchsia.base_perftests.filter
  19. fuchsia.content_unittests.filter
  20. fuchsia.mojo_unittests.filter
  21. fuchsia.net_perftests.filter
  22. fuchsia.net_unittests.filter
  23. fuchsia.services_unittests.filter
  24. fuchsia.ui_base_unittests.filter
  25. gpu.skiarenderer_vulkan_blink_web_tests.filter
  26. gpu.skiarenderer_vulkan_content_browsertests.filter
  27. mac_window_server_killers.browser_tests.filter
  28. mojo.fyi.network_webview_CTS_test.filter
  29. mojo.fyi.network_webview_instrumentation_test_apk.filter
  30. OWNERS
  31. PRESUBMIT.py
  32. README.md
  33. site_isolation_android.content_browsertests.filter
  34. skia_renderer.browser_tests.filter
  35. skia_renderer.content_browsertests.filter
  36. vulkan.content_browsertests.filter
  37. webrtc.content_unittests.filter
  38. webrtc_functional.browser_tests.filter
  39. webui_html_imports_polyfill_browser_tests.filter
  40. webui_html_imports_polyfill_interactive_ui_tests.filter
testing/buildbot/filters/README.md

Test filter files

Summary

This directory contains files that list tests that are not yet ready to run in a particular mode. For example - the cast-linux.content_browsertests.filter file lists tests that should be excluded when running content_browsertests on the Cast device or bot (e.g. on 'Cast Linux).

File syntax

Contents of test filter files follow the syntax below:

  • Empty lines are ignored.
  • Any text after ‘#’ on each line is treated as comments and ignored.
  • All other lines specify a single test name pattern.
    • Negative patterns: Patterns prefixed with the ‘-’ character specify tests to exclude from a test run.
    • Positive patterns: All other patterns specify tests to include in a test run.

A test will be executed if and only if it matches any of the positive patterns but does not match any of the negative patterns (please see documentation of --gtest_filter for more details and examples).

Please see [https://crbug.com/587527] for discussion how “positive” and “negative” test patterns should be combined in presence of both the --gtest_filter and the --test-launcher-filter-file command line flags.

Example test filter file for excluding a set of tests:

# crbug.com/417518: Get tests working w/ --site-per-process
-BrowserTest.OtherRedirectsDontForkProcess
-ChromeRenderProcessHostTest.*
-ReferrerPolicyTest.HttpsRedirect

# crbug.com/448592: Get extension browsertests working w/ --site-per-process
-IsolatedAppTest.CookieIsolation
-IsolatedAppTest.CrossProcessClientRedirect
-IsolatedAppTest.IsolatedAppProcessModel
-IsolatedAppTest.SubresourceCookieIsolation

Usage

When running tests on desktop platforms, the test filter file can be specified using --test-launcher-filter-file command line flag. Example test invocation (single filter file):

$ out/dbg/content_browsertests \
    --test-launcher-filter-file=testing/buildbot/filters/foo.content_browsertests.filter

Example test invocation (multiple filter files, separated by ‘;’):

$ out/dbg/content_browsertests \
    --test-launcher-filter-file=testing/buildbot/filters/foo.content_browsertests.filter;\
                                testing/buildbot/filters/foo.chromeos.content_browsertests.filter

When running tests on Android, the test filter file can also be specified using --test-launcher-filter-file command line flag. Example test invocation:

$ out/android/bin/run_content_browsertests \
    --test-launcher-filter-file=testing/buildbot/filters/foo.content_browsertests.filter

Applicability

Test filter files described here are currently only supported for gtest-based tests.

For excluding layout tests when running with a particular command line flag, see third_party/WebKit/LayoutTests/FlagExpectations/README.txt.

Adding new test filter files

Please use the following conventions when naming the new file:

  • Please include the name of the test executable (e.g. content_browsertests).
  • Please use .filter suffix.
  • Feel free to add other relevant things into the file name (e.g. the mode the file applies to - for example site-per-process).

When adding a new filter file, you will need to:

  • Update //testing/buildbot/filters/BUILD.gn.
  • Add //testing/buildbot/filters:foo_filters to the appropriate test target.
  • Add '--test-launcher-filter-file=../../testing/buildbot/filters/foo.filter' to the desired test suite(s) in test_suites.pyl.
  • Run testing/buildbot/generate_buildbot_json.py to update .json files.