docs/webui/testing_webui.md

<style>
.note::before {
  content: 'Note: ';
  font-variant: small-caps;
  font-style: italic;
}

.doc h1 {
  margin: 0;
}
</style>

# Testing WebUI Pages

[TOC]

Chromium WebUI pages should be tested using Mocha test suites.
[Mocha](https://mochajs.org) is a flexible, lightweight test framework with
strong async support. With Mocha, we can write small unit tests for individual
features of components instead of grouping everything into a monolithic browser
test.

There are a few pieces of Chromium WebUI infrastructure designed to make
setting up WebUI Mocha browser tests easier:

* `build_webui_tests` is a gn build rule that preprocesses test `.ts` files,
  compiles them to JavaScript, and places them in a `.grdp` file. This file
  can then be included in the WebUI test `.grd` so that the files can be
  served at runtime. This rule is further documented at
  [WebUI Build Configuration](./webui_build_configuration.md)
* `WebUIMochaBrowserTest` is a test base class that WebUI tests should
  inherit from. It handles browser test setup and teardown, injects Mocha,
  loads the `chrome://` page being tested, and imports the test files. This
  class also ensures code coverage metrics are collected when JS code
  coverage is enabled. The API for this class is documented in its
  [header
  file](https://source.chromium.org/chromium/chromium/src/+/main:chrome/test/base/web_ui_mocha_browser_test.h)

Note: For the default test loading logic in `WebUIMochaBrowserTest` to work,
`test_loader.html`, `test_loader_util.js` and `test_loader.js` should be
registered with the production Web UI's data source. This can be done manually,
or (most commonly) can be done by calling `SetupWebUIDataSource()`. If using
the test loader is not feasible for a given test - for example, test relies on
loading a specific URL such that loading `chrome://foo-bar/test_loader.html`
would break the test - see [Unusual Test Setups](#unusual-test-setups)

### Adding the BrowserTest
UIs should add a `.cc` file with the definition of their BrowserTest class in
`chrome/test/data/webui/my_webui_dir_name`. The directory structure
of `chrome/test/data/webui` should mirror that of
`chrome/browser/resources/` where the WebUI TypeScript/HTML code being tested
resides. The BrowserTest class defined in the `.cc` file should inherit from
`WebUIMochaBrowserTest`.
Example:

```c++
// chrome/test/data/webui/foo_bar/foo_bar_browsertest.cc

#include "base/test/scoped_feature_list.h"
#include "chrome/common/webui_url_constants.h"
#include "chrome/test/base/web_ui_mocha_browser_test.h"
#include "content/public/test/browser_test.h"

class FooBarBrowserTest : public WebUIMochaBrowserTest {
 protected:
  FooBarBrowserTest() {
    set_test_loader_host(chrome::kChromeUIFooBarHost);
  }

 private:
  // Necessary if you have features (or the UI itself) behind a feature flag.
  base::test::ScopedFeatureList scoped_feature_list_{foo_bar::kSomeNewFeature};
};
```

Note: If testing a `chrome-untrusted://` UI, also call:
```c++
set_test_loader_scheme(content::kChromeUIUntrustedScheme);
```
in the constructor.

After defining the class, create one or more `IN_PROC_BROWSER_TEST_F`
functions. Each should run one or more Mocha tests.

``` c++
typedef FooBarBrowserTest FooBarAppTest;
IN_PROC_BROWSER_TEST_F(FooBarAppTest, All) {
  // Run all Mocha test suites found in app_test
  RunTest("foo_bar/app_test.js", "mocha.run();");
}
```
Should you write a large number of tests in a single file (e.g., for
one particular custom element), it may be necessary to run different suites or
even individual tests from that same file in separate `IN_PROC_BROWSER_TEST_F`
invocations, to avoid having one very long test that is at risk of timing out.

Example:

```c++
// This class defines a convenience method to run one Mocha test suite in the
// search_box_test.js file at a time.
class FooBarSearchBoxTest: public FooBarBrowserTest {
 protected:
  void RunTestSuite(const std::string& suiteName) {
    FooBarBrowserTest::RunTest(
        "foo_bar/search_box_test.js",
        base::StringPrintf("runMochaSuite('%s');", suiteName.c_str()));
  }
};

IN_PROC_BROWSER_TEST_F(FooBarSearchBoxTest, SearchNonEmpty) {
  RunTestSuite("SearchNonEmpty");
}

IN_PROC_BROWSER_TEST_F(FooBarSearchBoxTest, SearchEmpty) {
  RunTestCase("SearchEmpty");
}

// This class defines a convenience method to run one Mocha test from the
// FooBarChildDialogTest suite at a time.
class FooBarChildDialogTest: public FooBarBrowserTest
 protected:
  void RunTestCase(const std::string& testCase) {
    FooBarBrowserTest::RunTest(
        "foo_bar/child_dialog_test.js",
        base::StringPrintf("runMochaTest('FooBarChildDialogTest', '%s');",
                           testCase.c_str()));
  }
};

IN_PROC_BROWSER_TEST_F(FooBarChildDialogTest, OpenDialog) {
  RunTestCase("OpenDialog");
}

IN_PROC_BROWSER_TEST_F(FooBarChildDialogTest, ClickButtons) {
  RunTestCase("ClickButtons");
}
```

### Adding the Mocha test files
Mocha test files should be added alongside the `.cc` file in the same
directory. Each test file should contain one or more Mocha test suites. Often
each major custom element of a UI will have its own Mocha test file, as will
many of the data classes.

A build time check exists as part of `build_webui_tests()` to ensure Mocha test
files are actually hooked up to a C++ test file. Mocha test files must end with
the `\_test.ts` suffix. Any other TS supporting files that don't actually hold
Mocha test cases should not finish with this suffix otherwise the automated
check will trigger erroneously.

`WebUIMochaBrowserTest` sets up a `WebUIDataSource` to serve all the test files
from `chrome://webui-test` (or `chrome-untrusted://webui-test`, if configured
via `set_test_loader_scheme`).

```ts
// chrome/test/data/webui/foo_bar/search_box_test.ts

import 'chrome://foo-bar/foo_bar.js';

import type {FooBarSearchBoxElement} from 'chrome://foo-bar/foo_bar.js';
import {assertEquals, assertFalse, assertTrue} from 'chrome://webui-test/chai_assert.js';
import {eventToPromise} from 'chrome://webui-test/test_util.js';

suite('SearchNonEmpty', function() {
  let searchBox: FooBarSearchBoxElement;

  setup(function() {
    document.body.innerHTML = window.trustedTypes!.emptyHTML;
    searchBox = document.createElement('foo-bar-search-box');
    document.body.appendChild(searchBox);

    // More setup goes here
  });

  test('search abc', async function () {
    const search = searchBox.shadowRoot!.querySelector('input');
    assertTrue(!!search);
    search.value = 'abc';
    await eventToPromise('search-change', searchBox);

    // Do some assertions
  });

  test('search def', async function () => {
    // etc
  });
});

suite('SearchEmpty', function() {
  let searchBox: FooBarSearchBoxElement;

  setup(function() {
    document.body.innerHTML = window.trustedTypes!.emptyHTML;
    const searchBox = document.createElement('foo-bar-search-box');
    document.body.appendChild(searchBox);

    // Different setup goes here
  });

  test('search abc when empty', async() {
    const search = searchBox.shadowRoot!.querySelector('input');
    assertTrue(!!search);
    search.value = 'abc';
    await eventToPromise('search-change', searchBox);

    // Do some assertions
  });
});

```
### Setting up the build
To build your tests as part of the `browser_tests` target, add a BUILD.gn file
in your folder:

```python
# chrome/test/data/webui/foo_bar/BUILD.gn

import("../build_webui_tests.gni")

build_webui_tests("build") {
  cc_test_files = [ "foo_bar_browsertest.cc" ]

  # Test files go here
  files = [
    "app_test.ts",
    "child_dialog_test.ts",
    "search_box_test.ts",
  ]

  # Path mapping the WebUI host URL to the path where the WebUI's compiled
  # TypeScript files are placed by the TS compiler.
  ts_path_mappings =
      [ "chrome://foo-bar/*|" +
        rebase_path("$root_gen_dir/chrome/browser/resources/foo_bar/tsc/*",
                    target_gen_dir) ]
  ts_deps = [
    "//chrome/browser/resources/foo_bar:build_ts",
  ]
}
```

Note: If testing a `chrome-untrusted://` UI, also set the
`is_chrome_untrusted` parameter for `build_webui_tests()` to `true`, to allow
importing shared test files from `chrome-untrusted://webui-test/` paths.

You then need to hook up the targets generated by this file, and list your
browsertest.cc source file, in `chrome/test/data/webui/BUILD.gn`:

```python
# chrome/test/data/webui/BUILD.gn

source_set("browser_tests") {
  testonly = true

  sources = [
    # Lots of files
    # Big list, add your .cc file in alphabetical order
    "foo_bar/foo_bar_browsertest.cc",
    # Lots more files
  ]

  # Conditionally added sources (e.g., platform-specific)

}

# Further down in the file...

generate_grd("build_grd") {
  testonly = true
  grd_prefix = "webui_test"
  output_files_base_dir = "test/data/grit"
  out_grd = "$target_gen_dir/resources.grd"

  deps = [
    ":build_chai_grdp",
    ":build_mocha_grdp",
    # Lots of other deps here
    # Add your build_grdp target:
    "foo_bar:build_grdp",
    # Lots more other deps
  ]

  grdp_files = [
    # Big list of grdp files
    # Add your grdp in alphabetical order
    "$target_gen_dir/foo_bar/resources.grdp",
    # Lots more grdp files
  ]

  # Conditionally added (e.g. platform-specific) grdps and deps go here
}
```

### Unusual Test Setups
Some UIs may need to rely on loading a particular URL, such that using the
default `chrome://foo-bar/test_loader.html` URL that is used to dynamically
import the test modules may not work. Other UIs need to do additional setup
on the C++ side of the test, and simply want to run a test on a specified
WebContents instance. These cases are accommodated by additional methods in
`WebUIMochaBrowserTest`. A few brief examples are provided below.

Example: loading a different URL
```c++
IN_PROC_BROWSER_TEST_F(FooBarBrowserTest, NonExistentUrl) {
  // Invoking the test from a non existent URL chrome://foo-bar/does-not-exist/.
  set_test_loader_host(
      std::string(chrome::kChromeFooBarHost) + "/does-not-exist");
  RunTestWithoutTestLoader("foo_bar/non_existent_url_test.js", "mocha.run()");
}
```

Example: Advanced setup required in C++
```c++
IN_PROC_BROWSER_TEST_F(FooDialogBrowserTest, MyTest) {
  ui_test_utils::NavigateToURLWithDisposition(
      browser(), GURL(chrome::kSomeOtherBaseUrl),
      WindowOpenDisposition::CURRENT_TAB,
      ui_test_utils::BROWSER_TEST_WAIT_FOR_LOAD_STOP);
  FooDialog* my_dialog = FooDialog::ShowConstrained(
      kDummyParam,
      browser()->tab_strip_model()->GetActiveWebContents(),
      browser()->window()->GetNativeWindow());
  content::WebContents* web_contents = my_dialog->webui_->GetWebContents();

  ASSERT_TRUE(RunTestOnWebContents(
      web_contents, "foo_dialog/foo_dialog_test.js", "mocha.run()",
      /*skip_test_loader=*/true));
}
```

### Running tests

You can build the `browser_tests` target and run your WebUI tests just like
other browser tests. You may filter tests by class or test name (the arguments
to `IN_PROC_BROWSER_TEST_F`) via `--gtest_filter`:

```sh
autoninja -C out/Release browser_tests
./out/Release/browser_tests --gtest_filter=FooBarSearchBoxTest.Empty
```

### Debugging tests

You can run a subset of the Mocha test cases, by using `test.only()` instead of
`test()`. For example:

```ts
test.only('MyTestcase', function() {...});
```

You can skip a subset of the Mocha test cases, by using `test.skip()` instead of
`test()`. For example:

```ts
test.skip('MyTestcase', function() {...});
```

When authoring a new test case or when investigating existing test failures, you
will often have a need to step through the code in the DevTools debugger, to
figure out what is happening. Use the `launchDebugger` helper function to do
that. Detailed instructions on how to use it exist in the source code
documentation [here](https://source.chromium.org/chromium/chromium/src/+/main:chrome/test/data/webui/test_util.ts;l=107-144;drc=b1866df4398a971088ba287d4c7efe704f6bc4b1).

### Disabling tests

Tests can be disabled at the GTest (C++) level or the Mocha (TypeScript)
level. Disabling at the Mocha level is preferred as it allows for more
granular control (skipping individual test cases rather than entire suites)
and provides better visibility into which specific tests are failing. If the
entire suite is flaky or failing, then disabling the GTest might be
appropriate.

If your team receives a flaky test bug and you notice that the whole C++ GTest
suite is disabled, before deflaking you should first land a CL to disable the
specific test cases that are causing problems. After that lands, you can
investigate the flakes or failures. This will reduce the chance that other
tests in the suite start failing while they are disabled.

#### Determining which test cases are failing
Test cases can fail or entire suites can time out if they take too long to run.
You can expand the "Summary" that appears below the "Failure Reason" and
search for the cases marked "failed" or the last test case marked "started" if
there is a timeout. Here is an example where the "RenamesBookmark" case failed.

```
[ RUN      ] SidePanelBookmarksAppTest.General2
[25749:440407:0512/171449.559368:INFO:components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc:239] No machine level policy manager exists.
Trying to load the allocator multiple times. This is *not* supported.
[25749:440407:0512/171449.854681:INFO:CONSOLE:35] "start", source: chrome://webui-test/mocha_adapter_simple.js (35)
...
[25749:440407:0512/171450.076260:INFO:CONSOLE:44] "    started: General Part2 SetsExpandedSearchResultDescription", source: chrome://webui-test/mocha_adapter_simple.js (44)
[25749:440407:0512/171450.101585:INFO:CONSOLE:47] "     passed: General Part2 SetsExpandedSearchResultDescription", source: chrome://webui-test/mocha_adapter_simple.js (47)
[25749:440407:0512/171450.102566:INFO:CONSOLE:44] "    started: General Part2 RenamesBookmark", source: chrome://webui-test/mocha_adapter_simple.js (44)
[25749:440407:0512/171450.114796:INFO:CONSOLE:58] "     failed: General Part2 RenamesBookmark
AssertionError: expected false to be true
    at assertTrue (chrome://webui-test/chai_assert.js:14:12)
    at Context.<anonymous> (chrome://webui-test/side_panel/bookmarks/power_bookmarks_app_test.js:587:13)", source: chrome://webui-test/mocha_adapter_simple.js (58)
[25749:440407:0512/171450.115013:INFO:CONSOLE:44] "    started: General Part2 BlursRenameInput", source: chrome://webui-test/mocha_adapter_simple.js (44)
[25749:440407:0512/171450.132939:INFO:CONSOLE:47] "     passed: General Part2 BlursRenameInput", source: chrome://webui-test/mocha_adapter_simple.js (47)
...
[25749:440407:0512/171450.385904:INFO:CONSOLE:62] "end: 18/19 ok", source: chrome://webui-test/mocha_adapter_simple.js (62)
[25749:440407:0512/171450.385944:INFO:CONSOLE:64] "TEST all complete, status=FAIL, duration=531ms", source: chrome://webui-test/mocha_adapter_simple.js (64)
```

Individual test cases also appear as an additional test result. See the
[below section](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/webui/testing_webui.md#reporting-webui-test-results-to-luci)
on how test cases show up in LUCI.

If there is a timeout, it is helpful to look at a few of the failures to see if
the same test case is always timing out or different tests (especially ones
near the end) are flaking. If different cases are flaking each run, the entire
suite will likely need to be disabled while an owner of the tests splits it
into two test suites. If the same case is always timing out, then that can be
disabled.

#### Disabling in Mocha (Preferred)

To disable a single test case for all platforms, use `test.skip()` instead of
`test()`:

```ts
// TODO(crbug.com/511960512): Flaky test.
test.skip('MyTestcase', function() {...});
```

To disable an entire suite, use `suite.skip()` instead of `suite()`:

```ts
// TODO(crbug.com/511960512): Flaky test.
suite.skip('MySuite', function() {...});
```

To disable a test for a specific platform, use `<if expr>`:

```ts
// TODO(crbug.com/511960512): Flaky test.
// <if expr="not is_macosx">
test('LogsBookmarkCountMetric', async () => {
  ...
}
// </if>
```

#### Disabling in GTest (Discouraged)

Disabling a C++ GTest suite using the `DISABLED_` prefix is discouraged. There
are often dozens of individual test cases under a single C++ GTest suite.
Disabling the whole suite when a single test is failing results in decreased
test coverage and a higher change that regressions could be introduced.

Disabling a Mocha test for a specific bot configuration (e.g. MSAN or ASAN)
isn't supported. If you encounter a failure, weigh the benefits of disabling
all tests for that bot configuration versus a specific test for all
configurations. If the configuration is only run for a single OS, you could
disable the test case for just that OS using `<if expr>` syntax described
above.

Here is how to disable a test in GTest if that is required:

```c++
// TODO(crbug.com/511960512): Flaky test.
IN_PROC_BROWSER_TEST_F(SidePanelBookmarksAppTest, DISABLED_General2) {
  SidePanelBookmarksTest::RunTest(
      "side_panel/bookmarks/power_bookmarks_app_test.js",
      "runMochaSuite('General Part2');");
}
```

A presubmit check enforced in `chrome/test/data/webui/` will trigger if
`DISABLED_` is added to C++ test files. This can be bypassed by adding
`SKIP_DISABLING_SUITES_CHECK: <reason>` to the CL description if disabling at the
C++ level is truly necessary (e.g., the entire test suite is flaky or failing).

### Reporting WebUI test results to LUCI

WebUI test results are automatically reported to LUCI. No special handling is
required of WebUI test authors.

A single GTest typically reports a single test result. A [new test reporting
mechanism was
added](https://chromium-review.googlesource.com/c/chromium/src/+/6358478) to
support reporting multiple results from a single GTest. Using this mechanism,
each Mocha JS test can have its own entry in LUCI (see screenshot). The
integration is accomplished by calling base::AddSubTestResult in
WebUIMochaBrowserTest.

![screenshot of WebUI JS test results in LUCI UI](images/luci_webui_js_test_results.png)

### Common errors

Tests that rely on focus, blur, or other input-related events need to be added
to the interactive_ui_tests build target rather than the browser_tests target.
browser_tests are run in parallel, and the window running the test will
gain/lose activation at unpredictable times.

Tests that touch production logic that depends on onFocus, onBlur or similar
events also needs to be added to interactive_ui_tests