Python Testing Guide

This guide helps contributors write tests in the Selenium Python codebase.

Test Framework

  • Tests use pytest.
  • Test HTML pages live in common/src/web/.
  • pages fixture loads test pages via pages.load("pageName.html").
  • Assertions use standard pytest assert statements.
import pytest
from selenium.webdriver.common.by import By

def test_element_is_displayed(driver, pages):
    pages.load("javascriptPage.html")

    element = driver.find_element(By.ID, "displayed")
    assert element.is_displayed() is True

@pytest.mark.xfail_safari(reason="Safari doesn't support this")
def test_something_safari_fails(driver, pages):
    # Expected to fail on Safari
    pass

Running Tests

Bazel creates test targets for each browser. Tests run in parallel by default.

bazel test //py/...  # All tests
bazel test //py:unit  # Unit tests (no browser)
bazel test //py:test-chrome  # Chrome browser tests
bazel test //py:test-firefox  # Firefox browser tests
bazel test //py:test-chrome-common  # Common (cross-browser) tests with Chrome

# A single test file with Chrome (target = test/<path>/<file>-<browser>[-variant]).
# Discover exact names with: bazel query //py:all | grep window_tests
bazel test //py:test/selenium/webdriver/common/window_tests-chrome

# With BiDi protocol
bazel test //py:test-chrome-bidi

# Against a Grid server (chrome and firefox only). The suite starts its own
# Selenium standalone server and talks to it with webdriver.Remote.
bazel test //py:test-chrome-remote
bazel test //py:test-chrome-grid     # only the Grid-specific tests
bazel test //py:test-remote          # every remote suite

# Test filters
bazel test //py/... --test_tag_filters=chrome

# Additional arguments
bazel test //py/... --flaky_test_attempts=3
bazel test //py/... --test_output=all
bazel test //py/... --test_output=streamed  # Live output for debugging
bazel test //py:test-chrome --headless

# Run a specific test in a test file
bazel test //py:test/selenium/webdriver/common/bidi/browsing_context_tests-chrome-bidi \
  --test_arg=-k \
  --test_arg=test_get_tree_with_child \

# View all targets
bazel query //py/...

By default on Linux and macOS, Bazel uses pinned versions of the browser and driver, so tests do not depend on what the machine has installed. To have Selenium Manager handle this instead, pass --pin_browsers=false. Selenium Manager uses an installed browser or a driver on PATH if present; to ensure the latest stable version, pass --test_env=SE_FORCE_BROWSER_DOWNLOAD=true --test_env=SE_SKIP_DRIVER_IN_PATH=true.

Our CI runs the x64 Linux targets on Remote Build Execution (RBE), which only has network access for the Rust and Selenium Manager tests. We use GitHub Actions runners only for what the RBE results cannot tell us, such as OS-specific behavior and Selenium Manager on other platforms.

Running Tests Without Bazel (using pytest)

You can run tests directly with pytest after setting up the development environment.

Setup

Install the required dependencies and copy the generated files into your local source tree:

./go py:local_dev

Running with pytest

# Run all tests in a directory
pytest py/test/selenium/webdriver/chrome/ --driver chrome

# Run a specific test file
pytest py/test/selenium/webdriver/common/window_tests.py

# Run a specific test function
pytest py/test/selenium/webdriver/common/window_tests.py::test_should_get_the_size_of_the_current_window

# With pytest options
pytest py/test/selenium/webdriver/chrome/ --driver chrome --headless -v

Note: For running BiDi tests, use the --bidi flag.

To run against a Grid server, add --remote. It starts a Selenium standalone server and runs the tests through webdriver.Remote, so it needs the Grid jar built first (bazel build //java/src/org/openqa/selenium/grid:selenium_server_deploy.jar).

A BiDi test file with Grid-specific behavior can be listed in REMOTE_BIDI_TESTS in py/BUILD.bazel; it then also runs with --bidi --remote as part of test-<browser>-remote.

Skipping Tests

Skips use pytest markers; each accepts optional reason and run parameters (run=False skips the test entirely instead of expecting a failure).

MarkerWhen to Use
@pytest.mark.xfail_chromeTest expected to fail on Chrome
@pytest.mark.xfail_firefoxTest expected to fail on Firefox
@pytest.mark.xfail_safariTest expected to fail on Safari
@pytest.mark.xfail_edgeTest expected to fail on Edge
@pytest.mark.xfail_ieTest expected to fail on IE
@pytest.mark.xfail_remoteTest expected to fail with Remote WebDriver
@pytest.mark.xfail_chrome(reason="Not implemented yet")
@pytest.mark.xfail_firefox(reason="https://bugzilla.mozilla.org/123")
def test_something(driver, pages):
    pass

@pytest.mark.xfail_safari(run=False)  # Skip entirely instead of xfail
def test_skip_safari(driver, pages):
    pass

Driver Lifecycle

MarkerWhen to Use
@pytest.mark.no_driver_after_testTeardown driver after test
@pytest.mark.needs_fresh_driverRestart driver for test isolation

Fixtures

We make use of pytest fixtures to simplify test setup/teardown. There are several built-in pytest fixtures, and many of our own internal fixtures. If a fixture is specific to a module, you will find it defined within the test file that uses it. If it is shared among several modules, you will find the main fixtures in conftest.py:

FixtureDescription
driverWebDriver instance, auto-parametrized by browser
pagesLoad test pages: pages.load("page.html") or pages.url("page.html")
webserverTest HTTP server reference
clean_driverFresh driver without parametrization
clean_optionsFresh browser options instance
headlessWhether the browser was started headless, for tests asserting on behavior a headless browser does not model (e.g. window focus)

Test Organization

py/test/
├── unit/                    # Unit tests (no browser)
│   └── selenium/webdriver/
└── selenium/webdriver/      # Integration tests
    ├── common/              # Cross-browser tests
    ├── chrome/
    ├── firefox/
    ├── safari/
    └── remote/

Test files end in _tests.py (e.g., visibility_tests.py).

Build Files

  • Adding tests shouldn't require Bazel changes—files matching *_tests.py are picked up automatically.
  • Make sure the test file is in a directory covered by existing py_test_suite targets.