| # Ruby Testing Guide | |
| This guide helps contributors write tests, maintain code style, and generate documentation for the Selenium Ruby bindings. | |
| ## Test Framework | |
| * Tests use RSpec. | |
| * Test HTML files live in `common/src/web/`. | |
| * **Helper methods:** `driver`, `wait`, `short_wait`, `long_wait`, `url_for`. | |
| ### Example Spec | |
| ```ruby | |
| module Selenium | |
| module WebDriver | |
| describe Element do | |
| it 'returns element text' do | |
| driver.get(url_for('simpleTest.html')) | |
| expect(driver.find_element(id: 'foo').text).to eq('expected') | |
| end | |
| it 'clicks element', pending_if: {browser: :safari, reason: 'Safari bug'} do | |
| # Pending on Safari | |
| end | |
| end | |
| end | |
| end | |
| ``` | |
| ## Running Tests | |
| Bazel generates a target per spec file for each supported browser and variant. | |
| ```shell | |
| bazel test //rb/spec/... # All tests | |
| bazel test //rb/spec/unit:unit # All unit specs (single target) | |
| bazel test //rb/spec/integration/... --test_tag_filters=chrome # Every Chrome variant | |
| bazel test //rb/spec/integration/... --test_tag_filters=firefox-local # Local Firefox only | |
| bazel test //rb/spec/integration/... --test_tag_filters=firefox-remote # Remote (Grid) Firefox only | |
| bazel test //rb/spec/integration/... --test_tag_filters=bidi # BiDi-mode across supporting browsers | |
| # Viewing Output | |
| bazel test //rb/... --test_output=all # See console output at the end | |
| bazel test //rb/... --test_output=streamed # See output in real-time (no parallel execution) | |
| ``` | |
| A browser tag (e.g. `firefox`) matches every variant of that browser (local, remote, beta, bidi); add a | |
| `-local`, `-remote`, or `-bidi` suffix to narrow it. Integration targets are named after the spec file with | |
| `_spec.rb` removed plus a browser-variant suffix (e.g. `driver-chrome`, `driver-chrome-remote`, | |
| `driver-chrome-beta-bidi`), so you can run a single spec directly: | |
| ```shell | |
| bazel test //rb/spec/integration/selenium/webdriver:driver-chrome | |
| ``` | |
| By default on Linux and macOS, Bazel uses pinned versions of the browser and driver, so tests do not | |
| depend on what the machine has installed. To have Selenium Manager handle this instead, pass | |
| `--pin_browsers=false`. Selenium Manager uses an installed browser or a driver on `PATH` if present; | |
| to ensure the latest stable version, pass | |
| `--test_env=SE_FORCE_BROWSER_DOWNLOAD=true --test_env=SE_SKIP_DRIVER_IN_PATH=true`. | |
| Our CI runs the x64 Linux targets on Remote Build Execution (RBE), which only has network access for | |
| the Rust and Selenium Manager tests. We use GitHub Actions runners only for what the RBE results | |
| cannot tell us, such as OS-specific behavior and Selenium Manager on other platforms. | |
| ### Using Rake | |
| The `rb/Rakefile` provides shortcuts for common tasks: | |
| ```shell | |
| rake update # Setup everything to run tests in RubyMine | |
| rake unit # Run unit tests | |
| rake spec # Run all integration tests in Chrome | |
| ``` | |
| ## Local Development Setup | |
| Before running tests, navigate to the `rb/` directory and install the required dependencies. A recent Bundler (2.4 or newer) is recommended. | |
| Bazel generates `lib/selenium/webdriver/version.rb` from the root `version.bzl`, so run `./go rb:local_dev` from the parent `selenium` directory first to write the copy Bundler needs. | |
| ```shell | |
| cd rb | |
| gem install bundler -v '~> 2.4' | |
| bundle install | |
| ``` | |
| **Note:** Local development still requires running Bazel to generate the atoms and devtools code. You can generate these artifacts by running: | |
| ```shell | |
| bundle exec rake update | |
| ``` | |
| Or from the parent `selenium` directory: | |
| ```shell | |
| ./go rb:local_dev | |
| ``` | |
| To test against a specific Ruby version, set it in `rb/.ruby-version` (Bazel's hermetic Ruby | |
| toolchain reads this file). It is checked in and bumped repo-wide, so treat a local edit as a | |
| temporary override and don't commit it: | |
| ```shell | |
| echo '<X.Y.Z>' > rb/.ruby-version | |
| ``` | |
| ### RubyMine IDE Setup | |
| If you want to use [RubyMine](https://www.jetbrains.com/ruby/) for development, you can configure it to use Bazel artifacts: | |
| 1. Open `rb/` as a main project directory. | |
| 2. Run `bundle exec rake update` as necessary to create up-to-date artifacts. If this does not work, run `./go rb:local_dev` from the `selenium` (parent) directory. | |
| 3. In <kbd>Settings / Languages & Frameworks / Ruby SDK and Gems</kbd> add new <kbd>Interpreter</kbd> pointing to `../bazel-selenium/external/rules_ruby_dist/dist/bin/ruby`. | |
| 4. You should now be able to run and debug any spec. It uses Chrome by default, but you can alter it using environment variables specified in the [Environment Variables](#environment-variables) section. | |
| ## Skipping Tests | |
| Guards are RSpec metadata on `describe`, `context`, or `it` blocks (all enclosing guards | |
| combine). Use one of the five keywords, and always include a `reason:` (a String, or an issue | |
| number): | |
| | Guard | Effect | | |
| | --- | --- | | |
| | `skip_if` | Skip when the config matches. | | |
| | `skip_unless` | Skip unless the config matches (run only when it matches). | | |
| | `pending_if` | Expect failure (pending) when the config matches. | | |
| | `pending_unless` | Expect failure (pending) unless the config matches. | | |
| | `flaky` | Like `skip_if`, reserved for unreliable/intermittent tests. | | |
| `except`, `only`, `exclude`, and `exclusive` are older aliases of `pending_if`, `pending_unless`, | |
| `skip_if`, and `skip_unless` respectively. | |
| **Matching:** within one Hash every pair must match (AND); an Array value for a key matches any | |
| listed value (OR). For an Array of Hashes, `skip_if`/`pending_if` trigger if **any** Hash matches, | |
| while `skip_unless`/`pending_unless` apply unless **every** Hash matches. | |
| ### Guard Conditions | |
| Conditions are registered in [`spec/integration/selenium/webdriver/spec_helper.rb`](spec/integration/selenium/webdriver/spec_helper.rb). | |
| | Condition | Values | | |
| | --- | --- | | |
| | `browser` | `:chrome`, `:firefox`, `:edge`, `:safari`, `:safari_preview`, `:ie` | | |
| | `browser_family` | `:chromium` (Chrome/Edge), `:safari` (Safari/Safari Preview), otherwise the `browser` value (e.g. `:firefox`) | | |
| | `driver` | `:remote` | | |
| | `platform` | `:linux`, `:macosx`, `:windows` | | |
| | `headless` | `true`, `false` | | |
| | `bidi` | `true`, `false` | | |
| | `version` | Browser version string, e.g. `'stable'` (from `WD_BROWSER_VERSION`) | | |
| | `rbe` | `true`, `false` (running on Remote Build Execution) | | |
| | `ci` | `:github`, `:jenkins`, `:appveyor` | | |
| Prefer `browser_family` over listing every member browser when a guard applies to a whole engine | |
| (e.g. `browser_family: :chromium` instead of `browser: %i[chrome edge]`). Use the exact `browser` | |
| condition when a guard is specific to one channel, such as `browser: :safari_preview` or | |
| `browser: :chrome, version: 'beta'`. | |
| ### Guard Examples | |
| ```ruby | |
| # Pending on Safari | |
| it 'does something', pending_if: {browser: :safari, reason: 'Safari bug'} do | |
| end | |
| # Pending everywhere except Chrome and Firefox | |
| it 'does something', pending_unless: {browser: %i[chrome firefox], reason: 'Only implemented in Chrome/Firefox'} do | |
| end | |
| # Pending on any Chromium-based browser (Chrome and Edge) | |
| it 'does something', pending_if: {browser_family: :chromium, reason: 'Chromium bug'} do | |
| end | |
| # Skip on the stable Firefox channel | |
| it 'does something', skip_if: {browser: :firefox, version: 'stable', reason: 'https://bugzil.la/123'} do | |
| end | |
| # Skip when running remotely on RBE | |
| it 'does something', skip_if: {driver: :remote, rbe: true, reason: 'Cannot start 2+ drivers at once.'} do | |
| end | |
| # Known-flaky on GitHub Actions | |
| it 'does something', flaky: {browser: :safari, ci: :github, reason: 'unreliable with downloads'} do | |
| end | |
| # Array of Hashes — skip if Firefox OR macOS (reason may be an issue number) | |
| it 'something', skip_if: [ | |
| {browser: :firefox, reason: 1234}, | |
| {platform: :macosx, reason: 5678} | |
| ] do | |
| end | |
| ``` | |
| ### Exception-Aware Pending | |
| `pending_if`/`except` also accept `exception: {class:, message:}` (`message:` optional — a Regexp matches | |
| as a pattern, a String matches exactly, like `raise_error`). The example is marked pending only when it | |
| fails with that exception; a wrong exception, `invalid argument`, assertion failure, or timeout still fails. | |
| ```ruby | |
| it 'does something', pending_if: {browser: :firefox, | |
| exception: {class: Selenium::WebDriver::Error::WebDriverError, | |
| message: /\Aunknown command:/}, | |
| reason: 'Firefox does not implement this command'} do | |
| end | |
| ``` | |
| ## Helpers | |
| From `spec_support/helpers.rb`: | |
| | Helper | Description | | |
| | --- | --- | | |
| | `driver` | Current WebDriver instance. | | |
| | `reset_driver!(...)` | Reset driver with new options. | | |
| | `url_for(filename)` | Get test page URL (from `common/src/web`). | | |
| | `wait` / `short_wait` / `long_wait` | Wait instances (10s, 3s, 30s). | | |
| | `wait_for_element(locator)` | Wait for element to appear. | | |
| | `wait_for_alert` | Wait for alert presence. | | |
| ## Asserting Log Output | |
| Every `WebDriver.logger` call should include an `id:` symbol (e.g. `logger.warn(msg, id: :safari_bidi)`). | |
| To assert on logging content (and hide it from test logs), do not stub the logger, instead use one of | |
| the [custom matchers](spec/rspec_matchers.rb): `have_error`, `have_warning`, `have_info`, and | |
| `have_deprecated`. | |
| ```ruby | |
| expect { SeleniumManager.binary }.to have_info(:selenium_manager) # id was logged, at info level | |
| expect { save_screenshot(png_path) }.not_to have_warning(:screenshot) # id was not logged | |
| ``` | |
| The match is the exact set of ids at that severity — an unexpected entry fails rather than slipping by | |
| — so assert several entries by passing the full set, e.g. `have_warning(%i[general specific])`. | |
| The id is provided so you don't have to assert on specific text, but if the message comes from an | |
| external source, you can assert on the contents as well: | |
| ```ruby | |
| expect { navigate }.to have_error(:ws, /This is fine!/) | |
| ``` | |
| Deprecations (`logger.deprecate`) are asserted with `have_deprecated`: | |
| ```ruby | |
| WebDriver.logger.deprecate('Old thing', 'New thing', id: :old_thing) # lib | |
| expect { call_old_thing }.to have_deprecated(:old_thing) # spec | |
| ``` | |
| ## Debugging | |
| ### Interactive REPL | |
| Instead of using `irb`, you can create an interactive REPL with all gems loaded using: | |
| ```shell | |
| bazel run //rb:console | |
| ``` | |
| ### Debugging with the `debug` gem | |
| Use the [`debug`](https://github.com/ruby/debug) gem to set breakpoints inside specs or library code: | |
| 1. Add `binding.break` where you want the debugger to stop. | |
| 2. Run the test with the `ruby_debug` configuration: | |
| ```shell | |
| bazel test --config ruby_debug <test> | |
| ``` | |
| 3. In a separate terminal, attach to the running debugger: | |
| ```shell | |
| bazel-selenium/external/bundle/bin/rdbg -A | |
| ``` | |
| ## Environment Variables | |
| ### BiDi Testing | |
| BiDi-enabled targets are generated with a `-bidi` suffix for the browsers that support it | |
| (they set `WEBDRIVER_BIDI=true` internally). Run them directly or filter by the `bidi` tag: | |
| ```shell | |
| # Run a spec's BiDi variant | |
| bazel test //rb/spec/integration/selenium/webdriver:driver-chrome-bidi | |
| # Run every BiDi variant across browsers that support it | |
| bazel test //rb/spec/integration/... --test_tag_filters=bidi | |
| ``` | |
| To enable BiDi on a target that is not a `-bidi` variant, pass the variable through `--test_env`: | |
| ```shell | |
| bazel test //rb/spec/integration/... --test_tag_filters=chrome --test_env=WEBDRIVER_BIDI=true | |
| ``` | |
| ### Common Variables | |
| Bazel sets the variables below automatically for each generated target. To override one (or set it | |
| on a target that does not), pass it with `--test_env`, e.g. | |
| `bazel test //rb/spec/integration/... --test_tag_filters=chrome --test_env=HEADLESS=true`. | |
| | Variable | Purpose | | |
| | --- | --- | | |
| | `WEBDRIVER_BIDI` | Enable the BiDi protocol (set by the `-bidi` targets). | | |
| | `HEADLESS` | Run Chrome, Edge, and Firefox in headless mode. | | |
| | `WD_SPEC_DRIVER` | Driver to test; a browser name or `remote` (set by Bazel). | | |
| | `WD_REMOTE_BROWSER` | When `WD_SPEC_DRIVER` is `remote`, the browser to test (set by Bazel). | | |
| | `WD_REMOTE_URL` | URL of an already-running server to use for remote tests. | | |
| | `DOWNLOAD_SERVER` | When `WD_REMOTE_URL` is unset, download and use the most recently released server for remote tests. | | |
| | `DISABLE_BUILD_CHECK` | For Chrome and Edge, ignore driver/browser version mismatches (allows testing Canary builds). | | |
| | `CHROME_BINARY` / `CHROMEDRIVER_BINARY` | Paths to a specific Chrome browser / ChromeDriver. | | |
| | `EDGE_BINARY` / `MSEDGEDRIVER_BINARY` | Paths to a specific Edge browser / msedgedriver. | | |
| | `FIREFOX_BINARY` / `GECKODRIVER_BINARY` | Paths to a specific Firefox browser / GeckoDriver. | | |
| ## Code Style & Linting | |
| Selenium enforces strict code style using **Rubocop**. CI will fail if linting errors are present. | |
| Configuration is defined in [`.rubocop.yml`](.rubocop.yml). **Prefer updating the configuration file over using in-file rubocop guards** (like `# rubocop:disable`) to maintain consistency across the codebase. | |
| ```shell | |
| # Check code style | |
| bundle exec rubocop | |
| # Auto-correct simple offenses | |
| bundle exec rubocop -A | |
| ``` | |
| ## Type Signatures with Steep | |
| Selenium Ruby uses **Steep** for gradual type checking with RBS (Ruby Signature) files. When you create a new class or modify existing classes, add or update the corresponding `.rbs` file in `sig/`, mirroring the directory structure of `lib/`. | |
| ```shell | |
| # Type check all files | |
| bundle exec steep check | |
| # Type check specific files | |
| bundle exec steep check lib/selenium/webdriver/my_class.rb | |
| ``` | |
| **Tips:** start with `untyped` for complex types and refine over time; prefer concrete types | |
| (`String`, `Integer`) where possible; use generics for collections (`Array[String]`, | |
| `Hash[Symbol, String]`). CI runs Steep, so ensure signatures are correct before submitting a PR. | |
| ## Documentation | |
| We use **YARD** for inline documentation. Ensure your changes are documented and generate valid HTML. | |
| ```shell | |
| # Generate documentation | |
| bundle exec yard doc | |
| # Run a local documentation server (view at http://localhost:8808) | |
| bundle exec yard server --reload | |
| ``` | |
| ## Test Organization | |
| ```text | |
| rb/spec/ | |
| ├── unit/ # Unit tests (no browser) | |
| │ └── selenium/webdriver/ | |
| └── integration/ # Integration tests | |
| └── selenium/webdriver/ | |
| ├── chrome/ | |
| ├── firefox/ | |
| ├── safari/ | |
| ├── bidi/ | |
| └── spec_support/ # Test helpers | |
| ``` | |
| Test files must end in `_spec.rb` (e.g., `driver_spec.rb`). | |
| ## Build Files | |
| * Adding tests shouldn't require Bazel changes—`rb_integration_test` uses glob patterns. | |
| * Make sure `*_spec.rb` files are in a directory with a `BUILD.bazel` containing `rb_integration_test`. |