Reland "Support remote Posix paths from Windows"

This relands commit b7b58eb412e462a01d3764aafe59840a090e5759.

Use the host-platform when checking for an existing driver binary.

Original change's description:
> Support remote Posix paths from Windows
> When using an adb device under Windows we get broken paths for the
> remote sh commands that are generate since the paths are encoded
> for the runner platform (windows) and not the targeted remote
> platform (adb).
> Add explicit type aliases for paths:
> - LocalPath: Paths that are only avialble on the host running platform
> - RemotePath: Paths that can be on either a remote or a local machine
> Note that LocalPath (pathlib.Path) inherits from
> RemotePath (pathlib.PurePath)
> In order to get the existing pyfakefs tests working we need to access
> the path aliases via an indirect load, otherwise pyfakefs is not able
> to properly path the Path classes.
> Probes now need to be more explicit when using local vs. remote Paths
> which requires more indirections via the Platform path helper methods
> for many path operations.
> Bug: 335309993
> Change-Id: Ife95fb6af5d9e90172447f64ada3a29feb697def
> Reviewed-on:
> Reviewed-by: Patrick Thier <>
> Commit-Queue: Camillo Bruni <>

Bug: 335309993
Change-Id: I51ff9e8cf27689202d8acd5b0e5667b818a13a21
Commit-Queue: Camillo Bruni <>
Reviewed-by: Patrick Thier <>
84 files changed
tree: 6b94e2a6797a67d10e8ad8d2793d14e9319a66e2
  1. binary_cache/
  2. chrome-extension-replay/
  3. config/
  4. crossbench/
  5. docs/
  6. tests/
  7. .coveragerc
  8. .gitattributes
  9. .gitignore
  10. .pylintrc
  11. .style.yapf
  12. .vpython3
  15. mypy.ini
  16. OWNERS
  17. poetry.lock
  19. pyproject.toml
  20. pytest.ini
  21. README.chromium


Crossbench is a cross-browser/cross-benchmark runner to extract performance numbers.

Mailing list:

Issues/Bugs: Tests > CrossBench

Supported Browsers: Chrome/Chromium, Firefox, Safari and Edge.

Supported OS: macOS, Android, linux and windows.

Basic usage:

Chromium Devs (with a full chromium checkout)

Use the ./ script directly to run benchmarks (requires chrome's vpython3)

Standalone installation

  • Use pip install crossbench,
  • or use the “poetry” package manager, see the development section.

Run the latest speedometer benchmark 20 times with the system default browser (chrome-stable):

# Run chrome-stable by default:
./ speedometer --repeat=20

# Compare chrome browser versions and a local chrome build on jetstream:
./ jetstream --browser=chrome-stable --browser=chrome-m90 --browser=$PATH

Profile individual line items (with pprof on linux):

./ speedometer --probe='profiling' --separate

Use a custom chrome build and only run a subset of the stories:

./ speedometer --browser=$PATH --probe='profiling' --story='jQuery.*'

Profile a website for 17 seconds on Chrome M100 (auto-downloading on macOS and linux):

./ loading --browser=chrome-m100 --probe='profiling',17s

Collect perfetto data from loading separate websites on multiple attached android devices using the device ID or unique device names (see adb devices -l):

./ loading --probe-config=./config/probe/perfetto.probe.config.example.hjson \
    --browser='Pixel_4:chrome-stable' --browser='AA00BB11:chrome-stable' \
    --parallel=platform \
    --url=,15s,,15s  --separate

Main Components


Crossbench supports running benchmarks on one or multiple browser configurations. The main implementation uses selenium for maximum system independence.

You can specify a browser with --browser=<name>. You can repeat the --browser argument to run multiple browser. If you need custom flags for multiple browsers use --browser-config (or pass simple flags after -- to the browser).

./ speedometer --browser=$BROWSER -- --enable-field-trial-config

Browser Config File

For more complex scenarios you can use a browser.config.hjson file. It allows you to specify multiple browser and multiple flag configurations in a single file and produce performance numbers with a single invocation.

./ speedometer --browser-config=config.hjson

The example file lists and explains all configuration details.


Probes define a way to extract arbitrary (performance) numbers from a host or running browser. This can reach from running simple JS-snippets to extract page-specific numbers to system-wide profiling.

Multiple probes can be added with repeated --probe='XXX' options. You can use the describe probes subcommand to list all probes:

# List all probes:
./ describe probes

# List help for an individual probe:
./ describe probe v8.log

Inline Probe Config

Some probes can be configured, either with inline json when using --probe or in a separate --probe-config hjson file. Use the describe command to list all options.

# Get probe config details:
./ describe probe v8.log

# Use inline hjson to configure a probe:
./ speedometer --probe='v8.log:{prof:true}'

Probe Config File

For complex probe setups you can use --probe-config=<file>. The example file lists and explains all configuration details. For the specific probe configuration properties consult the describe command.


Use the describe command to list all benchmark details:

# List all benchmark info:
./ describe benchmarks

# List an individual benchmark info:
./ describe benchmark speedometer_3.0

# List a benchmark's command line options:
./ speedometer_3.0 --help


Stories define sequences of browser interactions. This can be simply loading a URL and waiting for a given period of time, or in more complex scenarios, actively interact with a page and navigate multiple times.

Use --help or describe to list all stories for a benchmark:

./ speedometer --help

Use --stories to list individual story names, or use regular expression as filter.

./ speedometer --browser=$BROWSER --stories='.*Angular.*'



This project uses poetry deps and package scripts to setup the correct environment for testing and debugging.

# a) On debian:
sudo apt-get install python3.10 python3-poetry
# b) With python 3.8 to 3.10 installed already:
pip3 install poetry

Check that you have poetry on your path and make sure you have the right $PATH settings.

poetry --help || echo "Please update your \$PATH to include poetry bin location";
# Depending on your setup, add one of the following to your $PATH:
echo "`python3 -m site --user-base`/bin";
python3 -c "import sysconfig; print(sysconfig.get_path('scripts'))";

Install the necessary dependencies from the lock file using poetry:

# Select the python version you want to use (3.8 to 3.10):
poetry env use 3.10
poetry install

# For python 3.11 you have to skip pytype support:
poetry env use 3.11
poetry install --without=dev-pytype


For local development / non-chromium installation you should use poetry run cb ... instead of ./ ....

Side-note, beware that poetry eats up an empty --:

# With
./ speedometer ... -- --custom-chrome-flag ...
# With poetry:
poetry run cb speedometer ... -- -- --custom-chrome-flag ...


poetry run pytest

Run detailed test coverage:

poetry run pytest --cov=crossbench --cov-report=html

Run pytype type checker:

poetry run pytype -j auto .