commit | 91ddaf99938c502c3a287ca3fde9124d7ecea8bd | [log] [tgz] |
---|---|---|
author | Camillo Bruni <cbruni@chromium.org> | Mon Nov 28 16:37:39 2022 |
committer | Camillo Bruni <cbruni@chromium.org> | Mon Nov 28 16:37:39 2022 |
tree | 8a092630273347a9bd4a83740bd68d7364b6be79 | |
parent | 05773b9843818944901b7ca199fabaf9fa61d35a [diff] |
Improve documentation - Use shorter `cb` instead of `crossbench` command - Add more details for: - repeated --browser arguments, - inline --probe config, - separate --probe-config files. Change-Id: If8273c859b013a070d1ffcb470df98921b45668d Reviewed-on: https://chromium-review.googlesource.com/c/crossbench/+/4055960 Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Crossbench is a cross-browser/cross-benchmark runner to extract performance numbers.
Supported Browsers: Chrome/Chromium, Firefox, Safari and Edge.
Supported OS: macOS, linux and windows.
This project uses poetry deps and package scripts to setup the correct environment for testing and debugging.
pip3 install poetry
Install the necessary dependencies from lock file via poetry
poetry install
poetry run cb speedometer \ --browser=/path/to/chromium \ --stories=VanillaJS.* \ --probe=profiling \ --probe=v8.log
poetry run pytest
Run detailed test coverage:
poetry run pytest --cov=crossbench --cov-report=html
Run pytype type checker:
poetry run pytype -j auto crossbench
Crossbench supports running benchmarks on one or multiple browser configurations. The main implementation uses selenium for maximum system independence.
You can specify a browser with --browser=<name>
. You can repeat the --browser
argument to run multiple browser. If you need custom flags for multiple browsers use --browser-config
.
poetry run cb speedometer \ --browser=/path/to/chromium \ -- -- \ --browser-flag-foo \ --browser-flag-bar
For more complex scenarios you can use a browser.config.hjson file. It allows you to specify multiple browser and multiple flag configurations in a single file and produce performance numbers with a single invocation.
poetry run cb speedometer --browser-config=config.hjson
The example file lists and explains all configuration details.
Probes define a way to extract arbitrary (performance) numbers from a host or running browser. This can reach from running simple JS-snippets to extract page-specific numbers to system-wide profiling.
Multiple probes can be added with repeated --probe=XXX
options. You can use the describe
subcommand to list all probes:
poetry run cb describe probes
Some probes can be configured, either with inline json when using --probe
or in a separate --probe-config
hjson file. Use the describe
command to list all options.
poetry run cb describe probes v8.log poetry run cb speedometer --probe='v8.log{prof:true}'
For complex probe setups you can use --probe-config=<file>
. The example file lists and explains all configuration details. For the specific probe configuration properties consult the describe
command.
Use the describe
command to list all benchmark details:
poetry run cb describe benchmarks
Stories define sequences of browser interactions. This can be simply loading a URL and waiting for a given period of time, or in more complex scenarios, actively interact with a page and navigate multiple times.
Use --help
or describe to list all stories for a benchmark:
poetry run cb speedometer --help
Use --stories
to list individual comma-separated story names, or use a regular expression as filter.
poetry run cb speedometer \ --browser=/path/to/chromium \ --stories=VanillaJS.*