Telemetry is the performance testing framework used by Chrome. It allows you to perform arbitrary actions on a set of web pages (or any android application!) and report metrics about it. The framework abstracts:
Telemetry is designed for measuring performance rather than checking correctness. If you want to check for correctness, browser tests are your friend.
If you are a Chromium developer looking to add a new Telemetry benchmark to src/tools/perf/
, please make sure to read our Benchmark Policy first.
Telemetry provides two major functionality groups: those that provide test automation, and those that provide the capability to collect data.
The test automation facilities of Telemetry provide Python wrappers for a number of different system concepts.
adb shell
.The Telemetry framework lives in src/third_party/catapult/telemetry/
and performance benchmarks that use Telemetry live in src/tools/perf/
.
Telemetry offers a framework for collecting metrics that quantify the performance of automated actions in terms of benchmarks, measurements, and story sets.
run_benchmark
in tools/perf
.StoryTest
in the code) is responsible for setting up and tearing down the testing platform, and for collecting metrics that quantify the application scenario under test.StoryTest
(see “Timeline Based Measurement” below). If you think you do, please talk to us.StoryTest
that runs all available timeline-based metrics, and benchmarks that use it can filter relevant results.If you have questions, please email telemetry@chromium.org.
You can keep up with Telemetry related discussions by joining the telemetry group.
The recordings are not included in the Chromium source tree. If you are a Google partner, run gsutil config
to authenticate, then try running the test again. If you don't have gsutil
installed on your machine, you can find it in build/third_party/gsutil/gsutil
.
If you are not a Google partner, you can run on live sites with --use-live-sites` or record your own story set archive.
Your forwarder binary may be outdated. If you have built the forwarder in src/out that one will be used. if there isn't anything there Telemetry will default to downloading a pre-built binary. Try re-building the forwarder, or alternatively wiping the contents of src/out/
and running run_benchmark
, which should download the latest binary.
Make sure that your keychain is correctly configured.