The Telemetry loading benchmarks measure Chrome's loading performance under different network and caching conditions.
There are currently three loading benchmarks:
loading.desktop: A desktop-only benchmark in which each test case measures performance of loading a real world website (e.g: facebook, cnn, alibaba..).
loading.mobile: A mobile-only benchmark that parallels
loading.cluster_telemetry: A cluster Telemetry benchmark that uses the corpus of top 10 thousands URLs from Alexa. Unlike the other two loading benchmarks which are run continuously on the perf waterfall, this benchmark is triggered on-demand only.
If you're just trying to gauge whether your change has caused a loading regression, you can either run
loading.mobile through perf try job or you can run
loading.cluster_telemetry through Cluster Telemetry service (Cluster Telemetry is for Googler only).
For more in-depth analysis and shorter cycle times, it can be helpful to run the tests locally.
Once you've done this, you can start the Telemetry benchmark with:
./tools/perf/run_benchmark <benchmark_name> --browser=<browser>
benchmark_name can be
The loading test cases are divided into groups based on their network traffic settings and cache conditions.
All available traffic settings can be found in traffic_setting.py
All available caching conditions can be found in cache_temperature.py
Test cases of
loading.mobile are named with their corresponding settings. For example,
DevOpera_cold_3g test case loads
https://dev.opera.com/ with cold cache and 3G network setting.
In additions, the pages are also tagged with labels describing their content. e.g: ‘global’, ‘pwa’,...
To run only pages of one tags, add
--story-tag-filter=<tag name> flag to the run benchmark command.
The benchmark output several different loading metrics. The keys one are:
Besides those key metrics, there are also breakdown metrics that are meant to to make debugging regressions simpler. These metrics are updated often, for most up to date information, you can email firstname.lastname@example.org or email@example.com (Googlers only).
For example, to add a new case of loading
https://en.wikipedia.org/wiki/Cats_and_the_Internet on 2G and 3G networks with warm cache to
news group to
loading.desktop benchmark, you would write:
self.AddStories( tags=['news'], urls=[('https://en.wikipedia.org/wiki/Cats_and_the_Internet', 'wiki_cats')], cache_temperatures=[cache_temperature_module.WARM], traffic_settings=[traffic_setting_module.2G, traffic_setting_module.3G])
After adding the new page, record it and upload the page archive to cloud storage with:
$ ./tools/perf/record_wpr loading_desktop --browser=system \ --story-filter=wiki_cats --upload
If the extra story was added to
loading_desktop in the command above with