commit | dd2888041375720ca0a15df13594e27385ea28ae | [log] [tgz] |
---|---|---|
author | Peter Wen <wnwen@chromium.org> | Wed Jul 17 21:04:19 2024 |
committer | Chromium LUCI CQ <chromium-scoped@luci-project-accounts.iam.gserviceaccount.com> | Wed Jul 17 21:04:19 2024 |
tree | b3b17650b1cbf9923a055bc20a2f3b1853043245 | |
parent | 7db7ed89cff6698e553860ba280593ee8f38b8cb [diff] |
pgo: Run per story benchmarks on android Each telemetry benchmark has one or more stories associated with it. PGO on Android works by running an entire benchmark and all its associated stories and then copying over the files in /data/data/<package_name>/cache/pgo_profiles/*.profraw and merging them into a .profdata file. The .profdata files from various benchmark runs are then merged into a final profile.profdata file which is used in PGO-optimized builds. The telemetry platform code for Android starts the Android browser at the beginning of each story and sets up it profile. During profile setup all the files (except those in lib/) in /data/data/<package_name>/* are deleted. After each story run completes, the PGO profile data associated with that story run are saved to /data/data/<package_name>/cache/pgo_profiles/*.profraw files. This means that any subsequent stories start by clearing the PGO profile data collected during the preceding story run. Running any benchmark with more than one story results in only the PGO profile data from the last story in that benchmark being kept at the end of the benchmark. In order to obtain more comprehensive PGO profile coverage, this CL (on Android only) splits each benchmark by its stories, so that each story gets its own explicit benchmark run, independent from the others. This also has the benefit of enabling per-story repeats which in turn greatly reduces flakiness associated with any one of the stories failing causing the entire benchmark to fail. In principle this has the same effect for a multi-story benchmark as it does for a multi-benchmark suite. See https://crrev.com/c/5680092 for an example of improving from 13% success rate to 96% success rate just by adding repeats. This CL also addresses flakiness in PGO profile generation and merging on the LLVM side. Either profile merging would fail explicitly with an error code, or it would succeed but print out a warning to stderr that it produced an invalid profile. This CL ensures that the benchmark or story is repeated in both failure modes, as otherwise the failure propagates to the final merge of up to 16 *.profdata files, and that final merge will fail if any of the 16 *.profdata files are invalid individually. This CL also adds a passthrough for the -q flag to run_benchmark as that greatly reduces the noisiness of run_benchmark for successful runs. This way devs can focus on failure cases instead of length output from successful runs. This CL extends handling for Temporal PGO to all `llvm-profdata merge` commands. This codepath will be tested and extended in a later CL. The new --print-only-runnable flag for run_benchmark is used to collect the list of stories to run individually. Bug: 352333007 Change-Id: Ia5f839bc79bcc534c4df38b4366550bf00129229 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5718532 Auto-Submit: Peter Wen <wnwen@chromium.org> Commit-Queue: Peter Wen <wnwen@chromium.org> Reviewed-by: Andrew Grieve <agrieve@chromium.org> Cr-Commit-Position: refs/heads/main@{#1329121}
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all users to experience the web.
The project's web site is https://www.chromium.org.
To check out the source code locally, don't use git clone
! Instead, follow the instructions on how to get the code.
Documentation in the source is rooted in docs/README.md.
Learn how to Get Around the Chromium Source Code Directory Structure.
For historical reasons, there are some small top level directories. Now the guidance is that new top level directories are for product (e.g. Chrome, Android WebView, Ash). Even if these products have multiple executables, the code should be in subdirectories of the product.
If you found a bug, please file it at https://crbug.com/new.