We distribute prebuilt packages of LLVM binaries, including clang and lld, that all developers and bots pull at
gclient runhooks time. These binaries are just regular LLVM binaries built at a fixed upstream revision. This document describes how to build a package at a newer revision and update Chromium to it. An archive of all packages built so far is at https://is.gd/chromeclang
Check that https://ci.chromium.org/p/chromium/g/chromium.clang/console looks reasonably green.
Sync your Chromium tree to the latest revision to pick up any plugin changes
python tools/clang/scripts/upload_revision.py NNNN with the target LLVM SVN revision number. This creates a roll CL on a new branch, uploads it and starts tryjobs that build the compiler binaries into a staging bucket on Google Cloud Storage (GCS).
If the clang upload try bots succeed, copy the binaries from the staging bucket to the production one. For example:
$ export rev=123456-1 $ for x in Linux_x64 Mac Win ; do \ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/$x/clang-$rev.tgz \ gs://chromium-browser-clang/$x/clang-$rev.tgz ; \ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/$x/llvmobjdump-$rev.tgz \ gs://chromium-browser-clang/$x/llvmobjdump-$rev.tgz ; \ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/$x/llvmcfiverify-$rev.tgz \ gs://chromium-browser-clang/$x/llvmcfiverify-$rev.tgz ; \ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/$x/translation_unit-$rev.tgz \ gs://chromium-browser-clang/$x/translation_unit-$rev.tgz ; \ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/$x/llvm-code-coverage-$rev.tgz \ gs://chromium-browser-clang/$x/llvm-code-coverage-$rev.tgz ; \ done $ gsutil.py cp -n -a public-read gs://chromium-browser-clang-staging/Mac/lld-$rev.tgz \ gs://chromium-browser-clang/Mac/lld-$rev.tgz
Run the goma package update script to push these packages to goma. If you do not have the necessary credentials to do the upload, ask firstname.lastname@example.org to find someone who does
Run an exhaustive set of try jobs to test the new compiler:
git cl try && git cl try -B luci.chromium.try -b ios-device -b mac_chromium_asan_rel_ng \ -b linux_chromium_cfi_rel_ng \ -b linux_chromium_chromeos_asan_rel_ng -b linux_chromium_msan_rel_ng \ -b linux_chromium_chromeos_msan_rel_ng -b linux-chromeos-dbg \ -b win-asan -b chromeos-amd64-generic-cfi-thin-lto-rel
Optional: Start Pinpoint perf tryjobs. These are generally too noisy to catch minor regressions pre-commit, but make sure there are no large regressions.
a. (Log in to store OAuth2 token in the depot_tools cache. Only needs to be run once:)
$ PYTHONPATH=$(dirname $(which git-cl)) python -c"import auth;auth.OAUTH_CLIENT_ID='62121018386-h08uiaftreu4dr3c4alh3l7mogskvb7i.apps.googleusercontent.com';auth.OAUTH_CLIENT_SECRET='vc1fZfV1cZC6mgDSHV-KSPOz';print auth.get_authenticator_for_host('pinpoint',auth.make_auth_config()).login()"
b. Generate a fresh Oauth2 token:
$ TOKEN=$(PYTHONPATH=$(dirname $(which git-cl)) python -c"import auth;print auth.get_authenticator_for_host('pinpoint',auth.make_auth_config()).get_access_token().token")
c. Launch Pinpoint job:
$ curl -H"Authorization: Bearer $TOKEN" -F configuration=chromium-rel-win7-gpu-nvidia \ -F target=performance_test_suite -F benchmark=speedometer2 \ -F patch=https://chromium-review.googlesource.com/c/chromium/src/+/$(git cl issue | cut -d' ' -f3) \ -F start_git_hash=HEAD -F end_git_hash=HEAD https://pinpoint-dot-chromeperf.appspot.com/api/new
d. Use the URL returned by the command above to see the progress and result of the tryjob, checking that it doesn't regress significantly (> 10%). Post the URL to the codereview.
Commit roll CL from the first step
The bots will now pull the prebuilt binary, and goma will have a matching binary, too.
The clang package is downloaded unconditionally by all bots and devs. It's called “clang” for historical reasons, but nowadays also contains other mission-critical toolchain pieces besides clang.
We try to limit the contents of the clang package. They should meet these criteria:
If you want to add something to the clang package that doesn‘t (yet?) meet these criteria, you can make package.py upload it to a separate zip file and then download it on an opt-in basis by requiring users to run a script to download the additional zip file. You can structure your script in a way that it downloads your additional zip automatically if the script detects an old version on disk, that way users have to run the download script just once.
tools/clang/scripts/download_lld_mac.py is an example for this (It doesn’t do the “only download if old version is on disk or if requested” bit, and hence doesn't run as a default DEPS hook. TODO(thakis): Make coverage stuff a better example and link to that.)
If you‘re adding a new feature that you expect will meet the inclusion criteria eventually but doesn’t yet, start by having your things in a separate zip and move it to the main zip once the criteria are met.