Import changes for goma client

  - 0bd90576dddc384ddc5b593ba8acd1be1928836e Roll clang from 193e3617a6f4 to 31a7453f896c (6 revisions)
  - bae80e5b0930a6e5d95ed613f97bca1e67246314 fix PRESUBMIT.py
  - 1d06ebf80126dcb634c8cff895da4d0ba426523b Roll boringssl from e5b6c141a19b to ecb7e9ae5cf7 (5 revis...
  - 50546d07b2b6b129493e8e36c5dd99f9279053a7 Roll boringssl from 5a3eb9ea7e78 to e5b6c141a19b (10 revi...
  - 1028e472b99b923d2bb836e9b0a966827d416abb Roll clang from 9fc887ccded8 to 193e3617a6f4 (2 revisions)
  - 6e0a4302a227a84d47c3a95a9b3e1b7ee76014b4 Roll boringssl from 50e30518df50 to 5a3eb9ea7e78 (12 revi...
  - 8018c7643fe6434b67f767eb4539a8f06a5735da Roll clang from 46d57cd9ee27 to 9fc887ccded8 (13 revisions)
  - 32e34fd9e76e0624b9414e74673b58ff41740ecf do nothing in update_hook when compiler proxy is not runn...
  - a841a3081df67a74bf1414399d1766af1850a9f8 Revert "goma_ctl: do nothing in update_hook if not logged...
  - ce6fcc42af80f6e70b31dda1cdb50aff8600b615 Roll boringssl from 09096a98f303 to 50e30518df50 (10 revi...
  - 50bc07f6c2ede7864a87197561f80a1f9c159150 Roll clang from 74786020a98a to 46d57cd9ee27 (7 revisions)
  - 4d0f861cc76d3344ad863dd711e183de0e33f15a Roll boringssl from 58adb8e1d62d to 09096a98f303 (21 revi...
  - aabecf163c2adf3f6a7b11b7d916ae289de70666 Roll clang from 02d5529a3fa5 to 74786020a98a (12 revisions)
  - e56bbd354f943bd6974c9c004d8df430ded35558 Roll boringssl from 7ae2b910c130 to 58adb8e1d62d (6 revis...
  - 8c4efb7c9598346243d8f47238e11610c7117756 update clang features for 18.0.0
  - 142eb5f38c2c27a189d56a6a18ed55a15dc58708 use go 1.20.7
  - bce02e2cc49d97643cb428a0c37c49ff4b0f0b6c Roll boringssl from e4acd6cb5682 to 7ae2b910c130 (7 revis...
  - e68ad78c3501b4bcc308970c6b73d752c7e97a01 Roll clang from 12a3758d0c8d to 02d5529a3fa5 (3 revisions)
  - 2d1b9f927146217b68327d138f5f2c2a01f96945 remove old local cache items if exceeds max
  - 4d71debe835df8de346bd60b0bd181aefde39b59 Roll boringssl from e9f816b12b3e to e4acd6cb5682 (6 revis...
  - 7be6766f925c3eba4c6959ae8b18922fc732b57c Roll clang from 9e679731de1e to 12a3758d0c8d (3 revisions)
  - 8af0032ea9fbafc7ea7ce49ca2c3d57eb1a7c7ef goma_ctl: do nothing in update_hook if not logged in.
  - f305638440d0e5d34c4f20fc55c759f6856fefb7 Roll boringssl from 26ecb2a275ca to e9f816b12b3e (20 revi...
  - 8814b0f81c934f676d073965d83c588cb1546874 Roll clang from 4786f206b4a3 to 9e679731de1e (4 revisions)
  - af89d3eba259a6bdeb4f5de5b7870ead458656b1 Roll boringssl from 80dcb67d4481 to 26ecb2a275ca (20 revi...
  - b2f8cb095533ea81959c0554e7c1c26735fb5b92 Roll clang from 50358e9b66c2 to 4786f206b4a3 (6 revisions)
  - 3fceb408b5a08c8331653d66dd8ccc42b2f6264b Simplify BoringSSL build
  - b5c988e91cea736782ecefcc2449ea82b9f93e6a Roll boringssl from 9c0f22cdfbb6 to 80dcb67d4481 (18 revi...
  - a6577edb5743358bc8042fc5cea0fde921d8da08 Roll clang from bb0c3cb09901 to 50358e9b66c2 (9 revisions)
  - d3a35334b3d26dbfba514837fc33978642db15a6 GOMA_HERMETIC=error by default
  - cca5039b3fb99a10b28319d7ce470b5554c6452f Roll boringssl from a905bbb52a7b to 9c0f22cdfbb6 (10 revi...
  - 43db0e3b2ad3a55516d77a0eec02331bcebce7bd Roll clang from ff9b9e9ce251 to bb0c3cb09901 (3 revisions)
  - 71757d397c9ef4ba0659e546b7188f6bf1aa8996 Roll boringssl from e1b8685770d0 to a905bbb52a7b (11 revi...
  - c8c1fdb661ce614f92a47c963aadadb9ff11439b Roll clang from 573d371dff6d to ff9b9e9ce251 (3 revisions)
  - 5e8d5c363d418245bf759ff68bb30b6876cb1a86 Roll boringssl from 23ed9d3852bb to e1b8685770d0 (11 revi...
  - a16f7bdafe7255e8fd931db909eeb8152cd73718 Roll clang from 4d57e1de6bd7 to 573d371dff6d (9 revisions)
  - 71516817aa32f8126de0bd8ce1646a38bd7da53b use go 1.20.5
  - 43c1f7e346a89bf4f92c3519da928938b2ff2d09 Roll boringssl from 28c24092e39b to 23ed9d3852bb (23 revi...
  - 0c5f98ea86761b4735e99abb03639e310ab4c642 goma_ctl.py: remove some logs in update_hook
  - b1a415d9c160d4c32159388c0b6a77457e3fcdbf Roll clang from 2c485d86b9fd to 4d57e1de6bd7 (15 revisions)

Bug: b/286479835
GitOrigin-RevId: 0bd90576dddc384ddc5b593ba8acd1be1928836e
Change-Id: I4b9bf0d5ebd96126d43f9c2a233f093b5c7990ec
176 files changed
tree: 35cb6f970b1df12b14c8609f0169422c72496ab0
  1. base/
  2. benchmark/
  3. build/
  4. client/
  5. doc/
  6. go/
  7. infra/
  8. lib/
  9. test/
  10. testing/
  11. third_party/
  12. tools/
  13. .clang-format
  14. .gitallowed
  15. .gitignore
  16. .gn
  17. .style.yapf
  18. BUILD.gn
  19. codereview.settings
  20. CONTRIBUTING.md
  21. DEPS
  22. LICENSE
  23. OWNERS
  24. PRESUBMIT.py
  25. README.md
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: For non-Googler usage, please see Goma for Chromium Contributors.

Google employees interested in contributing to the goma client should use internal version. see http://go/ma-client-code

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ mkdir goma && cd goma
$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}. You can set this in your environment, but do not export it as it will make gomacc complain.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args="is_debug=false" out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11/14 features, we prefer to follow Chromium's guidelines.

How to use

For Chromium development

Goma can be integrated with Chromium development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "${GOMA_SRC}/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8088 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.