Import changes for goma client

  - c8539036078c5bb693935c6df58b891447ef02be Roll boringssl from bcecc7d834fc to a02b7435ca52 (17 revi...
  - 6bc1ebd594227de61c5bf8c0c7ffc4fa58788174 Roll clang from a443ff055ede to 2f9ce8ce378c (7 revisions)
  - 7de7a6a520e9185b069162ddf3f106a4b4834768 Roll boringssl from 298e6c2b9c97 to bcecc7d834fc (22 revi...
  - 44c6247b44480159c2b68130e8896cc3e670e6ba Roll clang from c284b5d132fc to a443ff055ede (3 revisions)
  - 1bddc48cb111fd6400c912d2ac9c59d20f074502 Roll boringssl from 6e723e5b37f7 to 298e6c2b9c97 (17 revi...
  - 563a87b9746b711119d708b6f43ad31ac4391baa Roll clang from 26f481ad11d2 to c284b5d132fc (6 revisions)
  - b232bd5716d3206752cb05b0c8d529431f9b8554 Roll boringssl from 58472cc752c9 to 6e723e5b37f7 (4 revis...
  - 25a398996dcd6519e952abdadc75dff4510fc8da Roll clang from fcfd861069ea to 26f481ad11d2 (8 revisions)
  - 977111066d40c44eaed2dbed00709d1e2d3b1443 Roll boringssl from 2e13e36e7477 to 58472cc752c9 (7 revis...
  - 56bac9041ce33b91331716c1e83afc313d8ad813 Roll clang from 4363a3a6af1e to fcfd861069ea (6 revisions)
  - ce7d87ddefaecfe060d77e717ce7c9d6515ccac4 update clang features for 17.0.0
  - b99b265a36d1a51f5c00c54254061e9b6b3e2e64 Roll boringssl from 898de8d09e10 to 2e13e36e7477 (7 revis...
  - 6b130b77f2ddb572d1ffb4847cd32ec46b89a4e5 Roll clang from 998617f817d5 to 4363a3a6af1e (6 revisions)
  - 1cf9396a10e698b8095ad017688ad440e56015b2 Roll client/third_party/jinja2/ 78d2f6721..b08cd4bc6 (498...
  - ddcdf67e7b0ba0f76f35e1ef0ed585ea4f61cfa9 Roll boringssl from 8aa51ddfcf1f to 898de8d09e10 (16 revi...
  - 944eee591b15bbafea49a7e58201604eefdd3a06 Roll clang from 2b181989bfb1 to 998617f817d5 (4 revisions)
  - 23fdbca70ef076a781508c7308cd2699821c55c7 compile_service: warning for "too large output buf size:"
  - 7084ff98cf3c9bcbcf1df90fa1aa5a7668f5e1d3 set GLOG_stderrthreshold to 3 (FATAL)
  - 607684bbf8cb3e0da91b298f735723393a0df4dc goma_ctl: do nothing in update_hook if not yet logged in
  - ec3c00a4b1f9843e86cef71ac04faef0d54c04b4 use yapf style
  - 1ab2f9c45ab7494d85dfd94c6785f41f3a25126f Roll boringssl from 082e953a134a to 8aa51ddfcf1f (3 revis...
  - dbf964c17b68029ec8fa40df126c6cb9eeadb432 Roll clang from 697c29733068 to 2b181989bfb1 (5 revisions)
  - 719851612dcbd587334d2f2a54cd9438b24b45bd use go 1.20.2
  - ffb47187d3017590ed8683cf6f146524c4902301 Roll client/third_party/boringssl/src 7864b1489..082e953a1
  - 2f3d4adeb3a577ba9908866e00a13491b5885852 Roll client/third_party/boringssl/src bab2f96e2..7864b1489
  - a307620e7310d4d1618680919f8a0092871e18c3 Roll clang from 59d4a6deb0f4 to 697c29733068 (3 revisions)

Bug: b/271506681
GitOrigin-RevId: c8539036078c5bb693935c6df58b891447ef02be
Change-Id: Iec73980b4b59c0274942e623462eabeb3948bcba
52 files changed
tree: 41ab8d602bf09a757c206f231dbe5371553207af
  1. base/
  2. benchmark/
  3. build/
  4. client/
  5. doc/
  6. go/
  7. infra/
  8. lib/
  9. test/
  10. testing/
  11. third_party/
  12. tools/
  13. .clang-format
  14. .gitallowed
  15. .gitignore
  16. .gn
  17. .style.yapf
  18. BUILD.gn
  19. codereview.settings
  20. CONTRIBUTING.md
  21. DEPS
  22. LICENSE
  23. OWNERS
  24. PRESUBMIT.py
  25. README.md
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: For non-Googler usage, please see Goma for Chromium Contributors.

Google employees interested in contributing to the goma client should use internal version. see http://go/ma-client-code

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ mkdir goma && cd goma
$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}. You can set this in your environment, but do not export it as it will make gomacc complain.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args="is_debug=false" out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11/14 features, we prefer to follow Chromium's guidelines.

How to use

For Chromium development

Goma can be integrated with Chromium development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "${GOMA_SRC}/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8088 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.