Google Git
Sign in
chromium / infra / goma / client / b398f18fab944184f795cdcf0062994597578912
commitb398f18fab944184f795cdcf0062994597578912[log] [tgz]
authorGoma Authors <goma-dev@google.com>Fri Apr 24 02:37:27 2020
committerYoshisato Yanagisawa <yyanagisawa@chromium.org>Thu Apr 30 04:53:49 2020
tree7b3242e674bad1a9358a2727b48f7d40da58ebfd
parent897c9f8406700b241acbcedefa1ab02486bdf063 [diff]
Import changes for goma client

  - 85e482ed03d76f5ae65568909ff9e97a2f22d9a6 don't upload file in side channel until server reports it...
  - 8461cb8660d0306e6155ceddc413dbc400131dbd Roll client/tools/clang d31d6db99fb8..04b99e7bf916 (1 com...
  - e0eafff679f4cc03c6419137f0ddc059cfaca00f Roll client/tools/clang d6b7380adfb5..d31d6db99fb8 (1 com...
  - 63a56d5f506a935725a2218915f0ea41d2295dc7 Roll client/tools/clang f5ace4f60d26..d6b7380adfb5 (1 com...
  - 93fdeaf5c8c8e437c474a7531e74f44db8d51fa7 Roll client/third_party/boringssl/src 2a8e294b7..367d64f84
  - ca68e02c60c7e31648d11c5eb2abdc79ab9ce89b Roll client/tools/clang 0d4e710a2d07..f5ace4f60d26 (2 com...
  - ff5e130d78ac84129489a49af14ef82b84907c0f Roll client/tools/clang f19bdb8edc90..0d4e710a2d07 (1 com...
  - f34f2e46609b5e81cbdd463061e037dd955486e0 Roll client/tools/clang bd1200e8be20..f19bdb8edc90 (1 com...
  - 40e5a8ced11a84893641e767369d9e038506e34c Roll client/tools/clang 698732d5db36..bd1200e8be20 (1 com...
  - 1f12f9c2037dfd865678f3dc50658d450d95e9ad Should not clear error_message even if local run succeeds.
  - 3820d965c4123bcff9362ef8873fad9f21cef0a9 remove WriteFileBlob
  - 5fb7c8c6cb46890607422cd2cc22917677acb08b Roll client/third_party/boringssl/src 72cff8193..2a8e294b7
  - b17c092dc633e9a6d3d23cb76c2068fe176a88f3 Roll client/tools/clang 28aeb03170a1..698732d5db36 (1 com...

GitOrigin-RevId: 85e482ed03d76f5ae65568909ff9e97a2f22d9a6
Change-Id: I6bf4a8d0e31828cf3c9cb70c6d22ae0a90ba7ef3
8 files changed
tree: 7b3242e674bad1a9358a2727b48f7d40da58ebfd
  1. base/
  2. benchmark/
  3. build/
  4. client/
  5. doc/
  6. infra/
  7. lib/
  8. test/
  9. testing/
  10. third_party/
  11. tools/
  12. .clang-format
  13. .gitignore
  14. .gn
  15. .style.yapf
  16. .vpython
  17. .vpython3
  18. BUILD.gn
  19. codereview.settings
  20. CONTRIBUTING.md
  21. DEPS
  22. LICENSE
  23. OWNERS
  24. PRESUBMIT.py
  25. README.md
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: currently the Goma backend is not available for non googlers. We're working so that chromium developers can use it. Stay tuned.

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}. You can set this in your environment, but do not export it as it will make gomacc complain.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args='is_debug=false' out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11 features, we prefer to follow chromium guideline: http://chromium-cpp.appspot.com/

How to use

For Chromium/Android development

Goma can be integrated with Chromium/Android development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "${GOMA_SRC}/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For Android

$ source build/envsetup.sh
$ lunch aosp_arm-eng
$ GOMA_DIR="${GOMA_SRC}/client/out/Release" USE_GOMA=true make -j4

Here, -j4 is not related to Goma parallelism. Android internally sets -j500 (or -j with NINJA_REMOTE_NUM_JOBS environment variable) for Goma.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8088 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.