Google Git
Sign in
chromium / infra / goma / client / f0ba7e139e282e702e74bd507b9440051e97072c
commitf0ba7e139e282e702e74bd507b9440051e97072c[log] [tgz]
authorGoma Authors <goma-dev@google.com>Fri Jan 22 12:49:45 2021
committerYoshisato Yanagisawa <yyanagisawa@chromium.org>Mon Jan 25 02:51:53 2021
treed6d19edec0b4fcff4c499240e6adceac34c55645
parent432acf306d0f92b538f298738466d4029795fb72 [diff]
Import changes for goma client

  - 10c6fdabf9edad62c385e0ac041fb2f9ca7ec791 Roll clang from 786cae1ea050 to 2cf38b3d214d (1 revision)
  - b74bfedf11dadfe21d31362290da8113e05c2bb8 Ignore failing simpletry.py tests.
  - 028a101d1329610bce1f527c61c75370a65d15da Update go to 1.15.7
  - a43ae2daf1dcf07a4fd26f374aee5c1015c7d1fa Hack for ChromeOS.
  - f5170895860cf0b4f246896fac517dd9d9a790ad Roll clang from 3c716375fd15 to 786cae1ea050 (1 revision)
  - 7607f6f14db1156fab770c959880a8e61e4777fb Deprecate legacy Goma backend.
  - 71b87c0c1de02f16f1a38909715a1634697a1d15 Roll clang from 2e4e6e975a09 to 3c716375fd15 (1 revision)
  - 0d255c15622fde18d8084a4fa5244db43bcfe9dc Roll clang from 683dad6035f2 to 2e4e6e975a09 (1 revision)
  - 585860f4677a2a521bfe70891e394fa1703906c2 goma_ctl report include goma env and `goma_auth.py config`
  - 39836b8a445714eed499ef7b34105982a3f36f7f Roll clang from 5747a7066f32 to 683dad6035f2 (1 revision)
  - 6da1b68a88fdb0ad0cdca76b6b4c143c43ff02ff Roll clang from 5d8d9fa65d19 to 5747a7066f32 (1 revision)
  - 273d559b51d7806450d94b4066dcb38b8a62da3a Roll clang from 450ae3e063f4 to 5d8d9fa65d19 (1 revision)
  - 2360a3d35cd89501ba4111ef91b61f1b58fcf94f Roll clang from 819010a7e649 to 450ae3e063f4 (1 revision)
  - a19a1e9cb6f8032db93432dc0075c7051364d115 clean up: update a hostname used in tests.
  - 9b4b3ccddce64e7fc463ece2fdb09eeb01c2999e Update go to 1.15.6.
  - 23dc72d33fadef92e1a362d792abd6e51e115de5 Roll clang from ba4ee034259c to 819010a7e649 (1 revision)
  - 67fdd51e8ca5e9c86c313c7eb04fed84e94f89d4 Roll client/third_party/boringssl/src 5d54832f1..595cdc29d
  - 591e28ad67abcc3a0704fe1446cabc7c909f8972 Update early access guide.
  - 316e0bbe4c804425066127355d790738066d4420 Roll clang from 55e7a3efbb33 to ba4ee034259c (1 revision)
  - ad94a9105e16b2e480d8ba56acca657255e65a95 Roll clang from 9206f998426b to 55e7a3efbb33 (1 revision)
  - d4275821d5ca90912941cdc4e120b4d3e8032ff7 goma_ctl: print ignored `goma_auth.py config`
  - 688a459c85f8f6a599ab7f487a219754889a83ea Roll clang from ff4f68d92c47 to 9206f998426b (1 revision)
  - 9e2121621ebf6f69c9e87de6d515d87324dbac2e Roll clang from b12d1c836e2b to ff4f68d92c47 (1 revision)
  - 209549adb0c6bddf097ef21235509499456df0cd Remove incompatible simpletry tests in simpletry.sh.
  - 373302c5ac98f80289a58e9c70227fda04ff8d4e gomacc: capture RBE_exec_root and RBE_platform
  - edc33b612c0237382110c6434630e96dec9a96bd Roll clang from 82425b6b98a7 to b12d1c836e2b (1 revision)
  - 8f826e8f34bf81cf6e396aa5fe2dc0d753d397c3 Roll clang from 2246bee280e9 to 82425b6b98a7 (1 revision)
  - f209d11e0615670f03b3fe8eef8f674154e7895c Roll client/third_party/boringssl/src 9422ac61f..5d54832f1
  - b2252d6de1f6197c69f2b64f0277a8c533ddf82e Roll clang from 01d7e1f0a1d0 to 2246bee280e9 (1 revision)
  - 87ad272dc4c403a630db1f5da8cd22da7bf2d057 Roll clang from e8d041d67901 to 01d7e1f0a1d0 (1 revision)

GitOrigin-RevId: 10c6fdabf9edad62c385e0ac041fb2f9ca7ec791
Change-Id: I69df230ee85bc1a48444565974416ef7abaefcf9
27 files changed
tree: d6d19edec0b4fcff4c499240e6adceac34c55645
  1. base/
  2. benchmark/
  3. build/
  4. client/
  5. doc/
  6. go/
  7. infra/
  8. lib/
  9. test/
  10. testing/
  11. third_party/
  12. tools/
  13. .clang-format
  14. .gitignore
  15. .gn
  16. .style.yapf
  17. .vpython
  18. .vpython3
  19. BUILD.gn
  20. codereview.settings
  21. CONTRIBUTING.md
  22. DEPS
  23. LICENSE
  24. OWNERS
  25. PRESUBMIT.py
  26. README.md
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: For non-Googler usage, please see Goma for Chromium Contributors.

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}. You can set this in your environment, but do not export it as it will make gomacc complain.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args='is_debug=false' out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11 features, we prefer to follow chromium guideline: http://chromium-cpp.appspot.com/

How to use

For Chromium/Android development

Goma can be integrated with Chromium/Android development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "${GOMA_SRC}/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For Android

$ source build/envsetup.sh
$ lunch aosp_arm-eng
$ GOMA_DIR="${GOMA_SRC}/client/out/Release" USE_GOMA=true make -j4

Here, -j4 is not related to Goma parallelism. Android internally sets -j500 (or -j with NINJA_REMOTE_NUM_JOBS environment variable) for Goma.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8088 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.