Google Git
Sign in
chromium / infra / goma / client / 01dbe2875fb5aaa3a3bd125b40afca28ce2faa26
commit01dbe2875fb5aaa3a3bd125b40afca28ce2faa26[log] [tgz]
authorGoma Authors <goma-dev@google.com>Fri Feb 01 10:38:50 2019
committerFumitoshi Ukai <ukai@google.com>Mon Feb 04 01:32:20 2019
treee5e7a8effd7eabf54aa33ae4e20761bbcbdc2618
parentd7e13a3345651f974e59fedbf0fa0cd5a28363fc [diff]
Import changes for goma client

  - 008c7a3d189ef97511c576a052d7d75442e31004 Fix typo
  - 8b8e906457cc1253edcee6a5ab6aef1c47e5c88f Add GOMA_ENABLE_REMOTE_CLANG_MODULES flag
  - f0282c0b6cd03da47515ffe32a854edb7eca270b Convert python script from python2 to python3 in build/
  - fc8cdd868a9c91a0128a07ed49eba375de8d3a05 Fix cc1 and cc1plus location on Linux
  - 8c947803dfa4ae2f3bc7a56c74eb5d18c8decad8 Support clang in chromeos chroot env for ATS
  - 0e79e0102dc873008970934bf9de01915dd4f878 Add more required files for Windows nacl-gcc build
  - f2a0bfe1173cf66ddf58106ec1990cb051faa2be client: Disambiguate "failed to get resource info" messages
  - b1a8cb7558b62440a8d470bacebc34efc59f3c8a Remove duplicated comment
  - 270af923a2cc42271bd46bc5f8f2ff8cd374de32 [copybara] exclude buildtools
  - c22ea75919493213a3bc3cd426268f896b0cb719 Support --target for clang-cl
  - cddecb04d1e5bccc0143ac13f159768bd0d8f82c add suggestion to install pywin32
  - 674f44b7f3e0f0592671ea63696662286127eeaa Remove workaround to build BoringSSL asm for Windows ASan.
  - 92224046a3b3b499b1ca744101df25cd28b144b4 Roll client/third_party/boringssl/src 32e59d2d3..8cbb5f8f2
  - b1d99374a3227ea1c1907e1ce8b121576a862eb3 Fix goma client build on Mac 10.14
  - c44414d12618597c348b8a70758ddc237960683a Revert "Show workaround as error message for mac SDK miss...
  - f5be43393dab7821eb2f54baa73f3ff33dd1e28d Show workaround as error message for mac SDK missing
  - 4fcf69043d3cb1c69a675f5a01606690bbc75b07 Run gomacc_close with another compiler_proxy instance.

GitOrigin-RevId: 008c7a3d189ef97511c576a052d7d75442e31004
Change-Id: Id138ef652f5cebcf4ed8783bc9872e8d88210396
63 files changed
tree: e5e7a8effd7eabf54aa33ae4e20761bbcbdc2618
  1. base/
  2. benchmark/
  3. build/
  4. client/
  5. doc/
  6. infra/
  7. lib/
  8. test/
  9. testing/
  10. third_party/
  11. tools/
  12. .clang-format
  13. .gitignore
  14. .gn
  15. BUILD.gn
  16. codereview.settings
  17. CONTRIBUTING.md
  18. DEPS
  19. LICENSE
  20. OWNERS
  21. PRESUBMIT.py
  22. README.md
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: currently the Goma backend is not available for non googlers. We're working so that chromium developers can use it. Stay tuned.

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args='is_debug=false' out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11 features, we prefer to follow chromium guideline: http://chromium-cpp.appspot.com/

How to use

For Chromium/Android development

Goma can be integrated with Chromium/Android development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "$GOMA_SRC/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For Android

$ source build/envsetup.sh
$ lunch aosp_arm-eng
$ GOMA_DIR=${GOMA_SRC}/client/out/Release USE_GOMA=true make -j4

Here, -j4 is not related to Goma parallelism. Android internally sets -j500 (or -j with NINJA_REMOTE_NUM_JOBS environment variable) for Goma.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8080 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.