Import changes for goma client

  - 48530404981888a8eb461bc8cc32598b78de5290 Roll client/third_party/boringssl/src ae1e08709..fdb48f986
  - a7825f9981f68a4a2f2e3305b834ac5780dd7388 Reland "Workaround: suspend presubmit check using gclient...
  - d578f6d982bfb6e1e27a519f45e9f23378836b1b show more log in spawner_posix
  - a8358bba9cd6809c29e76618acb9e837dc3617b0 openssl_engine: update the session structure if callback ...
  - 781b3ec4917439e8e26cdb496475f6bb5d9f37aa Revert "Workaround: suspend presubmit check using gclient...
  - e2544bc0bd6deb1154852a1414a9236c3a0ce897 Should not record abort case as update_required_files fai...
  - 2ca3d5d264df6834681dc068ef037ead5794b7d8 Workaround: suspend presubmit check using gclient_utils.
  - 2edb35512161f02ca6daf65d9bebee84a242beb3 Fix on finding ChromeOS clang deps.
  - 2ac2360b12ce4ce421fb24043174b92158c7359b statusz: no http_status doesn't mean conftestfailure
  - 6d04a3f0d6c00337492bd24bc2bffdb50127e404 close fd before _exit
  - 272fe2d590a62e196acba1770e83818f969e6728 Show sum of input file size

GitOrigin-RevId: 48530404981888a8eb461bc8cc32598b78de5290
Change-Id: Idcb40ce8fce3fb1a643321ad00ffe400c46f0163
18 files changed
tree: 4508566a6dc2af865b2cdb25f0ae093422586c76
  1. .clang-format
  2. .gitignore
  3. .gn
  4. BUILD.gn
  5. CONTRIBUTING.md
  6. DEPS
  7. LICENSE
  8. OWNERS
  9. PRESUBMIT.py
  10. README.md
  11. base/
  12. benchmark/
  13. build/
  14. client/
  15. codereview.settings
  16. doc/
  17. infra/
  18. lib/
  19. test/
  20. testing/
  21. third_party/
  22. tools/
README.md

Goma client

Goma is a distributed compiler service for open-source project such as Chromium and Android. It's some kind of replacement of distcc+ccache.

NOTE: currently the Goma backend is not available for non googlers. We're working so that chromium developers can use it. Stay tuned.

How Goma works

Goma hooks a compile request, and sends it to a backend compile server. If you have plenty of backend servers, a lot of compile can be processed in parallel, for example, -j100, -j500 or -j1000.

Also, the Goma backend caches the compile result. If the same compile request comes, the cached result is returned from the Goma cache server.

How to build

Goma client can be built on Linux, Mac, and Win.

Prerequisite

  1. Use 64bit platform (Linux, Mac or Win).
  2. Install depot_tools.
  3. Install dependencies.
    • On Mac, install Xcode.
    • On Windows, install Visual Studio 2017. Community edition is OK.

Checkout source

$ gclient config https://chromium.googlesource.com/infra/goma/client
$ gclient sync
$ cd client

We assume the Goma client code is checked out to ${GOMA_SRC}.

If you want to develop goma client, make goma client source unmanaged by gclient. Open .gclient file, and check "managed" value. If it's True, changed it to False. Otherwise, gclient will manage your repository, so your checkout can be unintentionally changed with gclient sync.

Move to client directory (which is under git repo), and configure git repository with your username and emails.

$ cd client
$ git config user.email 'foo@example.com'
$ git config user.name 'Your Name'

Build

$ cd "${GOMA_SRC}/client"
$ gclient sync
$ gn gen --args='is_debug=false' out/Release
$ ninja -C out/Release

Several important gn args

The build option can be modified with gn args.

is_debug=true/false
  Do debug build if true.
dcheck_always_on=true/false
  Enable DCHECK always (even in release build).
is_asan=true/false
  Use ASan build (with clang).
use_link_time_optimization=true/false
  Currently working only on Win. If true, /LTCG is enable.
use_lld=true/false
  Use lld for link (it will be fast)

Run unittest

$ cd "${GOMA_SRC}/client"
$ ./build/run_unittest.py --target=Release --build-dir=out

Coding style

Follow Google code style.

For C++11 features, we prefer to follow chromium guideline: http://chromium-cpp.appspot.com/

How to use

For Chromium/Android development

Goma can be integrated with Chromium/Android development easily.

  1. Build goma client
  2. Start compiler_proxy
$ "$GOMA_SRC/client/out/Release/goma_ctl.py" start

For Chromium

In Chromium src, specify the following args in gn args

use_goma = true
goma_dir = "${GOMA_SRC}/client/out/Release"  (Replace ${GOMA_SRC} to your checkout)

Then build like the following:

$ cd /path/to/chromium/src/out/Release
$ ninja -j100 chrome

More details are avairable in chromium's build instructions.

For Android

$ source build/envsetup.sh
$ lunch aosp_arm-eng
$ GOMA_DIR=${GOMA_SRC}/client/out/Release USE_GOMA=true make -j4

Here, -j4 is not related to Goma parallelism. Android internally sets -j500 (or -j with NINJA_REMOTE_NUM_JOBS environment variable) for Goma.

For general development

  1. Build Goma client
  2. Start compiler_proxy
$ ./goma_ctl.py ensure_start
  1. Change your build script so that gomacc is prepended to compiler command. For example:
$ gomacc clang++ -c foo.cc
  1. Build your product with make -j100, ninja -j100 or larger -j. Check http://localhost:8080 to see compiler_proxy is actually working.
  • You can use autoninja in depot_tools instead of specifying gomacc manually.