Google Git
Sign in
chromium / chromiumos / third_party / toolchain-utils / 3c36449dd4c66f7415f0a23b7908f5496b0dd743 / . / compiler_wrapper
tree: ad28f078f84fb5be782216068953cbfac8e28ad2 [path history] [tgz]
  1. testdata/
  2. android_config_test.go
  3. bisect_flag.go
  4. bisect_flag_test.go
  5. build.py
  6. bundle.py
  7. bundle.README
  8. ccache_flag.go
  9. ccache_flag_test.go
  10. clang_flags.go
  11. clang_flags_test.go
  12. clang_syntax_flag.go
  13. clang_syntax_flag_test.go
  14. clang_tidy_flag.go
  15. clang_tidy_flag_test.go
  16. command.go
  17. command_test.go
  18. compile_with_fallback.go
  19. compile_with_fallback_test.go
  20. compiler_wrapper.go
  21. compiler_wrapper_test.go
  22. config.go
  23. config_test.go
  24. crash_builds.go
  25. crash_builds_test.go
  26. cros_hardened_config_test.go
  27. cros_host_config_test.go
  28. cros_llvm_next_flags.go
  29. cros_nonhardened_config_test.go
  30. disable_werror_flag.go
  31. disable_werror_flag_test.go
  32. env.go
  33. env_test.go
  34. errors.go
  35. errors_test.go
  36. gcc_flags.go
  37. gcc_flags_test.go
  38. go.mod
  39. go_exec.go
  40. goldenutil_test.go
  41. install_compiler_wrapper.sh
  42. kernel_bug.go
  43. kernel_bug_test.go
  44. libc_exec.go
  45. libgcc_flags.go
  46. libgcc_flags_test.go
  47. main.go
  48. print_cmdline_flag.go
  49. print_cmdline_flag_test.go
  50. print_config_flag.go
  51. print_config_flag_test.go
  52. README.md
  53. remote_build_flag_test.go
  54. remote_build_flags.go
  55. reset_compiler_wrapper.sh
  56. rusage_flag.go
  57. rusage_flag_test.go
  58. sanitizer_flags.go
  59. sanitizer_flags_test.go
  60. stackprotector_flags.go
  61. stackprotector_flags_test.go
  62. sysroot_flag.go
  63. sysroot_flag_test.go
  64. testutil_test.go
  65. thumb_flags.go
  66. thumb_flags_test.go
  67. unsupported_flags.go
  68. unsupported_flags_test.go
  69. x64_flags.go
  70. x64_flags_test.go
compiler_wrapper/README.md

Compiler wrapper

See the comments on the top of main.go. Build is split into 2 steps via separate commands:

  • bundle: copies the sources and the build.py file into a folder.
  • build: builds the actual go binary, assuming it is executed from the folder created by bundle.py.

This allows to copy the sources to a ChromeOS / Android package, including the build script, and then build from there without a dependency on toolchain-utils itself.

Testing Inside the Chroot

To test updates to the wrapper locally:

Run install_compiler_wrapper.sh to install the new wrapper in the chroot:

(chroot) ~/trunk/src/third_party/toolchain-utils/compiler_wrapper/install_compiler_wrapper.sh

Then perform the tests, e.g. build with the new compiler.

Updating the Wrapper for ChromeOS

To update the wrapper for everyone, the new wrapper configuration must be copied into chromiumos-overlay, and new revisions of the gcc and llvm ebuilds must be created.

Copy over sources and build.py to chromiumos-overlay:

(chroot) /mnt/host/source/src/third_party/chromiumos-overlay/sys-devel/llvm/files/update_compiler_wrapper.sh

Rename chromiumos-overlay/sys-devel/llvm/llvm-${VERSION}.ebuild to the next revision number. For example, if the current version is 11.0_pre394483_p20200618-r2:

(chroot) cd ~/trunk/src/third_party/chromiumos-overlay
(chroot) git mv llvm-11.0_pre394483_p20200618-r2.ebuild llvm-11.0_pre394483_p20200618-r3.ebuild

Rename chromiumos-overlay/sys-devel/gcc/gcc-${VERSION}.ebuild to the next revision number. For example, if the current version is 10.2.0-r3:

(chroot) cd ~/trunk/src/third_party/chromiumos-overlay
(chroot) git mv sys-devel/gcc/gcc-10.2.0-r3.ebuild sys-devel/gcc/gcc-10.2.0-r4.ebuild

Commit those changes together with the changes made by update_compiler_wrapper.sh.

The changes can then be reviewed and submitted through the normal process.

Paths

build.py is called by these ebuilds:

  • third_party/chromiumos-overlay/sys-devel/llvm/llvm-*.ebuild
  • third_party/chromiumos-overlay/sys-devel/gcc/gcc-*.ebuild

Generated wrappers are stored here:

  • Sysroot wrapper with ccache: /usr/x86_64-pc-linux-gnu/<arch>/gcc-bin/10.2.0/sysroot_wrapper.hardened.ccache
  • Sysroot wrapper without ccache: /usr/x86_64-pc-linux-gnu/<arch>/gcc-bin/10.2.0/sysroot_wrapper.hardened.noccache
  • Clang host wrapper: /usr/bin/clang_host_wrapper
  • Gcc host wrapper: /usr/x86_64-pc-linux-gnu/gcc-bin/10.2.0/host_wrapper

Using the compiler wrapper to crash arbitrary compilations

When Clang crashes, its output can be extremely useful. Often, it will provide the user with a stack trace, and messages like:

clang-15: unable to execute command: Illegal instruction
clang-15: note: diagnostic msg: /tmp/clang_crash_diagnostics/foo-5420d2.c
clang-15: note: diagnostic msg: /tmp/clang_crash_diagnostics/foo-5420d2.sh

Where the artifacts at /tmp/clang_crash_diagnostics/foo-* are a full, self-contained reproducer of the inputs that caused the crash in question. Often, such a reproducer is very valuable to have even for cases where a crash doesn't happen (e.g., maybe Clang is now emitting an error where it used to not do so, and we want to bisect upstream LLVM with that info). Normally, collecting and crafting such a reproducer is a multi-step process, and can be error-prone; compile commands may rely on env vars, they may be done within chroots, they may rely on being executed in a particular directory, they may rely on intermediate state, etc.

Because of the usefulness of these crash reports, our wrapper supports crashing Clang even on files that ordinarily don't cause Clang to crash. For various reasons (b/236736327), this support currently requires rebuilding and redeploying the wrapper in order to work. That said, this could be a valuable tool for devs interested in creating a self-contained reproducer without having to manually reproduce the environment in which a particular build was performed.