Improve presubmit no-network handling

Recent gerrit issues made it obvious that "git cl presubmit" relies on
gerrit much more than most people would expect (the expectation is zero
for many people). This makes presubmits flaky or much slower under poor
network conditions, and it means that the presubmit step may drastically
underestimate how long it takes to run because of a
cl.FetchDescription() that may occur outside of the timed portion of the
presubmits.

This change wraps more network-touching steps in try/except blocks, to
make them robust. It also gets them to check for the existence of a
PRESUBMIT_SKIP_NETWORK environment variable. And, it prints the elapsed
time to get the CL description if this is inordinately long.

Bug: 1350227
Change-Id: I7954fd50e928fd24975a4f61a316cb280542ebbd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3813095
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
3 files changed
tree: 4514ba17ae4ccf4a3e55536b94eedfea9f6a9a8f
  1. bootstrap/
  2. fetch_configs/
  3. git-templates/
  4. infra/
  5. man/
  6. python-bin/
  7. python2-bin/
  8. recipes/
  9. testing_support/
  10. tests/
  11. third_party/
  12. win_toolchain/
  13. zsh-goodies/
  14. .cipd_impl.ps1
  15. .gitattributes
  16. .gitignore
  17. .isort.cfg
  18. .style.yapf
  19. .vpython
  20. .vpython3
  21. auth.py
  22. autoninja
  23. autoninja.bat
  24. autoninja.py
  25. bb
  26. bb.bat
  27. black
  28. bootstrap_python3
  29. breakpad.py
  30. cipd
  31. cipd.bat
  32. cipd_bin_setup.bat
  33. cipd_bin_setup.sh
  34. cipd_client_version
  35. cipd_client_version.digests
  36. cipd_manifest.txt
  37. cipd_manifest.versions
  38. cit
  39. cit.bat
  40. cit.py
  41. clang-format
  42. clang-format.bat
  43. clang_format.py
  44. clang_format_merge_driver
  45. clang_format_merge_driver.bat
  46. clang_format_merge_driver.py
  47. codereview.settings
  48. compile_single_file
  49. compile_single_file.bat
  50. compile_single_file.py
  51. cpplint.bat
  52. cpplint.py
  53. cpplint_chromium.py
  54. cros
  55. CROS_OWNERS
  56. crosjobs
  57. detect_host_arch.py
  58. DIR_METADATA
  59. dirmd
  60. dirmd.bat
  61. download_from_google_storage
  62. download_from_google_storage.bat
  63. download_from_google_storage.py
  64. ensure_bootstrap
  65. fetch
  66. fetch.bat
  67. fetch.py
  68. fix_encoding.py
  69. gclient
  70. gclient-new-workdir.py
  71. gclient.bat
  72. gclient.py
  73. gclient_completion.sh
  74. gclient_eval.py
  75. gclient_paths.py
  76. gclient_scm.py
  77. gclient_utils.py
  78. gerrit_client.py
  79. gerrit_util.py
  80. git-cache
  81. git-cl
  82. git-crrev-parse
  83. git-drover
  84. git-find-releases
  85. git-footers
  86. git-freeze
  87. git-gs
  88. git-hyper-blame
  89. git-map
  90. git-map-branches
  91. git-mark-merge-base
  92. git-migrate-default-branch
  93. git-nav-downstream
  94. git-nav-upstream
  95. git-new-branch
  96. git-number
  97. git-rebase-update
  98. git-rename-branch
  99. git-reparent-branch
  100. git-retry
  101. git-runhooks
  102. git-squash-branch
  103. git-thaw
  104. git-upstream-diff
  105. git_cache.py
  106. git_cl.py
  107. git_cl_completion.sh
  108. git_common.py
  109. git_dates.py
  110. git_drover.py
  111. git_find_releases.py
  112. git_footers.py
  113. git_freezer.py
  114. git_hyper_blame.py
  115. git_map.py
  116. git_map_branches.py
  117. git_mark_merge_base.py
  118. git_migrate_default_branch.py
  119. git_nav_downstream.py
  120. git_new_branch.py
  121. git_number.py
  122. git_rebase_update.py
  123. git_rename_branch.py
  124. git_reparent_branch.py
  125. git_retry.py
  126. git_squash_branch.py
  127. git_upstream_diff.py
  128. gn
  129. gn.bat
  130. gn.py
  131. goma_auth
  132. goma_auth.bat
  133. goma_ctl
  134. goma_ctl.bat
  135. GOMA_OWNERS
  136. gsutil.py
  137. gsutil.py.bat
  138. gsutil.vpython3
  139. isort
  140. led
  141. led.bat
  142. LICENSE
  143. lockfile.py
  144. luci-auth
  145. luci-auth.bat
  146. LUCI_OWNERS
  147. lucicfg
  148. lucicfg.bat
  149. mac_toolchain
  150. metrics.py
  151. metrics.README.md
  152. metrics_utils.py
  153. my_activity.py
  154. ninja
  155. ninja-linux64
  156. ninja-mac
  157. ninja.exe
  158. ninjalog.README.md
  159. ninjalog_uploader.py
  160. ninjalog_uploader_wrapper.py
  161. OWNERS
  162. owners.py
  163. owners_client.py
  164. owners_finder.py
  165. pinpoint
  166. pinpoint.bat
  167. post_build_ninja_summary.py
  168. PRESUBMIT.py
  169. presubmit_canned_checks.py
  170. presubmit_canned_checks_test.py
  171. presubmit_canned_checks_test_mocks.py
  172. presubmit_support.py
  173. profile.xml
  174. prpc
  175. prpc.bat
  176. pylint-1.5
  177. pylint-1.5.bat
  178. pylint-2.6
  179. pylint-2.6.bat
  180. pylint-2.7
  181. pylint-2.7.bat
  182. pylint_main.py
  183. pylintrc
  184. python_runner.sh
  185. rdb
  186. rdb.bat
  187. rdb_wrapper.py
  188. README.gclient.md
  189. README.git-cl.md
  190. README.md
  191. repo
  192. repo_launcher
  193. roll-dep
  194. roll-dep.bat
  195. roll_dep.py
  196. rustfmt.py
  197. scm.py
  198. setup_color.py
  199. split_cl.py
  200. subcommand.py
  201. subprocess2.py
  202. swift-format
  203. swift_format.py
  204. update_depot_tools
  205. update_depot_tools.bat
  206. update_depot_tools_toggle.py
  207. upload_metrics.py
  208. upload_to_google_storage.py
  209. utils.py
  210. vpython
  211. vpython.bat
  212. vpython3
  213. vpython3.bat
  214. WATCHLISTS
  215. watchlists.py
  216. weekly
  217. whitespace.txt
  218. win32imports.py
  219. wtf
  220. yapf
  221. yapf.bat
README.md

depot_tools

Tools for working with Chromium development. It requires python 2.7 or 3.8 for python 3 support.

Tools

The most important tools are:

  • fetch: A gclient wrapper to checkout a project. Use fetch --help for more details.
  • gclient: A meta-checkout tool. Think repo or git submodules, except that it support OS-specific rules, e.g. do not checkout Windows only dependencies when checking out for Android. Use gclient help for more details and README.gclient.md.
  • git cl: A code review tool to interact with Rietveld or Gerrit. Use git cl help for more details and README.git-cl.md.
  • roll-dep: A gclient dependency management tool to submit a dep roll, updating a dependency to a newer revision.

There are a lot of git utilities included.

Updating

depot_tools updates itself automatically when running gclient tool. To disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0 or run ./update_depot_tools_toggle.py --disable.

To update package manually, run update_depot_tools.bat on Windows, or ./update_depot_tools on Linux or Mac.

On Windows only, running gclient will install git and python.

Contributing

To contribute change for review:

git new-branch <somename>
# Hack
git add .
git commit -a -m "Fixes goat teleporting"
# find reviewers
git cl owners
git log -- <yourfiles>

# Request a review.
git cl upload -r reviewer1@chromium.org,reviewer2@chromium.org --send-mail

# Edit change description if needed.
git cl desc

# If change is approved, flag it to be committed.
git cl set-commit

# If change needs more work.
git rebase-update
...
git cl upload -t "Fixes goat teleporter destination to be Australia"

See also open bugs, open reviews, forum or report problems.

cpplint.py

Until 2018, our cpplint.py was a copy of the upstream version at https://github.com/google/styleguide/tree/gh-pages/cpplint. Unfortunately, that repository is not maintained any more. If you want to update cpplint.py in depot_tools, just upload a patch to do so. We will figure out a long-term strategy via issue https://crbug.com/916550.

Note that the cpplint.py here is also used by the Tricium analyzer, so if the cpplint.py here changes, we should also update the copy used there.