siso: strip trailing double quotes from Windows arguments

On Windows, siso.bat wraps all arguments in double quotes ("%*") to
protect special characters like '^'. However, Windows command-line
parsing rules treat a trailing backslash before a double quote as an
escape character (\").

This causes paths ending with a backslash (e.g. .\out\Default\) to be
parsed as .\out\Default" by Python. Since siso.py splits these combined
arguments on spaces without stripping quotes, the trailing double quote
was left on the path argument (.\out\Default"). Siso then failed to
chdir into this invalid path.

To recover the correct path on Windows, this change strips the trailing
double quote from the very last argument, which is the only argument
affected by siso.bat's outer double-quoting. This prevents corrupting any
legitimate trailing double quotes on other arguments (such as quote-terminated
middle arguments).

Added comprehensive parameterized unit tests to cover all these argument
splitting and stripping scenarios.

Change-Id: I5b8f12a9ce1719bd725798237cd815a551bcd72e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7894035
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
2 files changed
tree: b20a2dd9497e3b95c52773aa48cc9058a742a45b
  1. .vscode/
  2. agents/
  3. bootstrap/
  4. fetch_configs/
  5. hooks/
  6. infra/
  7. infra_lib/
  8. jj/
  9. man/
  10. mcp/
  11. metadata/
  12. python-bin/
  13. python2-bin/
  14. recipes/
  15. testing_support/
  16. tests/
  17. third_party/
  18. win_toolchain/
  19. zsh-goodies/
  20. .cipd_impl.ps1
  21. .flake8
  22. .git-blame-ignore-revs
  23. .gitattributes
  24. .gitignore
  25. .isort.cfg
  26. .style.yapf
  27. .vpython3
  28. android_build_server_helper.py
  29. auth.py
  30. autoninja
  31. autoninja.bat
  32. autoninja.py
  33. bb
  34. bb.bat
  35. black
  36. bootstrap_python3
  37. breakpad.py
  38. BUILD_OWNERS
  39. build_telemetry
  40. build_telemetry.bat
  41. build_telemetry.py
  42. caffeinate.py
  43. cipd
  44. cipd.bat
  45. cipd_bin_setup.bat
  46. cipd_bin_setup.sh
  47. cipd_client_version
  48. cipd_client_version.digests
  49. cipd_manifest.txt
  50. cipd_manifest.versions
  51. cipd_manifest_cros_python2.txt
  52. cipd_manifest_cros_python2.versions
  53. clang-format
  54. clang-format.bat
  55. clang_format.py
  56. codereview.settings
  57. compile_single_file
  58. compile_single_file.bat
  59. compile_single_file.py
  60. cpplint.bat
  61. cpplint.py
  62. cpplint_chromium.py
  63. create_temp_file.py
  64. cros
  65. CROS_OWNERS
  66. cros_python2.vpython
  67. detect_host_arch.py
  68. DIR_METADATA
  69. dirmd
  70. dirmd.bat
  71. download_from_google_storage
  72. download_from_google_storage.bat
  73. download_from_google_storage.py
  74. ensure_bootstrap
  75. fetch
  76. fetch.bat
  77. fetch.py
  78. gclient
  79. gclient-new-workdir.py
  80. gclient.bat
  81. gclient.py
  82. gclient_completion.sh
  83. gclient_eval.py
  84. gclient_paths.py
  85. gclient_scm.py
  86. gclient_utils.py
  87. gerrit_cache.py
  88. gerrit_client.py
  89. gerrit_util.py
  90. git-cache
  91. git-cl
  92. git-credential-luci
  93. git-credential-luci.bat
  94. git-crrev-parse
  95. git-drover
  96. git-find-releases
  97. git-footers
  98. git-freeze
  99. git-gs
  100. git-hyper-blame
  101. git-map
  102. git-map-branches
  103. git-mark-merge-base
  104. git-nav-downstream
  105. git-nav-upstream
  106. git-new-branch
  107. git-number
  108. git-rebase-update
  109. git-rename-branch
  110. git-reparent-branch
  111. git-retry
  112. git-runhooks
  113. git-squash-branch
  114. git-squash-branch-tree
  115. git-thaw
  116. git-upstream-diff
  117. git_auth.py
  118. git_cache.py
  119. git_cl.py
  120. git_cl_completion.sh
  121. git_common.py
  122. git_completion.sh
  123. git_dates.py
  124. git_find_releases.py
  125. git_footers.py
  126. git_freezer.py
  127. git_hyper_blame.py
  128. git_map.py
  129. git_map_branches.py
  130. git_mark_merge_base.py
  131. git_nav_downstream.py
  132. git_new_branch.py
  133. git_number.py
  134. git_rebase_update.py
  135. git_rename_branch.py
  136. git_reparent_branch.py
  137. git_retry.py
  138. git_squash_branch.py
  139. git_squash_branch_tree.py
  140. git_upstream_diff.py
  141. gn
  142. gn.bat
  143. gn.py
  144. gn_helper.py
  145. google-java-format
  146. google-java-format.bat
  147. google_java_format.py
  148. gsutil.py
  149. gsutil.py.bat
  150. gsutil.py.vpython3
  151. isort
  152. led
  153. led.bat
  154. LICENSE
  155. lockfile.py
  156. luci-auth
  157. luci-auth-fido2-plugin.bat
  158. luci-auth-ssh-helper
  159. luci-auth-ssh-helper.bat
  160. luci-auth-ssh-plugin
  161. luci-auth-ssh-plugin.bat
  162. luci-auth.bat
  163. luci_auth_fido2_plugin.py
  164. LUCI_OWNERS
  165. lucicfg
  166. lucicfg.bat
  167. mac_toolchain
  168. markdown_format.py
  169. metrics-xml-format
  170. metrics.py
  171. metrics.README.md
  172. metrics_utils.py
  173. metrics_xml_format.py
  174. my_activity.py
  175. newauth.py
  176. ninja
  177. ninja.bat
  178. ninja.py
  179. ninjalog_uploader.py
  180. OWNERS
  181. owners_client.py
  182. owners_finder.py
  183. pinpoint
  184. pinpoint.bat
  185. post_build_ninja_summary.py
  186. PRESUBMIT.py
  187. presubmit_canned_checks.py
  188. presubmit_diff.py
  189. presubmit_support.py
  190. prpc
  191. prpc.bat
  192. pylint-2.17
  193. pylint-2.17.bat
  194. pylint-2.6
  195. pylint-2.6.bat
  196. pylint-2.7
  197. pylint-2.7.bat
  198. pylint-3.2
  199. pylint-3.2.bat
  200. pylint_main.py
  201. pylintrc-2.17
  202. pylintrc-2.6
  203. pylintrc-2.7
  204. pylintrc-3.2
  205. pyproject.toml
  206. python_runner.sh
  207. rdb
  208. rdb.bat
  209. rdb_wrapper.py
  210. README.gclient.md
  211. README.git-cl.md
  212. README.google_cloud_storage.md
  213. README.md
  214. reclient_helper.py
  215. RECLIENT_OWNERS
  216. reclientreport
  217. reclientreport.bat
  218. reclientreport.py
  219. repo
  220. repo_launcher
  221. roll-dep
  222. roll-dep.bat
  223. roll_dep.py
  224. roll_downstream_gcs_deps.py
  225. rustfmt.py
  226. scm.py
  227. setup_color.py
  228. siso
  229. siso.bat
  230. siso.py
  231. split_cl.py
  232. subcommand.py
  233. subprocess2.py
  234. swift-format
  235. swift_format.py
  236. update_depot_tools
  237. update_depot_tools.bat
  238. update_depot_tools_toggle.py
  239. upload_metrics.py
  240. upload_to_google_storage.py
  241. upload_to_google_storage_first_class.py
  242. utils.py
  243. vpython3
  244. vpython3.bat
  245. WATCHLISTS
  246. watchlists.py
  247. whitespace.txt
  248. win32imports.py
  249. yapf
  250. yapf.bat
README.md

depot_tools

Tools for Chromium development. Requires Python 3.8+.

gsutil: Python 3.9–3.13 required.

Tools

The most important tools are:

  • fetch: A gclient wrapper to checkout a project. Use fetch --help for more details.
  • gclient: A meta-checkout tool. Think repo or git submodules, except that it support OS-specific rules, e.g. do not checkout Windows only dependencies when checking out for Android. Use gclient help for more details and README.gclient.md.
  • git cl: A code review tool to interact with Rietveld or Gerrit. Use git cl help for more details and README.git-cl.md.
  • roll-dep: A gclient dependency management tool to submit a dep roll, updating a dependency to a newer revision.

There are a lot of git utilities included.

Also, includes shell script/batch file for tools required to build chromium, e.g.

  • gn: a meta-build system that generates build files for Ninja
  • autoninja: a wrapper for siso and ninja.
  • siso: a build tool that aims to significantly speed up Chromium's build.
  • ninja: a small build system with a focus on speed. deprecated by Siso.

These shell script/batch file runs python script with python-bin/python3 that find binaries in chromium checkout, and run with proper setup/check. To use these wrappers, you need to initialize/bootstrap depot_tools (using gclient, update_depot_tools or ensure_bootstrap).

Installing

See set-up documentation.

depot_tools is also available in

These depot_tools would not be initialized/bootstrapped (i.e. no python-bin/python3 binary available), so the build tool wrapper won't work, unless it is explicitly initialized by ensure_bootstrap. Or, directly call the python script instead of using the shell script/batch file.

Updating

depot_tools updates itself automatically when running gclient tool. To disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0 or run ./update_depot_tools_toggle.py --disable.

To update package manually, run update_depot_tools.bat on Windows, or ./update_depot_tools on Linux or Mac.

Running gclient will install python3 binary.

Contributing

To contribute change for review:

git new-branch <somename>
# Hack
git add .
git commit -a -m "Fixes goat teleporting"
# find reviewers
git cl owners
git log -- <yourfiles>

# Request a review.
git cl upload -r reviewer1@chromium.org,reviewer2@chromium.org --send-mail

# Edit change description if needed.
git cl desc

# If change is approved, flag it to be committed.
git cl set-commit

# If change needs more work.
git rebase-update
...
git cl upload -t "Fixes goat teleporter destination to be Australia"

See also open bugs, open reviews, forum or report problems.

cpplint.py

Until 2018, our cpplint.py was a copy of the upstream version at https://github.com/google/styleguide/tree/gh-pages/cpplint. Unfortunately, that repository is not maintained any more. If you want to update cpplint.py in depot_tools, just upload a patch to do so. We will figure out a long-term strategy via issue https://crbug.com/916550.