Try using depot_tools' gsutil.py in catapults's GS lib gsutil is being migrated to the gcloud sdk: https://docs.cloud.google.com/storage/docs/gsutil-transition-to-gcloud Likely due to that issue, it doesn't seem to be possible to generate static .boto files any longer on dev workstations. Nor is the vendored gsutil in catapult able to pick up the creds dropped by the gcloud sdk when authed with it. All of this is solved by the gsutil.py in depot_tools, since it magically inserts `luci-auth context ...` calls into the gsutil invocation: https://source.chromium.org/chromium/chromium/src/+/main:third_party/depot_tools/gsutil.py So this CL just bypasses catapult's vendored gsutil and uses depot_tools if depot_tools is on PATH. Bug: chromium:492147922 Change-Id: I660401f9c1b5329987fffba009586071ab874d1e Reviewed-on: https://chromium-review.googlesource.com/c/catapult/+/7667499 Reviewed-by: Brian Sheedy <bsheedy@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org>
diff --git a/common/py_utils/py_utils/cloud_storage.py b/common/py_utils/py_utils/cloud_storage.py index 3a36a57..1655712 100644 --- a/common/py_utils/py_utils/cloud_storage.py +++ b/common/py_utils/py_utils/cloud_storage.py
@@ -49,6 +49,16 @@ _GSUTIL_PATH = os.path.join(py_utils.GetCatapultDir(), 'third_party', 'gsutil', 'gsutil') +# The gsutil vendored in catapult is unable to authenticate downloads on +# user machines due to +# https://docs.cloud.google.com/storage/docs/gsutil-transition-to-gcloud. +# The version of gsutil.py in depot_tools has fixes to overcome these issues. +# Long-term, this should probably switch to using gcloud SDK. +_gsutil_from_path = shutil.which('gsutil.py') +if (_gsutil_from_path + and os.path.basename(os.path.dirname(_gsutil_from_path)) == 'depot_tools'): + _GSUTIL_PATH = _gsutil_from_path + # TODO(tbarzic): A workaround for http://crbug.com/386416 and # http://crbug.com/359293. See |_RunCommand|. _CROS_GSUTIL_HOME_WAR = '/home/chromeos-test/'