[client] lazy hash and improve symlink processing

- Stop hashing in file_to_metadata(). The hashing must be delayed for
  tarballs, as only the hash of the whole tarball is needed, not the
  hash of each of its individual files.
- Small performance improvement for symlinks, to not call stat() twice
  on them. This is still the case for other files. :/
- Make isolate_storage.Item more 'immutable'. This is to help with data
  flow.
- Make isolate_storage.Item know how to calculate its own digest. This
  is needed for tarfiles, as they won't be written to disk, nor be fully
  kept in memory. This removes the need for prepare().
- As a small performance improvement, do not saves isolated files
  created for directory to disk anymore. This should be fine even on
  32-bit process, as the size and number of isolated files is generally
  small.

This code is the final (!) preparation for adding tarball support to
python isolated upload.

Bug: 825418
Change-Id: I9e68cb2b2b9cf7bd9c3d099c32168f0414b3c14a
Reviewed-on: https://chromium-review.googlesource.com/c/1358673
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Mirrored-From: https://chromium.googlesource.com/infra/luci/luci-py
Cr-Mirrored-Commit: 7560565e15538cfca0387172aeb4123110cc20a1
6 files changed
tree: 0dc5b2a490b4fa68e04aadaa0f62473f1378f0a3
  1. example/
  2. libs/
  3. proto/
  4. tests/
  5. third_party/
  6. tools/
  7. utils/
  8. .gitignore
  9. auth.py
  10. cipd.py
  11. isolate.py
  12. isolate_format.py
  13. isolate_storage.py
  14. isolated_format.py
  15. isolateserver.py
  16. LICENSE
  17. local_caching.py
  18. LUCI_CONTEXT.md
  19. OWNERS
  20. PRESUBMIT.py
  21. README.md
  22. README.py
  23. run_isolated.py
  24. swarming.py
  25. trace_inputs.py
  26. WATCHLISTS
README.md

LUCI Python Client

This is the Python Client code for LUCI. It's part of the main python repo, and is also mirrored into a standalone client-py repo.

To contribute changes you need to clone the main python repo and then use git cl upload. The client-py repo mirror (which appears in Chromium's third_party directory) cannot be contributed to directly.

License

This project is licensed under Apache v2.0 license. See LICENSE for details.