[client] Further use generator during enumeration

Leverage that TaskChannel can now be used as a multithreaded python generator.

This permits to send inputs to /contains RPC before the full isolated is
rasterized, which helps with reducing overall latency.

This change, coupled with the 3 previous ones results in 10 seconds saving on a
Windows 7 VM on a 2658 files/639MiB dataset.

The tarring should live between expand_directories_and_symlinks() and
_directory_to_metadata().

Bug: 825418
Change-Id: I9321250ade2a3838a7009351fc06295ad1e23399
Reviewed-on: https://chromium-review.googlesource.com/c/1349376
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Mirrored-From: https://chromium.googlesource.com/infra/luci/luci-py
Cr-Mirrored-Commit: 637736ab96e7bc28d3f8bcce4e96a291869a03fc
6 files changed
tree: d0362f9c4d6c2453d1bcb7dfd94590d3989d23a2
  1. example/
  2. libs/
  3. proto/
  4. tests/
  5. third_party/
  6. tools/
  7. utils/
  8. .gitignore
  9. auth.py
  10. cipd.py
  11. isolate.py
  12. isolate_format.py
  13. isolate_storage.py
  14. isolated_format.py
  15. isolateserver.py
  16. LICENSE
  17. local_caching.py
  18. LUCI_CONTEXT.md
  19. OWNERS
  20. PRESUBMIT.py
  21. README.md
  22. README.py
  23. run_isolated.py
  24. swarming.py
  25. trace_inputs.py
  26. WATCHLISTS
README.md

LUCI Python Client

This is the Python Client code for LUCI. It's part of the main python repo, and is also mirrored into a standalone client-py repo.

To contribute changes you need to clone the main python repo and then use git cl upload. The client-py repo mirror (which appears in Chromium's third_party directory) cannot be contributed to directly.

License

This project is licensed under Apache v2.0 license. See LICENSE for details.