[gitsource] Add fetcher implementation.

Adds the main 'fetcher' implementation. This will be used as the
primary interface for obtaining blob data from the remote.

This implements multiple prefetch optimizations - one which will
fire based on a callback which can process the tree of the target
commit to pull objects under the target subdirectory which look like
the right files to get, and a second where we can specifically
ensure that a batch of arbitrary objects is cached.

R=fancl, fancl@chromium.org, vadimsh@chromium.org

Bug: 367699933
Change-Id: I21ef7e588939675e85babfdd499076c4254a215c
Reviewed-on: https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/6313604
Auto-Submit: Robbie Iannucci <iannucci@google.com>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@google.com>
16 files changed
tree: aab4c728573eff40fbebe3cf82c762302b6276e3
  1. analysis/
  2. appengine/
  3. auth/
  4. auth_service/
  5. bisection/
  6. build/
  7. buildbucket/
  8. casviewer/
  9. cipd/
  10. cipkg/
  11. client/
  12. common/
  13. config/
  14. config_service/
  15. cv/
  16. deploy/
  17. examples/
  18. gae/
  19. gce/
  20. grpc/
  21. hardcoded/
  22. led/
  23. logdog/
  24. luci_notify/
  25. lucicfg/
  26. lucictx/
  27. luciexe/
  28. mailer/
  29. milo/
  30. mmutex/
  31. provenance/
  32. recipes_py/
  33. resultdb/
  34. scheduler/
  35. scripts/
  36. server/
  37. source_index/
  38. standalone/
  39. starlark/
  40. swarming/
  41. teams/
  42. third_party/
  43. tokenserver/
  44. tools/
  45. tree_status/
  46. vpython/
  47. web/
  48. .gitallowed
  49. .gitignore
  50. .go-lintable
  51. AUTHORS
  52. codereview.settings
  53. CONTRIBUTING.md
  54. CONTRIBUTORS
  55. go.mod
  56. go.sum
  57. LICENSE
  58. OWNERS
  59. PRESUBMIT.py
  60. README.md
  61. staticcheck.conf
  62. tools.go
  63. WATCHLISTS
README.md

luci-go: LUCI services and tools in Go

GoReference

Installing

LUCI Go code is meant to be worked on from an Chromium infra.git checkout, which enforces packages versions and Go toolchain version. First get fetch via depot_tools.git then run:

fetch infra
cd infra/go
eval `./env.py`
cd src/go.chromium.org/luci

It is now possible to directly install tools with go install:

go install go.chromium.org/luci/auth/client/cmd/...@latest
go install go.chromium.org/luci/buildbucket/cmd/...@latest
go install go.chromium.org/luci/cipd/client/cmd/...@latest
go install go.chromium.org/luci/client/cmd/...@latest
go install go.chromium.org/luci/cv/cmd/...@latest
go install go.chromium.org/luci/gce/cmd/...@latest
go install go.chromium.org/luci/grpc/cmd/...@latest
go install go.chromium.org/luci/logdog/client/cmd/...@latest
go install go.chromium.org/luci/luci_notify/cmd/...@latest
go install go.chromium.org/luci/lucicfg/cmd/...@latest
go install go.chromium.org/luci/luciexe/legacy/cmd/...@latest
go install go.chromium.org/luci/mailer/cmd/...@latest
go install go.chromium.org/luci/mmutex/cmd/...@latest
go install go.chromium.org/luci/resultdb/cmd/...@latest
go install go.chromium.org/luci/server/cmd/...@latest
go install go.chromium.org/luci/swarming/cmd/...@latest
go install go.chromium.org/luci/tokenserver/cmd/...@latest
go install go.chromium.org/luci/tools/cmd/...@latest

Contributing

Contributing uses the same flow as Chromium contributions.