[swarming] Refactor `monitor` implementation.

This just moves code around and adds interface boundaries.
The only non-trivial change is parallel `Finalize(...)`
calls (will be important later).

The primary motivation is to make `monitor` more extendable.
In particular, there are 4 mechanisms in Swarming that need to
periodically visit every bot. Instead of 4 separate crons each
issuing datastore queries, we can have one query with 4
"visitors" passed to it. Existing `report-bots` cron will do
that.

Refactor active tasks scanner to be structured the same way
for consistency (it isn't hard). There's currently no plan to
extend it though.

Another goal of this refactoring is to move code out of
the giant main.go and make it more modular and testable.
All missing tests are added as well.

One minor logical change is dropping of tsmon.State after every
flush instead of resetting it. We were resetting all metrics
reported to the state, this is equivalent to dropping the state
and starting with a new fresh state.

R=chanli@chromium.org

Change-Id: I15380f4e102eaced28208f424e974e6f6feddd99
Reviewed-on: https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/5671720
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
14 files changed
tree: 4409cbba314e19dc8374ad9422a54cd89f028861
  1. analysis/
  2. appengine/
  3. auth/
  4. auth_service/
  5. bisection/
  6. build/
  7. buildbucket/
  8. casviewer/
  9. cipd/
  10. cipkg/
  11. client/
  12. common/
  13. config/
  14. config_service/
  15. cv/
  16. deploy/
  17. examples/
  18. gae/
  19. gce/
  20. grpc/
  21. hardcoded/
  22. led/
  23. logdog/
  24. luci_notify/
  25. lucicfg/
  26. lucictx/
  27. luciexe/
  28. mailer/
  29. milo/
  30. mmutex/
  31. provenance/
  32. resultdb/
  33. scheduler/
  34. scripts/
  35. server/
  36. standalone/
  37. starlark/
  38. swarming/
  39. teams/
  40. third_party/
  41. tokenserver/
  42. tools/
  43. tree_status/
  44. vpython/
  45. web/
  46. .gitallowed
  47. .gitattributes
  48. .gitignore
  49. .go-lintable
  50. AUTHORS
  51. codereview.settings
  52. CONTRIBUTING.md
  53. CONTRIBUTORS
  54. go.mod
  55. go.sum
  56. LICENSE
  57. OWNERS
  58. PRESUBMIT.py
  59. README.md
  60. tools.go
  61. WATCHLISTS
README.md

luci-go: LUCI services and tools in Go

GoReference

Installing

LUCI Go code is meant to be worked on from an Chromium infra.git checkout, which enforces packages versions and Go toolchain version. First get fetch via depot_tools.git then run:

fetch infra
cd infra/go
eval `./env.py`
cd src/go.chromium.org/luci

It is now possible to directly install tools with go install:

go install go.chromium.org/luci/auth/client/cmd/...@latest
go install go.chromium.org/luci/buildbucket/cmd/...@latest
go install go.chromium.org/luci/cipd/client/cmd/...@latest
go install go.chromium.org/luci/client/cmd/...@latest
go install go.chromium.org/luci/cv/cmd/...@latest
go install go.chromium.org/luci/gce/cmd/...@latest
go install go.chromium.org/luci/grpc/cmd/...@latest
go install go.chromium.org/luci/logdog/client/cmd/...@latest
go install go.chromium.org/luci/luci_notify/cmd/...@latest
go install go.chromium.org/luci/lucicfg/cmd/...@latest
go install go.chromium.org/luci/luciexe/legacy/cmd/...@latest
go install go.chromium.org/luci/mailer/cmd/...@latest
go install go.chromium.org/luci/mmutex/cmd/...@latest
go install go.chromium.org/luci/resultdb/cmd/...@latest
go install go.chromium.org/luci/server/cmd/...@latest
go install go.chromium.org/luci/swarming/cmd/...@latest
go install go.chromium.org/luci/tokenserver/cmd/...@latest
go install go.chromium.org/luci/tools/cmd/...@latest

Contributing

Contributing uses the same flow as Chromium contributions.