Google Git
Sign in
chromium / external / github.com / luci / luci-go / 18ae6bdbc6652e10610d55e4528d4bb3c624f813
commit18ae6bdbc6652e10610d55e4528d4bb3c624f813[log] [tgz]
authordnj <dnj@chromium.org>Thu Aug 03 21:14:18 2017
committerCommit Bot <commit-bot@chromium.org>Thu Aug 03 21:14:18 2017
tree7cc4f0fe7c3b9e728168dab7a418ef71aca8acbf
parentda9d004ddcf295086e66df450f6779839259be9c [diff]
[logdog] Replace Tumble with push queues.

Replace Tumble with push queues for archival.

Currently, LogDog uses the Tumble journaling state machine to manage its
archival. Tumble is generally overkill for this one-state task, but was
chosen because it seemed, at the time, likely that it would be used
everywhere in LUCI.

Almost two years later, LogDog is the only production major user of
Tumble. Since it barely scrapes the power of Tumble, and since Tumble
itself is rather opaque in its operations, this trade-off is not
worthwhile. Instead, we replace Tumble with task queues.

When a log stream is registered, an "expired" task will be enqueued to
handle it once the stream expires (if it never gets terminated). When
the stream is terminated, the expiration task is deleted, replaced with
a shorter-term archival task.

We leave Tumble and its mutation handling in-place because, in
production, there is still a Tumble backlog to process through. This
should be empited within a few days, and we can finish the removal.

BUG=chromium:751925
TEST=unit

Review-Url: https://codereview.chromium.org/2989333002
23 files changed
tree: 7cc4f0fe7c3b9e728168dab7a418ef71aca8acbf
  1. appengine/
  2. buildbucket/
  3. cipd/
  4. client/
  5. common/
  6. deploytool/
  7. dm/
  8. examples/
  9. grpc/
  10. hardcoded/
  11. infra/
  12. logdog/
  13. luci_config/
  14. lucictx/
  15. milo/
  16. mmutex/
  17. scheduler/
  18. scripts/
  19. server/
  20. swarming/
  21. tokenserver/
  22. tools/
  23. tumble/
  24. vpython/
  25. web/
  26. .travis.yml
  27. AUTHORS
  28. codereview.settings
  29. CONTRIBUTING.md
  30. CONTRIBUTORS
  31. LICENSE
  32. luci-deploy.cfg
  33. pre-commit-go.yml
  34. PRESUBMIT.py
  35. README.md
  36. WATCHLISTS
README.md

luci-go: LUCI services and tools in Go

GoDoc Build Status Coverage Status

Installing

go get -u github.com/luci/luci-go/client/cmd/...
go get -u github.com/luci/luci-go/server/cmd/...

Code layout

  • /appengine/... contains AppEngine server code. It imports packages from /common/... and /server/....
  • /client/... contains all client code.
  • /common/... contains code and structures shared between all of /appengine/..., /client/... and /server/...; for example, the structures used by the server APIs. These are inherently APIs.
  • /deploytool/... contains the LUCI cloud services deployment tool.
  • /logdog/... contains LogDog client and server code, as well as APIs, protobufs, and support libraries.
  • /server/... contains standalone server code. Its packages are reusable by /appengine/....
  • /tools/... contains support tools used by other LUCI components.

Versioning

  • Branch go1 contains the stable code.
  • Branch master constains the latest code.

Contributing

  • Sign the Google CLA.
  • Make sure your user.email and user.name are configured in git config.
  • Install test-only packages: go get -u -t github.com/luci/luci-go/client/...
  • Install the pcg git hook: go get -u github.com/maruel/pre-commit-go/cmd/... && pcg

Run the following to setup the code review tool and create your first review:

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git $HOME/src/depot_tools
export PATH="$PATH:$HOME/src/depot_tools"
cd $GOROOT/github.com/luci/luci-go
git checkout -b work origin/master

# hack hack

git commit -a -m "This is awesome\nR=joe@example.com"
# This will ask for your Google Account credentials.
git cl upload -s
# Wait for LGTM over email.
# Check the commit queue box in codereview website.
# Wait for the change to be tested and landed automatically.

Use git cl help and git cl help <cmd> for more details.