commit 2280219f647e62fcc435017c500329e922455ea2
author Vadim Shtayura <vadimsh@chromium.org> Thu May 02 20:53:31 2024
committer LUCI CQ <infra-scoped@luci-project-accounts.iam.gserviceaccount.com> Thu May 02 20:53:31 2024
tree ddb5fca41b2a9ea7b4cf05c4fdbe183d6565fe8e
parent e7fe336166fddc9feca7c497117bff6d2f411cc3

[swarming] Add a mechanism to switch BQ exports Python => Go.

Each BQ export task (either Python or Go) has a timestamp of
the start of the time interval it exports.

We'll have an entity with the time when the switch should happen.
This entity will be read by both Python and Go task queue tasks
that do actual exports.

In the initial state (when the entity is missing or the exported
time range starts before designated switch time), Python TQ tasks
will do the export and Go tasks will just silently do nothing.

Once timestamp of export tasks cross the threshold, Python tasks
will start doing nothing and Go tasks will do actual exports.

As long as the time threshold is not inside of any exported
interval, we should have no gaps. Luckily both Python and Go
exporters align their export intervals on 1m grid, so as long
as we switch at an exact minute, there will be no gaps.

Such a separate entity allows to decouple deployment from the actual
switch. We'll deploy new Go code and pass correct CLI flags to
configure the export first. It will be running in noop mode
essentially, but we'll be able to confirm configs look good etc.

Once it is fully deploy and unlikely to be rolled back due to some
other unrelated reason, we'll create the entity to schedule the
switchover. The entity will be created semi-manually by modifying
datastore from a workstation by calling SetMigrationState.

R=chanli@chromium.org

Change-Id: Ica2d4dc34d82bc4b7bf7fe161fd6ad0c3763bd0a
Reviewed-on: https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/5508235
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Chan Li <chanli@chromium.org>