check_ethernet: defer action if recently (un)locked

See $BUG for more info. When a test releases this lock, it doesn't
guarantee that upstream connectivity is completely restored; it only
guarantees that the test has finished with any of its destructive
actions (e.g., it has finished restarting Shill; it has finished
resetting its IP address; it has finished its suspend/resume cycle). The
DUT could still be renegotiating DHCP, for instance.

Tests will be updated to update the file modification time before
releasing the lock, and so we can use that to determine whether a test
may have recently disturbed connectivity. If an actor has disturbed
connectivity in the last 30 seconds, we skip any attempt at
detection/recovery, and just wait for the next time around.

BUG=chromium:1083044
TEST=run a network test that stops shill, in a loop, while
     simultaneously running check_ethernet.hook in a tight loop;
     we expect not to see check_ethernet.hook triggering any recovery
     actions (and so, never see it log "successful after XX seconds")
     `while : ; do tast -verbose run -failfortests \
        ${HOST} network.DefaultProfile || break; done`
     on DUT:
     `stop recover_duts
      while : ; do
        /usr/local/libexec/recover-duts/hooks/check_ethernet.hook 2>&1 | \
        tee /dev/tty | grep -q 'successful after [0-9]* seconds' && break
      done`

Cq-Depend: chromium:2293100
Change-Id: Ifc25e33347f0cfbe300a66c3081a76c0088c4aa9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crostestutils/+/2293019
Tested-by: Brian Norris <briannorris@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
Reviewed-by: Matthew Wang <matthewmwang@chromium.org>
Commit-Queue: Brian Norris <briannorris@chromium.org>
1 file changed