cros_build_lib: Wait for LV to appear before mounting

Sometimes after running lvchange, the device takes a few seconds to
appear.  Check for the device and retry a couple of times as needed to
wait for it.  This is similar to the logic that was previously added to
wait for vgchange.

BUG=chromium:809570
TEST=cros tryjob edgar-paladin-tryjob

Change-Id: I77648232a7af36285253f5267ae2f90c4dfad063
Reviewed-on: https://chromium-review.googlesource.com/904731
Commit-Ready: Benjamin Gordon <bmgordon@chromium.org>
Tested-by: Benjamin Gordon <bmgordon@chromium.org>
Reviewed-by: Benjamin Gordon <bmgordon@chromium.org>
diff --git a/lib/cros_build_lib.py b/lib/cros_build_lib.py
index 1c1ffc0..6b32e32 100644
--- a/lib/cros_build_lib.py
+++ b/lib/cros_build_lib.py
@@ -59,6 +59,10 @@
 CHROOT_THINPOOL_NAME = 'thinpool'
 
 
+# Max times to recheck the result of an lvm command that doesn't finish quickly.
+_MAX_LVM_RETRIES = 3
+
+
 def ShellQuote(s):
   """Quote |s| in a way that is safe for use in a shell.
 
@@ -1644,6 +1648,21 @@
     SudoRunCommand(cmd, capture_output=True, print_cmd=False)
 
   osutils.SafeMakedirsNonRoot(chroot)
+
+  # Sometimes lvchange can take a few seconds to run.  Try to wait for the
+  # device to appear before mounting it.
+  count = 0
+  while not os.path.exists(chroot_dev_path):
+    if count > _MAX_LVM_RETRIES:
+      logging.error('Device %s still does not exist.  Expect mounting the '
+                    'filesystem to fail.', chroot_dev_path)
+      break
+
+    count += 1
+    logging.warning('Device file %s does not exist yet on try %d.',
+                    chroot_dev_path, count)
+    time.sleep(1)
+
   cmd = ['mount', '-text4', '-onoatime', chroot_dev_path, chroot]
   SudoRunCommand(cmd, capture_output=True, print_cmd=False)