siso: remove a stale non-directory ancestor when flush creates parent dirs

A previous build can leave a regular file at a path where the current
build needs a directory. Flush already handles this type mismatch for
the flushed path itself, but not for its ancestors. When a member is
flushed on its own - the step producing its parent directory was
skipped this build, and neither ClearStaleFileForDirOutput nor
ensureActionOutputDirs touched the path at plan time - MkdirAll
failed with ENOTDIR against the stale file.

Replace the bare MkdirAll in flushRegularFile and flushDir with
mkdirAllForFlush. On failure it walks up to the first existing path
component, removes it if it is not a directory, and retries. The
recorded entry below the path proves that every ancestor must be a
directory, and at most one non-directory can exist on the path, so a
single removal is enough.

Symlinks need care. A symlink that resolves to a directory is a valid
path component and is kept. A symlink is removed only on positive
proof that it is wrong: it resolves to a non-directory, or it is
dangling. If its target cannot be statted at all (EACCES, EIO), the
flush fails instead of removing what may be an intentional symlink.

The removal and retry run under a mutex. Flush materializes outputs
concurrently, and two outputs below the same stale file must not both
act on their own Lstat: one would recreate the directory and write
its output while the other still sees the stale file, and its
RemoveAll would then delete the sibling's fresh output.

Change-Id: I3c6f4641e4b185e9d0ff87db8c740420ab97dab6
Reviewed-on: https://chromium-review.googlesource.com/c/build/+/8037826
Auto-Submit: Philipp Wollermann <philwo@google.com>
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Neri Marschik <nerima@google.com>
Reviewed-by: Yuke Liao <liaoyuke@google.com>
2 files changed
tree: ffe9afa47ce3b6c628e62760e0051362961bd1b5
  1. bench/
  2. gong/
  3. hashigo/
  4. infra/
  5. kajiya/
  6. kzipinfo/
  7. remote-apis/
  8. runmc/
  9. siso/
  10. .gitignore
  11. .golangci.yml
  12. .style.yapf
  13. BUILD_OWNERS
  14. BUILD_TEAM_OWNERS
  15. CONTRIBUTING.md
  16. LICENSE
  17. OWNERS
  18. PRESUBMIT.py
  19. README.md
  20. WATCHLISTS
README.md

build.git repository

This repository contains tools developed and owned by the Chrome Build Team.

Quick start

The steps for getting the code are:

  1. Install depot_tools
  2. Run git clone https://chromium.googlesource.com/build

We use the standard Go module workflow to work on our projects.

Links