Repo documentation for depot_tools

bot_update — Recipe module to ensure a checkout is consistent on a bot.
depot_tools — The depot_tools module provides safe functions to access paths within the depot_tools repo.
gclient
gerrit
git
git_cl
gitiles
gsutil
osx_sdk — The osx_sdk module provides safe functions to access a semi-hermetic XCode installation.
presubmit
tryserver
windows_sdk — The windows_sdk module provides safe functions to access a hermetic Microsoft Visual Studio installation which is derived from Chromium's MSVC toolchain.

bot_update:examples/full
bot_update:tests/do_not_retry_patch_failures_in_cq
bot_update:tests/download_topics
bot_update:tests/ensure_checkout
bot_update:tests/ensure_checkout_return_custom_result
depot_tools:examples/full
gclient:examples/full
gclient:tests/diff_deps
gclient:tests/patch_project
gclient:tests/sync_failure
gerrit:examples/full
git:examples/full
git:tests/number
git_cl:examples/full
gitiles:examples/full
gitiles:tests/parse_repo_url
gsutil:examples/custom_boto
gsutil:examples/full
osx_sdk:examples/full
presubmit:examples/full
presubmit:tests/execute
presubmit:tests/prepare
tryserver:examples/full
tryserver:tests/gerrit_change_fetch_ref_timeout
tryserver:tests/gerrit_change_owner
tryserver:tests/gerrit_change_target_ref
tryserver:tests/get_change_description
tryserver:tests/get_files_affected_by_patch
tryserver:tests/get_footers
tryserver:tests/require_is_tryserver
windows_sdk:examples/full

Recipe Modules

recipe_modules / bot_update

DEPS: depot_tools, gclient, gerrit, gitiles, gsutil, tryserver, recipe_engine/archive, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/cv, recipe_engine/json, recipe_engine/led, recipe_engine/milo, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/warning

Recipe module to ensure a checkout is consistent on a bot.

class BotUpdateApi(RecipeApi):

— def __call__(self, name, cmd, **kwargs):

Wrapper for easy calling of bot_update.

— def deapply_patch(self, bot_update_result):

Deapplies a patch, taking care of DEPS and solution revisions properly.

— def ensure_checkout(self, gclient_config=None, suffix=None, patch=True, update_presentation=True, patch_root=None, with_branch_heads=False, with_tags=False, no_fetch_tags=False, refs=None, clobber=False, root_solution_revision=None, gerrit_no_reset=False, gerrit_no_rebase_patch_ref=False, assert_one_gerrit_change=True, patch_refs=None, ignore_input_commit=False, add_blamelists=False, set_output_commit=False, step_test_data=None, enforce_fetch=False, download_topics=False, recipe_revision_overrides=None, step_tags=None, **kwargs):

Args:

gclient_config: The gclient configuration to use when running bot_update. If omitted, the current gclient configuration is used.
no_fetch_tags: When true, the root git repo being checked out will not fetch any tags referenced from the references being fetched. When a repo has many references, it can become a performance bottleneck, so avoid tags if the checkout will not need them present.
ignore_input_commit: if True, ignore api.buildbucket.gitiles_commit. Exists for historical reasons. Please do not use.
add_blamelists: if True, add blamelist pins for all of the repos that had revisions specified in the gclient config.
set_output_commit: if True, mark the checked out commit as the primary output commit of this build, i.e. call api.buildbucket.set_output_gitiles_commit. In case of multiple repos, the repo is the one specified in api.buildbucket.gitiles_commit or the first configured solution. When sorting builds by commit position, this commit will be used. Requires falsy ignore_input_commit.
step_test_data: a null function that returns test bot_update.py output. Use test_api.output_json to generate test data.
enforce_fetch: Enforce a new fetch to refresh the git cache, even if the solution revision passed in already exists in the current git cache.
assert_one_gerrit_change: if True, assert that there is at most one change in self.m.buildbucket.build.input.gerrit_changes, because bot_update module ONLY supports one change. Users may specify a change via tryserver.set_change() and explicitly set this flag False.
download_topics: If True, gclient downloads and patches locally from all open Gerrit CLs that have the same topic as the tested patch ref.
recipe_revision_overrides: a dict {deps_name: revision} of revision overrides passed in from the recipe. These should be revisions unique to each particular build/recipe run. e.g. the recipe might parse a gerrit change's commit message to get this revision override requested by the author.
step_tags: a dict {tag name: tag value} of tags to add to the step

— def get_project_revision_properties(self, project_name, gclient_config=None):

Returns all property names used for storing the checked-out revision of a given project.

Args:

project_name (str): The name of a checked-out project as deps path, e.g. src or src/v8.
gclient_config: The gclient configuration to use. If omitted, the current gclient configuration is used.

Returns (list of str): All properties that'll hold the checked-out revision of the given project. An empty list if no such properties exist.

@property
— def last_returned_properties(self):

— def resolve_fixed_revision(self, bot_update_result, name):

Sets a fixed revision for a single dependency using project revision properties.

recipe_modules / depot_tools

DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/platform, recipe_engine/runtime

The depot_tools module provides safe functions to access paths within the depot_tools repo.

class DepotToolsApi(RecipeApi):

@property
— def cros_path(self):

@property
— def download_from_google_storage_path(self):

@property
— def gn_py_path(self):

@property
— def gsutil_py_path(self):

@contextlib.contextmanager
— def on_path(self):

Use this context manager to put depot_tools on $PATH.

Example:

with api.depot_tools.on_path():
  # run some steps

@property
— def presubmit_support_py_path(self):

@property
— def root(self):

Returns (Path): The “depot_tools” root directory.

@property
— def upload_to_google_storage_path(self):

recipe_modules / gclient

DEPS: git, gitiles, tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

class GclientApi(RecipeApi):

@property
— def DepsDiffException(self):

— def __call__(self, name, cmd, infra_step=True, **kwargs):

Wrapper for easy calling of gclient steps.

— def break_locks(self):

Remove all index.lock files. If a previous run of git crashed, bot was reset, etc... we might end up with leftover index.lock files.

— def checkout(self, gclient_config=None, revert=RevertOnTryserver, inject_parent_got_revision=True, extra_sync_flags=None, **kwargs):

Return a step generator function for gclient checkouts.

@staticmethod
— def config_to_pythonish(cfg):

— def diff_deps(self, cwd):

— def get_config_defaults(self):

— def get_gerrit_patch_root(self, gclient_config=None):

Returns local path to the repo where gerrit patch will be applied.

If there is no patch, returns None. If patch is specified, but such repo is not found among configured solutions or repo_path_map, returns name of the first solution. This is done solely for backward compatibility with existing tests. Please do not rely on this logic in new code. Instead, properly map a repository to a local path using repo_path_map. TODO(nodir): remove this. Update all recipe tests to specify a git_repo matching the recipe.

— def get_repo_path(self, repo_url, gclient_config=None):

Returns local path to the repo checkout given its url.

Consults cfg.repo_path_map and fallbacks to urls in configured solutions.

Returns None if not found.

@staticmethod
— def got_revision_reverse_mapping(cfg):

Returns the merged got_revision_reverse_mapping.

Returns (dict): A mapping from property name -> project name. It merges the values of the deprecated got_revision_mapping and the new got_revision_reverse_mapping.

— def inject_parent_got_revision(self, gclient_config=None, override=False):

Match gclient config to build revisions obtained from build_properties.

Args: gclient_config (gclient config object) - The config to manipulate. A value of None manipulates the module's built-in config (self.c). override (bool) - If True, will forcibly set revision and custom_vars even if the config already contains values for them.

— def resolve_revision(self, revision):

— def roll_deps(self, deps_path, dep_updates, strip_prefix_for_gitlink=None, test_data=None):

Updates DEPS file to desired revisions, and returns all requried file changes.

Args: deps_path - Path to DEPS file that will be modified. dep_updates - A map of dependencies to update (key = dependency name, value = revision). strip_prefix_for_gitlink - Prefix that will be removed when adding gitlinks. This is only useful for repositories that use use_relative_path = True. That's currently only chromium/src.

Returns: A map of all files that need to be modified (key = file path, value = file content) in addition to DEPS file itself. Note: that git submodules (gitlinks) are treated as files and content is a commit hash. Note: we expect DEPS to be in the root of the project.

— def runhooks(self, args=None, name=‘runhooks’, **kwargs):

— def set_patch_repo_revision(self, gclient_config=None):

Updates config revision corresponding to patched project.

Useful for bot_update only, as this is the only consumer of gclient‘s config revision map. This doesn’t overwrite the revision if it was already set.

@spec_alias.deleter
— def spec_alias(self):

— def sync(self, cfg, extra_sync_flags=None, **kwargs):

@use_mirror.setter
— def use_mirror(self, val):

recipe_modules / gerrit

DEPS: recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time

class GerritApi(RecipeApi):

Module for interact with Gerrit endpoints

— def __call__(self, name, cmd, infra_step=True, **kwargs):

Wrapper for easy calling of gerrit_utils steps.

— def abandon_change(self, host, change, message=None, name=None, step_test_data=None):

— def call_raw_api(self, host, path, method=None, body=None, accept_statuses=None, name=None, **kwargs):

Call an arbitrary Gerrit API that returns a JSON response.

Returns: The JSON response data.

— def create_gerrit_branch(self, host, project, branch, commit, **kwargs):

Creates a new branch from given project and commit

Returns: The ref of the branch created

— def create_gerrit_tag(self, host, project, tag, commit, **kwargs):

Creates a new tag at the given commit.

Returns: The ref of the tag created.

— def get_change_description(self, host, change, patchset, timeout=None, step_test_data=None):

Gets the description for a given CL and patchset.

Args: host: URL of Gerrit host to query. change: The change number. patchset: The patchset number.

Returns: The description corresponding to given CL and patchset.

— def get_changes(self, host, query_params, start=None, limit=None, o_params=None, step_test_data=None, **kwargs):

Queries changes for the given host.

Args:

host: URL of Gerrit host to query.
query_params: Query parameters as list of (key, value) tuples to form a query as documented here: https://gerrit-review.googlesource.com/Documentation/user-search.html#search-operators
start: How many changes to skip (starting with the most recent).
limit: Maximum number of results to return.
o_params: A list of additional output specifiers, as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#list-changes
step_test_data: Optional mock test data for the underlying gerrit client.

Returns: A list of change dicts as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#list-changes

— def get_gerrit_branch(self, host, project, branch, **kwargs):

Gets a branch from given project and commit

Returns: The revision of the branch

— def get_related_changes(self, host, change, revision=‘current’, step_test_data=None):

Queries related changes for a given host, change, and revision.

Args:

host: URL of Gerrit host to query.
change: The change-id of the change to get related changes for as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#change-id
revision: The revision-id of the revision to get related changes for as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#revision-id This defaults to current, which names the most recent patch set.
step_test_data: Optional mock test data for the underlying gerrit client.

Returns: A related changes dictionary as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#related-changes-info

— def get_revision_info(self, host, change, patchset, timeout=None, step_test_data=None):

Returns the info for a given patchset of a given change.

Args: host: Gerrit host to query. change: The change number. patchset: The patchset number.

Returns: A dict for the target revision as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#list-changes

— def move_changes(self, host, project, from_branch, to_branch, step_test_data=None):

— def restore_change(self, host, change, message=None, name=None, step_test_data=None):

— def set_change_label(self, host, change, label_name, label_value, name=None, step_test_data=None):

— def update_files(self, host, project, branch, new_contents_by_file_path, commit_msg, params=frozenset([‘status=NEW’]), cc_list=frozenset([]), submit=False, submit_later=False, step_test_data_create_change=None, step_test_data_submit_change=None):

Update a set of files by creating and submitting a Gerrit CL.

Args:

host: URL of Gerrit host to name.
project: Gerrit project name, e.g. chromium/src.
branch: The branch to land the change, e.g. main
new_contents_by_file_path: Dict of the new contents with file path as the key.
commit_msg: Description to add to the CL.
params: A list of additional ChangeInput specifiers, with format ‘key=value’.
cc_list: A list of addresses to notify.
submit: Should land this CL instantly.
submit_later: If this change has related CLs, we may want to commit them in a chain. So only set Bot-Commit+1, making it ready for submit together. Ignored if submit is True.
step_test_data_create_change: Optional mock test data for the step create gerrit change.
step_test_data_submit_change: Optional mock test data for the step submit gerrit change.

Returns: A ChangeInfo dictionary as documented here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#create-change Or if the change is submitted, here: https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#submit-change

recipe_modules / git

DEPS: depot_tools, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

class GitApi(RecipeApi):

— def __call__(self, *args, **kwargs):

Returns a git command step.

— def bundle_create(self, bundle_path, rev_list_args=None, **kwargs):

Runs ‘git bundle create’ on a Git repository.

Args:

bundle_path (Path): The path of the output bundle.
refs (list): The list of refs to include in the bundle. If None, all refs in the Git checkout will be bundled.
kwargs: Forwarded to ‘call’.

— def cat_file_at_commit(self, file_path, commit_hash, remote_name=None, **kwargs):

Outputs the contents of a file at a given revision.

— def checkout(self, url, ref=None, dir_path=None, recursive=False, submodules=True, submodule_update_force=False, keep_paths=None, step_suffix=None, curl_trace_file=None, raise_on_failure=True, set_got_revision=False, remote_name=None, display_fetch_size=None, file_name=None, submodule_update_recursive=True, use_git_cache=False, progress=True, tags=False, depth=None):

Performs a full git checkout and returns sha1 of checked out revision.

Args:

url (str): url of remote repo to use as upstream
ref (str): ref to fetch and check out
dir_path (Path): optional directory to clone into
recursive (bool): whether to recursively fetch submodules or not
submodules (bool): whether to sync and update submodules or not
submodule_update_force (bool): whether to update submodules with --force
keep_paths (iterable of strings): paths to ignore during git-clean; paths are gitignore-style patterns relative to checkout_path.
step_suffix (str): suffix to add to a each step name
curl_trace_file (Path): if not None, dump GIT_CURL_VERBOSE=1 trace to that file. Useful for debugging git issue reproducible only on bots. It has a side effect of all stderr output of ‘git fetch’ going to that file.
raise_on_failure (bool): if False, ignore errors during fetch or checkout.
set_got_revision (bool): if True, resolves HEAD and sets got_revision property.
remote_name (str): name of the git remote to use
display_fetch_size (bool): if True, run git count-objects before and after fetch and display delta. Adds two more steps. Defaults to False.
file_name (str): optional path to a single file to checkout.
submodule_update_recursive (bool): if True, updates submodules recursively.
use_git_cache (bool): if True, git cache will be used for this checkout. WARNING, this is EXPERIMENTAL!!! This wasn't tested with:
- submodules
- since origin url is modified to a local path, may cause problem with scripts that do “git fetch origin” or “git push origin”.
- arbitrary refs such refs/whatever/not-fetched-by-default-to-cache progress (bool): whether to show progress for fetch or not
tags (bool): Also fetch tags.
depth (int): if > 0, limit fetching to the given number of commits from the tips of the remote tree.

Returns: If the checkout was successful, this returns the commit hash of the checked-out-repo. Otherwise this returns None.

— def config_get(self, prop_name, **kwargs):

Returns git config output.

Args:

prop_name: (str) The name of the config property to query.
kwargs: Forwarded to ‘call’.

Returns: (str) The Git config output, or None if no output was generated.

— def count_objects(self, previous_result=None, raise_on_failure=False, **kwargs):

Returns git count-objects result as a dict.

Args:

previous_result (dict): the result of previous count_objects call. If passed, delta is reported in the log and step text.
raise_on_failure (bool): if True, an exception will be raised if the operation fails. Defaults to False.

Returns: A dict of count-object values, or None if count-object run failed.

— def fetch_tags(self, remote_name=None, **kwargs):

Fetches all tags from the remote.

— def get_remote_url(self, remote_name=None, **kwargs):

Returns the remote Git repository URL, or None.

Args:

remote_name: (str) The name of the remote to query, defaults to ‘origin’.
kwargs: Forwarded to ‘call’.

Returns: (str) The URL of the remote Git repository, or None.

— def get_timestamp(self, commit=‘HEAD’, test_data=None, **kwargs):

Find and return the timestamp of the given commit.

— def new_branch(self, branch, name=None, upstream=None, upstream_current=False, **kwargs):

Runs git new-branch on a Git repository, to be used before git cl upload.

Args:

branch (str): new branch name, which must not yet exist.
name (str): step name.
upstream (str): to origin/main.
upstream_current (bool): whether to use ‘--upstream_current’.
kwargs: Forwarded to ‘call’.

— def number(self, commitrefs=None, test_values=None):

Computes the generation number of some commits.

Args:

commitrefs (list[str]): A list of commit references. If none are provided, the generation number for HEAD will be retrieved.
test_values (list[str]): A list of numbers to use as the return value during tests. It is an error if the length of the list does not match the number of commitrefs (1 if commitrefs is not provided).

Returns: A list of strings containing the generation numbers of the commits. If non-empty commitrefs was provided, the order of the returned numbers will correspond to the order of the provided commitrefs.

— def rebase(self, name_prefix, branch, dir_path, remote_name=None, **kwargs):

Runs rebase HEAD onto branch

Args:

name_prefix (str): a prefix used for the step names
branch (str): a branch name or a hash to rebase onto
dir_path (Path): directory to clone into
remote_name (str): the remote name to rebase from if not origin

recipe_modules / git_cl

DEPS: recipe_engine/context, recipe_engine/raw_io, recipe_engine/step

class GitClApi(RecipeApi):

— def get_description(self, patch_url=None, **kwargs):

DEPRECATED. Consider using gerrit.get_change_description instead.

— def issue(self, **kwargs):

— def set_default_repo_location(self, path: Optional[Path]):

Sets the working directory where git cl will run, unless cwd from the context module has been set.

If you set path to None, this will remove the default.

— def set_description(self, description, patch_url=None, **kwargs):

— def upload(self, message, upload_args=None, **kwargs):

recipe_modules / gitiles

DEPS: recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url

class Gitiles(RecipeApi):

Module for polling a git repository using the Gitiles web interface.

— def canonicalize_repo_url(self, repo_url):

Returns a canonical form of repo_url. If not recognized, returns as is.

— def commit_log(self, url, commit, step_name=None, attempts=None):

Returns: (dict) the Gitiles commit log structure for a given commit.

Args:

url (str): The base repository URL.
commit (str): The commit hash.
step_name (str): If not None, override the step name.
attempts (int): Number of times to try the request before failing.

— def download_archive(self, repository_url, destination, revision=‘refs/heads/main’):

Downloads an archive of the repo and extracts it to destination.

If the gitiles server attempts to provide a tarball with paths which escape destination, this function will extract all valid files and then raise StepFailure with an attribute StepFailure.gitiles_skipped_files containing the names of the files that were skipped.

Args:

repository_url (str): Full URL to the repository
destination (Path): Local path to extract the archive to. Must not exist prior to this call.
revision (str): The ref or revision in the repo to download. Defaults to ‘refs/heads/main’.

— def download_file(self, repository_url, file_path, branch=‘main’, step_name=None, attempts=None, **kwargs):

Downloads raw file content from a Gitiles repository.

Args:

repository_url (str): Full URL to the repository.
branch (str): Branch of the repository.
file_path (str): Relative path to the file from the repository root.
step_name (str): Custom name for this step (optional).
attempts (int): Number of times to try the request before failing.

Returns: Raw file content.

— def log(self, url, ref, limit=0, cursor=None, step_name=None, attempts=None, **kwargs):

Returns the most recent commits under the given ref with properties.

Args:

url (str): URL of the remote repository.
ref (str): Name of the desired ref (see Gitiles.refs).
limit (int): Number of commits to limit the fetching to. Gitiles does not return all commits in one call; instead paging is used. 0 implies to return whatever first gerrit responds with. Otherwise, paging will be used to fetch at least this many commits, but all fetched commits will be returned.
cursor (str or None): The paging cursor used to fetch the next page.
step_name (str): Custom name for this step (optional).

Returns: A tuple of (commits, cursor). Commits are a list of commits (as Gitiles dict structure) in reverse chronological order. The number of commits may be higher than limit argument. Cursor can be used for subsequent calls to log for paging. If None, signals that there are no more commits to fetch.

— def parse_repo_url(self, repo_url):

Returns (host, project) pair.

Returns (None, None) if repo_url is not recognized.

— def refs(self, url, step_name=‘refs’, attempts=None):

Returns a list of refs in the remote repository.

— def unparse_repo_url(self, host, project):

Generates a Gitiles repo URL. See also parse_repo_url.

recipe_modules / gsutil

DEPS: recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/step

class GSUtilApi(RecipeApi):

— def __call__(self, cmd, name=None, use_retry_wrapper=True, version=None, parallel_upload=False, multithreaded=False, infra_step=True, dry_run=False, **kwargs):

A step to run arbitrary gsutil commands.

On LUCI this should automatically use the ambient task account credentials. On Buildbot, this assumes that gsutil authentication environment variables (AWS_CREDENTIAL_FILE and BOTO_CONFIG) are already set, though if you want to set them to something else you can always do so using the env={} kwarg.

Note also that gsutil does its own wildcard processing, so wildcards are valid in file-like portions of the cmd. See ‘gsutil help wildcards’.

Args:

cmd (List[str|Path]) - Arguments to pass to gsutil. Include gsutil-level options first (see ‘gsutil help options’).
name (str) - Name of the step to use. Defaults to the first non-flag token in the cmd.
dry_run (bool): If True, don't actually run the step; just log what the step would have been.

— def cat(self, url, args=None, **kwargs):

@contextlib.contextmanager
— def configure_gsutil(self, **kwargs):

Temporarily configures the behavior of gsutil.

For the duration of its context, this method will temporarily append a custom Boto file to the BOTO_PATH env var without overwriting bbagent's BOTO_CONFIG. See https://cloud.google.com/storage/docs/boto-gsutil for possible configurations.

Args: kwargs: Every keyword arg is treated as config line in the temp Boto file.

— def copy(self, source_bucket, source, dest_bucket, dest, args=None, link_name=‘gsutil.copy’, metadata=None, unauthenticated_url=False, **kwargs):

— def download(self, bucket, source, dest, args=None, **kwargs):

— def download_url(self, url, dest, args=None, **kwargs):

@property
— def gsutil_py_path(self):

— def list(self, url, args=None, **kwargs):

— def remove_url(self, url, args=None, **kwargs):

— def signurl(self, private_key_file, bucket, dest, args=None, **kwargs):

— def stat(self, url, args=None, **kwargs):

— def upload(self, source, bucket, dest, args=None, link_name=‘gsutil.upload’, metadata=None, unauthenticated_url=False, **kwargs):

recipe_modules / osx_sdk

DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/version

The osx_sdk module provides safe functions to access a semi-hermetic XCode installation.

Available only to Google-run bots.

class OSXSDKApi(RecipeApi):

API for using OS X SDK distributed via CIPD.

@contextmanager
— def __call__(self, kind):

Sets up the XCode SDK environment.

Is a no-op on non-mac platforms.

This will deploy the helper tool and the XCode.app bundle at api.path.cache_dir / 'osx_sdk'.

To avoid machines rebuilding these on every run, set up a named cache in your cr-buildbucket.cfg file like:

caches: {
  # Cache for mac_toolchain tool and XCode.app
  name: "osx_sdk"
  path: "osx_sdk"
}

If you have builders which e.g. use a non-current SDK, you can give them a uniqely named cache:

caches: {
  # Cache for N-1 version mac_toolchain tool and XCode.app
  name: "osx_sdk_old"
  path: "osx_sdk"
}

Similarly, if you have mac and iOS builders you may want to distinguish the cache name by adding ‘_ios’ to it. However, if you're sharing the same bots for both mac and iOS, consider having a single cache and just always fetching the iOS version. This will lead to lower overall disk utilization and should help to reduce cache thrashing.

Usage: with api.osx_sdk(‘mac’): # sdk with mac build bits

with api.osx_sdk(‘ios’): # sdk with mac+iOS build bits

Args: kind (‘mac’|‘ios’): How the SDK should be configured. iOS includes the base XCode distribution, as well as the iOS simulators (which can be quite large).

Raises: StepFailure or InfraFailure.

— def initialize(self):

recipe_modules / presubmit

DEPS: bot_update, depot_tools, gclient, git, tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/cv, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/resultdb, recipe_engine/step

class PresubmitApi(RecipeApi):

— def __call__(self, *args, **kwargs):

Returns a presubmit step.

— def execute(self, bot_update_step, skip_owners=False, run_all=False):

Runs presubmit and sets summary markdown if applicable.

Args:

bot_update_step: the StepResult from a previously executed bot_update step.
skip_owners: a boolean indicating whether Owners checks should be skipped.

Returns: a RawResult object, suitable for being returned from RunSteps.

— def prepare(self, root_solution_revision=None):

Sets up a presubmit run.

This includes:

setting up the checkout w/ bot_update
locally committing the applied patch
running hooks, if requested

This expects the gclient configuration to already have been set.

Args: root_solution_revision: revision of the root solution

Returns: the StepResult from the bot_update step.

@property
— def presubmit_support_path(self):

recipe_modules / tryserver

DEPS: gerrit, git, git_cl, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/led, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

class TryserverApi(RecipeApi):

@property
— def constants(self):

@property
— def gerrit_change(self):

Returns current gerrit change, if there is exactly one.

Returns a self.m.buildbucket.common_pb2.GerritChange or None.

@property
— def gerrit_change_fetch_ref(self):

Returns gerrit patch ref, e.g. "refs/heads/45/12345/6, or None.

Populated iff gerrit_change is populated.

@property
— def gerrit_change_number(self):

Returns gerrit change patchset, e.g. 12345 for a patch ref of “refs/heads/45/12345/6”.

Populated iff gerrit_change is populated. Returns None if not populated.

@property
— def gerrit_change_owner(self):

Returns owner of the current Gerrit CL.

Populated iff gerrit_change is populated. Is a dictionary with keys like “name”.

@property
— def gerrit_change_repo_host(self):

Returns the host of the gitiles repo of the current Gerrit CL.

Populated iff gerrit_change is populated.

@property
— def gerrit_change_repo_project(self):

Returns the project of the gitiles repo of the current Gerrit CL.

Populated iff gerrit_change is populated.

@property
— def gerrit_change_repo_url(self):

Returns canonical URL of the gitiles repo of the current Gerrit CL.

Populated iff gerrit_change is populated.

@property
— def gerrit_change_review_url(self):

Returns the review URL for the active patchset.

@property
— def gerrit_change_target_ref(self):

Returns gerrit change destination ref, e.g. “refs/heads/main”.

Populated iff gerrit_change is populated.

@property
— def gerrit_patchset_number(self):

Returns gerrit change patchset, e.g. 6 for a patch ref of “refs/heads/45/12345/6”.

Populated iff gerrit_change is populated Returns None if not populated..

— def get_change_description(self):

Gets the CL description.

— def get_files_affected_by_patch(self, patch_root, report_files_via_property=None, **kwargs):

Returns list of paths to files affected by the patch.

Args:

patch_root: path relative to api.path[‘root’], usually obtained from api.gclient.get_gerrit_patch_root().
report_files_via_property: name of the output property to report the list of the files. If None (default), do not report.

Returned paths will be relative to to api.path[‘root’].

— def get_footer(self, tag, patch_text=None):

Gets a specific tag from a CL description

— def get_footers(self, patch_text=None):

Retrieves footers from the patch description.

footers are machine readable tags embedded in commit messages. See git-footers documentation for more information.

— def initialize(self):

@property
— def is_gerrit_issue(self):

Returns true iff the properties exist to match a Gerrit issue.

@property
— def is_patch_in_git(self):

@property
— def is_tryserver(self):

Returns true iff we have a change to check out.

— def normalize_footer_name(self, footer):

— def require_is_tryserver(self):

— def set_change(self, change):

Set the gerrit change for this module.

Args:

change: a self.m.buildbucket.common_pb2.GerritChange.

— def set_compile_failure_tryjob_result(self):

Mark the tryjob result as a compile failure.

— def set_invalid_test_results_tryjob_result(self):

Mark the tryjob result as having invalid test results.

This means we run some tests, but the results were not valid (e.g. no list of specific test cases that failed, or too many tests failing, etc).

— def set_patch_failure_tryjob_result(self):

Mark the tryjob result as failure to apply the patch.

— def set_subproject_tag(self, subproject_tag):

Adds a subproject tag to the build.

This can be used to distinguish between builds that execute different steps depending on what was patched, e.g. blink vs. pure chromium patches.

— def set_test_expired_tryjob_result(self):

Mark the tryjob result as a test expiration.

This means a test task expired and was never scheduled, most likely due to lack of capacity.

— def set_test_failure_tryjob_result(self):

Mark the tryjob result as a test failure.

This means we started running actual tests (not prerequisite steps like checkout or compile), and some of these tests have failed.

— def set_test_timeout_tryjob_result(self):

Mark the tryjob result as a test timeout.

This means tests were scheduled but didn't finish executing within the timeout.

@property
— def valid_footers(self):

recipe_modules / windows_sdk

DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/step

The windows_sdk module provides safe functions to access a hermetic Microsoft Visual Studio installation which is derived from Chromium's MSVC toolchain.

See (internal):

go/chromium-msvc-toolchain
go/windows-sdk-cipd-update

Available only on Google-run bots.

class WindowsSDKApi(RecipeApi):

API for using Windows SDK distributed via CIPD.

@contextmanager
— def __call__(self, path=None, version=None, enabled=True, target_arch=‘x64’):

Sets up the SDK environment when enabled.

Args:

path (path): Path to a directory where to install the SDK (default is api.path.cache_dir / ‘windows_sdk’)
version (str): CIPD version of the SDK (default is set via $infra/windows_sdk.version property)
enabled (bool): Whether the SDK should be used or not.
target_arch (str): ‘x86’, ‘x64’, or ‘arm64’

Yields: If enabled, yields SDKPaths object with paths to well-known roots within the deployed bundle: * win_sdk - a Path to the root of the extracted Windows SDK. * dia_sdk - a Path to the root of the extracted Debug Interface Access SDK.

Raises: StepFailure or InfraFailure.

Recipes

recipes / bot_update:examples/full

DEPS: bot_update, gclient, gerrit, tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime

@recipe_api.ignore_warnings(‘^depot_tools/BOT_UPDATE_CUSTOM_RESULT_ATTRIBUTES$’)
— def RunSteps(api):

recipes / bot_update:tests/do_not_retry_patch_failures_in_cq

DEPS: bot_update, gclient, recipe_engine/buildbucket, recipe_engine/cv, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / bot_update:tests/download_topics

DEPS: bot_update, gclient, recipe_engine/json

— def RunSteps(api):

recipes / bot_update:tests/ensure_checkout

DEPS: bot_update, gclient, recipe_engine/json

— def RunSteps(api):

recipes / bot_update:tests/ensure_checkout_return_custom_result

DEPS: bot_update, gclient, recipe_engine/assertions, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/properties

@recipe_api.ignore_warnings(‘^depot_tools/BOT_UPDATE_CUSTOM_RESULT_ATTRIBUTES$’)
— def RunSteps(api, expected_checkout_dir, expected_source_root_name, expected_patch_root_name):