Repo documentation for build

If you need to upload or download build artifacts (or any other files) for something other than Chromium flavor, consider using ‘zip’ + ‘gsutil’ or ‘isolate’ modules instead.

— def clusterfuzz_archive(self, build_dir, update_properties, gs_bucket, archive_prefix, archive_subdir_suffix='', gs_acl=None, revision_dir=None, primary_project=None, bitness=None, use_legacy=True, **kwargs):

Archives and uploads a build to google storage.

The build is filtered by a list of file exclusions and then zipped. It is uploaded to google storage with some metadata about the commit position and revision attached. The zip file follows the naming pattern used by clusterfuzz. The file pattern is: ---.zip

If the build is experimental, -experimental is appended in the name.

Example: cool-project-linux-release-refs_heads_b1-12345.zip The archive name is “cool-project” and there's no component build. The commit is on a branch called b1 at commit position number 12345.

Example: cool-project-mac-debug-x10-component-234.zip The archive name is “cool-project” and the component's name is “x10”. The component is checked out in branch master with commit position number 234.

Args: build_dir: The absolute path to the build output directory, e.g. [slave-build]/src/out/Release update_properties: The properties from the bot_update step (containing commit information) gs_bucket: Name of the google storage bucket to upload to archive_prefix: Prefix of the archive zip file archive_subdir_suffix: Optional suffix to the google storage subdirectory name that contains the archive files gs_acl: ACL used for the file on google storage revision_dir: Optional component name if the main revision for this archive is a component revision primary_project: Optional project name for specifying the revision of the checkout bitness: The bitness of the build (32 or 64) to distinguish archive names. use_legacy: Specify if legacy paths and archive names should be used. Set to false for new builders.

— def download_and_unzip_build(self, step_name, target, build_url, src_dir=None, build_revision=None, build_archive_url=None, **kwargs):

Returns a step invoking extract_build.py to download and unzip a Chromium build.

— def legacy_download_url(self, gs_bucket_name, extra_url_components=None):

Returns a url suitable for downloading a Chromium build from Google Storage.

extra_url_components, if specified, should be a string without a trailing ‘/’ which is inserted in the middle of the URL.

The builder_name, or parent_buildername, is always automatically inserted into the URL.

— def legacy_platform_name(self):

Replicates the behavior of PlatformName() in chromium_utils.py.

— def legacy_upload_url(self, gs_bucket_name, extra_url_components=None):

Returns a url suitable for uploading a Chromium build to Google Storage.

extra_url_components, if specified, should be a string without a trailing ‘/’ which is inserted in the middle of the URL.

The builder_name, or parent_buildername, is always automatically inserted into the URL.

— def zip_and_upload_build(self, step_name, target, build_url=None, src_dir=None, build_revision=None, cros_board=None, package_dsym_files=False, exclude_files=None, exclude_perf_test_files=False, update_properties=None, store_by_hash=True, platform=None, **kwargs):

Returns a step invoking zip_build.py to zip up a Chromium build. If build_url is specified, also uploads the build.

recipe_modules / build

DEPS: depot_tools/depot_tools, recipe_engine/context, recipe_engine/python, recipe_engine/step

class ToolsBuildApi(RecipeApi):

@contextlib.contextmanager
— def gsutil_py_env(self):

Augments environment with slave_utils.py parameters.

— def python(self, name, script, args=None, show_path=True, **kwargs):

Bootstraps a Python through “tools/build”'s “runit.py”.

This function has the same semantics as the “recipe_engine/python” module's call method. It augments the call to run the invoked script through “runit.py”, which runs the targeted script within the “tools/build” Python path enviornment.

@property
— def runit_py(self):

@property
— def slave_utils_args(self):

Returns (list): A list of arguments to supply to configure slave_utils parameters. See slave_utils.py's AddArgs method.

TODO(dnj): This function and its invocations should be deprecated in favor of using environment variables via “add_slave_utils_kwargs”. The script invocation path for some of these is just too intertwined to confidently apply this via explicit args everywhere.

recipe_modules / chromite

DEPS: goma, repo, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, depot_tools/gitiles, depot_tools/infra_paths, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

class ChromiteApi(RecipeApi):

— def build_packages(self, board, args=None, **kwargs):

Run the build_packages script inside the chroot.

Used by the internal goma recipe.

— def cbuildbot(self, name, config, args=None, **kwargs):

Runs the cbuildbot command defined by the arguments.

Args: name: (str) The name of the command step. config: (str) The name of the ‘cbuildbot’ configuration to invoke. args: (list) If not None, addition arguments to pass to ‘cbuildbot’.

Returns: (Step) The step that was run.

— def check_repository(self, repo_type_key, value):

Scans through registered repositories for a specified value.

Args: repo_type_key (str): The key in the ‘repositories’ config to scan through. value (str): The value to scan for. Returns (bool): True if the value was found.

— def checkout(self, manifest_url=None, repo_url=None, branch=None, repo_sync_args=None):

— def checkout_chromite(self):

Checks out the configured Chromite branch.

@property
— def chromite_path(self):

— def configure(self, properties, config_map, **KWARGS):

Loads configuration from build properties into this recipe config.

Args: properties (Properties): The build properties object. config_map (dict): The configuration map to use. KWARGS: Additional keyword arguments to forward to the configuration.

— def cros_sdk(self, name, cmd, args=None, environ=None, chroot_cmd=None, **kwargs):

Return a step to run a command inside the cros_sdk.

Used by the internal goma recipe.

@property
— def depot_tools_path(self):

— def gclient_config(self):

Generate a ‘gclient’ configuration to check out Chromite.

Return: (config) A ‘gclient’ recipe module configuration.

— def get_config_defaults(self):

— def load_manifest_config(self, repository, revision):

Loads manifest-specified parameters from the manifest commit.

This method parses the commit log for the following information:

The branch to build (From the “Automatic”: tag).
The build ID (from the CrOS-Build-Id: tag).

Args: repository (str): The URL of the repository hosting the change. revision (str): The revision hash to load the build ID from.

— def run(self, args=None):

Runs the configured ‘cbuildbot’ build.

This workflow uses the registered configuration dictionary to make master- and builder-specific changes to the standard workflow.

The specific workflow paths that are taken are also influenced by several build properties.

TODO(dnj): When CrOS migrates away from BuildBot, replace property inferences with command-line parameters.

This workflow:

Checks out the specified ‘cbuildbot’ repository.
Pulls information based on the configured change's repository/revision to pass to ‘cbuildbot’.
Executes the ‘cbuildbot’ command.

Args: args (list): Initial argument list, expanded based on other values. Returns: (Step) the ‘cbuildbot’ execution step.

— def run_cbuildbot(self, args=None, goma_canary=False):

Performs a Chromite repository checkout, then runs cbuildbot.

Args: args (list): Initial argument list, see run() for details. goma_canary (bool): Use canary version of goma if True.

— def setup_board(self, board, args=None, **kwargs):

Run the setup_board script inside the chroot.

Used by the internal goma recipe.

— def with_system_python(self):

Prepare a directory with the system python binary available.

This is designed to make it possible to mask “bundled python” out of the standard path without hiding any other binaries.

Returns: (context manager) A context manager that inserts system python into the front of PATH.

recipe_modules / chromium

DEPS: adb, build, chromite, gn, goma, depot_tools/bot_update, depot_tools/cipd, depot_tools/depot_tools, depot_tools/gclient, depot_tools/git, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

class ChromiumApi(RecipeApi):

@_with_chromium_layout
— def archive_build(self, step_name, gs_bucket, gs_acl=None, mode=None, build_name=None, **kwargs):

Returns a step invoking archive_build.py to archive a Chromium build.

@property
— def build_properties(self):

@contextlib.contextmanager
— def chromium_layout(self):

Context manager that must be entered prior to performing any Chromium recipe operations. This is responsible for basic enviornment initialization.

See “ensure_chromium_layout” for more information.

— def clean_outdir(self):

Removes old files from the out directory.

This is done to stop unbounded growth of the out directory as the files with version numbers in file-name will not be overwritten by the next build.

— def clobber_if_needed(self):

Add an explicit clobber step if requested.

@_with_chromium_layout
— def compile(self, targets=None, name=None, out_dir=None, target=None, use_goma_module=False, **kwargs):

Return a compile.py invocation.

Args: targets: List of build targets to compile. name: Name of compile step. out_dir: Output directory for the compile. target: Custom config name to use in the output directory (defaults to “Release” or “Debug”). use_goma_module (bool): If True, use the goma recipe module.

— def configure_bot(self, builders_dict, additional_configs=None):

Sets up the configurations and gclient to be ready for bot update.

builders_dict is a dict of mastername -> ‘builders’ -> buildername -> bot_config.

The current mastername and buildername are looked up from the build properties; we then apply the configs specified in bot_config as appropriate.

Returns a tuple of (buildername, bot_config) for subsequent use in the recipe.

— def delete_old_mac_toolchain(self):

Remove the old “hermetic” toolchain cache.

This is to expose any lingering dependencies on the old cache.

— def ensure_chromium_layout(self):

Ensures that Chromium build layout is installed.

Note: the layout must be installed into the engine context. The “chromium_layout” context manager is probably a better way to access this.

Returns (ChromiumApi.Layout): The configured Chromium build layout.

— def ensure_goma(self, client_type=‘release’):

— def ensure_mac_toolchain(self):

— def ensure_toolchains(self):

— def get_annotate_by_test_name(self, _):

@_with_chromium_layout
— def get_clang_version(self, **kwargs):

— def get_config_defaults(self):

— def get_cros_chrome_sdk_wrapper(self):

Returns: a wrapper command for ‘cros chrome-sdk’

Args: external: (bool) If True, force the wrapper to prefer external board configurations over internal ones, even if the latter is available. clean: (bool) If True, instruct the wrapper to clean any previous state data.

— def get_env(self):

— def get_mac_toolchain_installer(self):

— def get_version(self):

Returns a dictionary describing the version.

The dictionary will map the name of the portion of the version to its numeric value e.g. { ‘MAJOR’": ‘51’, ‘MINOR’: ‘0’, ‘BUILD’: ‘2704’, ‘PATCH’: ‘0’ }

— def get_version_from_file(self, version_file_path, step_name=‘get version’):

Returns the version information from a specified file.

The dictionary will map the name of the portion of the version to its numeric value e.g. { ‘MAJOR’": ‘51’, ‘MINOR’: ‘0’, ‘BUILD’: ‘2704’, ‘PATCH’: ‘0’ }

@_with_chromium_layout
— def mb_analyze(self, mastername, buildername, analyze_input, name=None, mb_path=None, mb_config_path=None, chromium_config=None, build_dir=None, phase=None, **kwargs):

Determine which targets need to be built and tested.

Args: mastername: The name of the master for the build configuration to analyze. buildername: The name of the builder for the build configuration to analyze. analyze_input: a dict of the following form: { ‘files’: [‘affected/file1’, ‘affected/file2’, ...], ‘test_targets’: [‘test_target1’, ‘test_target2’, ...], ‘additional_compile_targets’: [‘target1’, ‘target2’, ...], }

Returns: The StepResult from the analyze command.

@_with_chromium_layout
— def mb_gen(self, mastername, buildername, name=None, mb_path=None, mb_config_path=None, use_goma=True, isolated_targets=None, build_dir=None, phase=None, android_version_code=None, android_version_name=None, gn_args_location=None, gn_args_max_text_lines=None, recursive_lookup=False, **kwargs):

Generate the build files in the source tree.

Args: mastername: The name of the master for the build configuration to generate build files for. buildername: The name of the builder for the build configuration to generate build files for. name: The name of the step. If not provided ‘generate_build_files’ will be used. mb_path: The path to the source directory containing the mb.py script. If not provided, the subdirectory tools/mb within the source tree will be used. mb_config_path: The path to the configuration file containing the master and builder specifications to be used by mb. If not provided, the project_generator.config_path config value will be used. If that is falsey, then mb_config.pyl under the directory identified by mb_path will be used. gn_args_location: Controls where the GN args for the build should be presented. By default or if gn.DEFAULT, the args will be in step_text if the count of lines is less than gn_args_max_text_lines or the logs otherwise. To force the presentation to the step_text or logs, use gn.TEXT or gn.LOGS, respectively. gn_args_max_text_lines: The maximum number of lines of GN args to display in the step_text when using the default behavior for displaying GN args. recursive_lookup: Whether the lookup of the GN arguments should recursively expand imported args files.

Returns: The content of the args.gn file.

@_with_chromium_layout
— def mb_isolate_everything(self, mastername, buildername, use_goma=True, mb_path=None, mb_config_path=None, name=None, build_dir=None, android_version_code=None, android_version_name=None, phase=None, **kwargs):

@_with_chromium_layout
— def mb_lookup(self, mastername, buildername, name=None, mb_path=None, mb_config_path=None, recursive=False, chromium_config=None, phase=None, use_goma=True, android_version_code=None, android_version_name=None, gn_args_location=None, gn_args_max_text_lines=None):

Lookup the GN args for the build.

Args: mastername: The name of the master for the build configuration to look up. buildername: The name of the builder for the build configuration to look up. name: The name of the step. If not provided ‘lookup GN args’ will be used. mb_path: The path to the source directory containing the mb.py script. If not provided, the subdirectory tools/mb within the source tree will be used. mb_config_path: The path to the configuration file containing the master and builder specifications to be used by mb. If not provided, the project_generator.config_path config value will be used. If that is falsey, then mb_config.pyl under the directory identified by mb_path will be used. recursive: Whether the lookup should recursively expand imported args files. chromium_config: The chromium config object to use. If not provided, self.c will be used. gn_args_location: Controls where the GN args for the build should be presented. By default or if gn.DEFAULT, the args will be in step_text if the count of lines is less than gn_args_max_text_lines or the logs otherwise. To force the presentation to the step_text or logs, use gn.TEXT or gn.LOGS, respectively. gn_args_max_text_lines: The maximum number of lines of GN args to display in the step_text when using the default behavior for displaying GN args.

Returns: The content of the args.gn file.

@property
— def output_dir(self):

Return the path to the built executable directory.

— def process_dumps(self, **kwargs):

@_with_chromium_layout
— def run_gn(self, use_goma=False, gn_path=None, build_dir=None, **kwargs):

@_with_chromium_layout
— def run_mb_cmd(self, name, mb_command, mastername, buildername, mb_path=None, mb_config_path=None, chromium_config=None, phase=None, use_goma=True, android_version_code=None, android_version_name=None, additional_args=None, **kwargs):

Run an arbitrary mb command.

Args: name: The name of the step. mb_command: The mb command to run. mastername: The name of the master of the configuration to run mb for. buildername: The name of the builder of the configuration to run mb for. mb_path: The path to the source directory containing the mb.py script. If not provided, the subdirectory tools/mb within the source tree will be used. mb_config_path: The path to the configuration file containing the master and builder specifications to be used by mb. If not provided, the project_generator.config_path config value will be used. If that is falsey, then mb_config.pyl under the directory identified by mb_path will be used. chromium_config: The chromium config object to use. If not provided, self.c will be used. additional_args: Any args to the mb script besodes those for setting the master, builder and the path to the config file. **kwargs: Additional arguments to be forwarded onto the python API.

@_with_chromium_layout
— def runhooks(self, env=None, **kwargs):

Run the build-configuration hooks for chromium.

@_with_chromium_layout
— def runtest(self, test, args=None, xvfb=False, name=None, annotate=None, results_url=None, perf_dashboard_id=None, test_type=None, python_mode=False, point_id=None, revision=None, webkit_revision=None, test_launcher_summary_output=None, perf_id=None, perf_config=None, chartjson_file=False, use_histograms=False, tee_stdout_file=None, **kwargs):

Return a runtest.py invocation.

— def set_build_properties(self, props):

@_with_chromium_layout
— def sizes(self, results_url=None, perf_id=None, platform=None, **kwargs):

Return a sizes.py invocation. This uses runtests.py to upload the results to the perf dashboard.

— def taskkill(self):

@returns_placeholder
— def test_launcher_filter(self, tests):

recipe_modules / chromium_android

DEPS: adb, archive, build, chromium, chromium_checkout, test_utils, zip, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/git, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile, recipe_engine/time, recipe_engine/url

class AndroidApi(RecipeApi):

— def adb_install_apk(self, apk, allow_downgrade=False, keep_data=False, devices=None):

— def android_build_wrapper(self, logcat_gs_bucket=‘chromium-android’):

— def android_test_wrapper(self, _logcat_gs_bucket=‘chromium-android’):

— def apk_path(self, apk):

— def asan_device_setup(self):

— def asan_device_teardown(self):

— def authorize_adb_devices(self):

@property
— def blacklist_file(self):

— def clean_local_files(self, clean_pyc_files=True):

— def common_tests_final_steps(self, logcat_gs_bucket=‘chromium-android’, force_latest_version=False, checkout_dir=None):

— def common_tests_setup_steps(self, perf_setup=False, **provision_kwargs):

— def configure_from_properties(self, config_name, **kwargs):

— def copy_gtest_results(self, result_step, active_step):

@property
— def coverage_dir(self):

— def coverage_report(self, upload=True, **kwargs):

Creates an EMMA HTML report and optionally uploads it to storage bucket.

Creates an EMMA HTML report using generate_emma_html.py, and uploads the HTML report to the chrome-code-coverage storage bucket if |upload| is True.

Args: upload: Uploads EMMA HTML report to storage bucket unless False is passed in. **kwargs: Kwargs for python and gsutil steps.

Returns: step_result: The recipe result from generating the coverage report.

— def create_adb_symlink(self):

— def create_result_details(self, step_name, json_results_file):

— def device_recovery(self, restart_usb=False, **kwargs):

— def device_status(self, **kwargs):

— def device_status_check(self):

@property
— def devices(self):

— def download_build(self, bucket, path, extract_path=None, globs=None):

@property
— def file_changes_path(self):

Get or create the path to the file containing changes for this revision.

This file will contain a dict mapping file paths to lists of changed lines for each file. This is used to generate incremental coverage reports.

— def generate_breakpad_symbols(self, symbols_dir, binary_path, root_chromium_dir):

Generate breakpad symbols.

This step requires dump_syms binary to exist in the build dir.

Args: symbols_dir: The directory to dump the breakpad symbols to. binary_path: Path to binary to generate symbols for. root_chromium_dir: Root Chromium directory.

— def get_changed_lines_for_revision(self):

Saves a JSON file containing the files/lines requiring coverage analysis.

Saves a JSON object mapping file paths to lists of changed lines to the coverage directory.

— def get_config_defaults(self):

— def git_number(self, commitrefs=None, step_test_data=None, **kwargs):

@contextlib.contextmanager
— def handle_exit_codes(self):

Handles exit codes emitted by the test runner and other scripts.

— def host_info(self, args=None, **kwargs):

— def incremental_coverage_report(self):

Creates an incremental code coverage report.

Generates a JSON file containing incremental coverage stats. Requires |file_changes_path| to contain a file with a valid JSON object.

— def init_and_sync(self, gclient_config=‘android_bare’, with_branch_heads=False, use_bot_update=True, use_git_cache=True, manifest_name=None):

@property
— def known_devices_file(self):

— def logcat_dump(self, gs_bucket=None):

— def make_zip_archive(self, step_name, archive_name, files=None, preserve_paths=True, include_filters=None, exclude_filters=None, **kwargs):

Creates and stores the archive file.

Args: step_name: Name of the step. archive_name: Name of the archive file. files: If specified, only include files here instead of out/. preserve_paths: If True, files will be stored using the subdirectories in the archive. include_filters: List of globs to be included in the archive. exclude_filters: List of globs to be excluded from the archive.

— def monkey_test(self, **kwargs):

— def non_blacklisted_devices(self):

@property
— def out_path(self):

— def provision_devices(self, skip_wipe=False, disable_location=False, min_battery_level=None, disable_network=False, disable_java_debug=False, max_battery_temp=None, reboot_timeout=None, emulators=False, **kwargs):

— def resource_sizes(self, apk_path, chartjson_file=False, perf_id=None, step_suffix=''):

— def run_bisect_script(self, extra_src=‘‘, path_to_config=’’, **kwargs):

— def run_instrumentation_suite(self, name, test_apk=None, apk_under_test=None, additional_apks=None, flakiness_dashboard=None, annotation=None, except_annotation=None, screenshot=False, verbose=False, tool=None, json_results_file=None, timeout_scale=None, strict_mode=None, suffix=None, num_retries=None, device_flags=None, wrapper_script_suite_name=None, result_details=False, store_tombstones=False, trace_output=False, args=None, **kwargs):

— def run_java_unit_test_suite(self, suite, verbose=True, json_results_file=None, suffix=None, **kwargs):

— def run_sharded_perf_tests(self, config, test_type_transform=None, upload_archives_to_bucket=None, timestamp_as_point_id=False):

Run the perf tests from the given config file.

config: the path of the config file containing perf tests. test_type_transform: a lambda transforming the test name to the test_type to upload to. upload_archives_to_bucket: an optional string, if given will create an archive of all output files per test and upload to the bucket specified. timestamp_as_point_id: if True, will use a unix timestamp as a point_id to identify values in the perf dashboard; otherwise the default (commit position) is used.

— def run_telemetry_browser_test(self, test_name, browser=‘android-chromium’):

Run a telemetry browser test.

— def run_test_suite(self, suite, verbose=True, tool=None, result_details=False, store_tombstones=False, name=None, json_results_file=None, shard_timeout=None, args=None, **kwargs):

— def run_tree_truth(self, additional_repos=None):

— def run_webview_cts(self, android_platform, arch, command_line_args=None, suffix=None, json_results_file=None, result_details=False):

— def shutdown_device_monitor(self):

— def spawn_device_monitor(self):

— def spawn_logcat_monitor(self):

— def stack_tool_steps(self, force_latest_version=False):

— def stackwalker(self, root_chromium_dir, binary_paths):

Runs stack walker tool to symbolize breakpad crashes.

This step requires logcat file. The logcat monitor must have been run on the bot.

Args: binary_paths: Paths to binaries to generate breakpad symbols. root_chromium_dir: Root Chromium directory.

— def staged_files_matching_filter(self, diff_filter):

Returns list of files changed matching the provided diff-filter.

Args: diff_filter: A string to be used as the diff-filter.

Returns: A list of file paths (strings) matching the provided |diff-filter|.

— def supersize_archive(self, apk_path, size_path, step_suffix=''):

Creates a .size file for the given .apk or .minimal.apks.

— def test_report(self):

— def test_runner(self, step_name, args=None, wrapper_script_suite_name=None, pass_adb_path=True, **kwargs):

Wrapper for the python testrunner script.

Args: step_name: Name of the step. args: Testrunner arguments.

— def upload_apks_for_bisect(self, update_properties, bucket, path):

Uploads android apks for functional bisects.

— def upload_build(self, bucket, path):

— def use_devil_adb(self):

— def wait_for_devices(self, devices, timeout=None):

— def zip_and_upload_build(self, _):

recipe_modules / chromium_checkout

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

class ChromiumCheckoutApi(RecipeApi):

— def ensure_checkout(self, bot_config, **kwargs):

Wrapper for bot_update.ensure_checkout with chromium-specific additions.

— def get_checkout_dir(self, bot_config):

Returns directory where checkout can be created.

TODO: Cache this result for a given “bot_config.checkout_dir” value so we don't have to “makedirs” each time the checkout dir is queried.

— def get_files_affected_by_patch(self, relative_to=‘src/’, cwd=None):

Returns list of POSIX paths of files affected by patch for “analyze”.

Paths are relative to relative_to which for analyze should be ‘src/’.

@property
— def working_dir(self):

Returns parent directory of the checkout.

Requires |ensure_checkout| to be called first.

recipe_modules / chromium_gsutil

DEPS: build, recipe_engine/context, recipe_engine/path, recipe_engine/python, recipe_engine/step

class ChromiumGSUtilApi(RecipeApi):

— def download_latest_file(self, base_url, partial_name, destination, name=‘Download latest file from GS’):

Get the latest archived object with the given base url and partial name.

Args: base_url: Base Google Storage archive URL (gs://...) containing the build. partial_name: Partial name of the archive file to download. destination: Destination file/directory where the file will be downloaded. name: The name of the step.

recipe_modules / chromium_swarming

DEPS: build, chromium, chromium_checkout, isolate, swarming_client, test_utils, traceback, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/swarming

class SwarmingApi(RecipeApi):

Recipe module to use swarming.py tool to run tasks on Swarming.

General usage:

Tweak default task parameters applied to all swarming tasks (such as default_dimensions and default_priority).
Isolate some test using ‘isolate’ recipe module. Get isolated hash as a result of that process.
Create a task configuration using ‘task(...)’ method, providing isolated hash obtained previously.
Tweak the task parameters. This step is optional.
Launch the task on swarming by calling ‘trigger_task(...)’.
Continue doing useful work locally while the task is running concurrently on swarming.
Wait for task to finish and collect its result (exit code, logs) by calling ‘collect_task(...)’.

recipe_modules / chromium_tests

DEPS: adb, archive, build, chromium, chromium_android, chromium_checkout, chromium_swarming, clang_coverage, filter, gn, goma, isolate, perf_dashboard, puppet_service_account, test_results, test_utils, traceback, trigger, zip, depot_tools/bot_update, depot_tools/gclient, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/scheduler, recipe_engine/step, recipe_engine/tempfile, recipe_engine/time

class ChromiumTestsApi(RecipeApi):

— def add_builders(self, builders):

Adds builders to our builder map

@property
— def all_generators(self):

— def archive_build(self, mastername, buildername, update_step, bot_db):

Archive the build if the bot is configured to do so.

See api.archive.clusterfuzz_archive and archive_build.py for more information.

This is currently used to store builds long-term and to transfer them to clusterfuzz.

@property
— def builders(self):

— def compile_specific_targets(self, bot_config, update_step, bot_db, compile_targets, tests_including_triggered, mb_mastername=None, mb_buildername=None, mb_phase=None, mb_config_path=None, mb_recursive_lookup=False, override_bot_type=None):

Runs compile and related steps for given builder.

Allows finer-grained control about exact compile targets used.

We don‘t use the given mastername and buildername to run MB, because they may be the values of the continuous builder the trybot may be configured to match; instead we need to use the actual mastername and buildername we’re running on (Default to the “mastername” and “buildername” in the build properties -- self.m.properties, but could be overridden by mb_mastername and mb_buildername), because it may be configured with different MB settings.

However, recipes used by Findit for culprit finding may still set (mb_mastername, mb_buildername) = (mastername, buildername) to exactly match a given continuous builder.

— def configure_build(self, bot_config, override_bot_type=None):

— def create_bot_config_object(self, bot_ids, builders=None):

— def create_bot_db_from_master_dict(self, mastername, master_dict):

— def create_bot_db_object(self):

— def create_bot_id(self, mastername, buildername, testername=None):

— def create_test_runner(self, tests, suffix='', serialize_tests=False):

Creates a test runner to run a set of tests.

Args api: API of the calling recipe. tests: List of step.Test objects to be run. suffix: Suffix to be passed when running the tests. serialize_tests: True if this bot should run all tests serially (specifically, tests run on Swarming). Used to reduce the load generated by waterfall bots.

Returns: A function that can be passed to setup_chromium_tests or run directly.

— def deapply_patch(self, bot_update_step):

— def download_and_unzip_build(self, mastername, buildername, update_step, bot_db, build_archive_url=None, build_revision=None, override_bot_type=None, read_gn_args=True):

— def generate_tests_from_source_side_spec(self, source_side_spec, builder_dict, buildername, mastername, swarming_dimensions, scripts_compile_targets, bot_update_step):

— def generate_tests_from_test_spec(self, *args):

— def get_android_version_details(self, bot_config, log_details=False):

— def get_common_args_for_scripts(self, bot_config=None):

— def get_compile_targets(self, bot_config, bot_db, tests):

— def get_compile_targets_for_scripts(self, bot_config=None):

This gets the combined compile_targets information from the //testing/scripts/get_compile_targets.py script.

This script returns the compile targets for all of the ‘script tests’ in chromium (including ones that we don't plan to run on this configuration, see TODO). The information is returned in the following format:

{ “some_script_name.py”: [“list”, “of”, “compile”, “targets”], }

Where “some_script_name.py” corresponds to “//testing/scripts/some_script_name.py”.

Returns: The compile target data in the form described above.

TODO:

Only gather targets for the scripts that we might concievably run.

— def get_config_defaults(self):

— def get_tests(self, bot_config, bot_db):

Returns a tuple: list of tests, and list of tests on the triggered testers.

— def is_precommit_mode(self):

Returns a Boolean indicating whether this module is running in precommit mode; i.e., whether tests are running before the changes are committed.

— def log(self, message):

— def main_waterfall_steps(self, mb_config_path=None, builders=None):

— def package_build(self, mastername, buildername, update_step, bot_db, reasons=None):

Zip and upload the build to google storage.

This is currently used for transfer between builder and tester, including bisect testers.

Note that:

this will only upload when called from pure builders. On builder_testers and testers, this is a no-op.
this is a no-op for builders that upload to clusterfuzz; those are handled in archive_build.
this may upload twice on perf builders.

— def prepare_checkout(self, bot_config, **kwargs):

— def read_source_side_spec(self, source_side_spec_file):

— def run_mb_and_compile(self, compile_targets, isolated_targets, name_suffix, mb_mastername=None, mb_buildername=None, mb_phase=None, mb_config_path=None, mb_recursive_lookup=False, android_version_code=None, android_version_name=None):

— def run_tests_with_and_without_changes(self, builders, mirrored_bots, deapply_changes):

— def runhooks(self, update_step):

— def set_precommit_mode(self):

Configures this module to indicate that tests are running before the changes are committed. This must be called very early in the recipe, certainly before prepare_checkout, and the action can not be undone.

— def set_up_swarming(self, bot_config):

@property
— def steps(self):

The steps module, which contains various Test python classes.

Usage is generally discouraged.

— def trigger_child_builds(self, mastername, buildername, update_step, bot_db, additional_properties=None):

— def trybot_steps(self, builders=None, trybots=None):

@property
— def trybots(self):

@contextlib.contextmanager
— def wrap_chromium_tests(self, bot_config, tests=None):

recipe_modules / clang_coverage

DEPS: chromium, chromium_checkout, swarming_client, depot_tools/gclient, depot_tools/git, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class ClangCoverageApi(RecipeApi):

This module contains apis to interact with llvm-cov and llvm-profdata.

@property
— def cov_executable(self):

Returns the path to the llvm-cov executable.

@property
— def generate_individual_metadata(self):

True if we should generate per-target metadata for this build.

— def instrument(self, affected_files):

Saves source paths to generate coverage instrumentation for to a file.

Args: affected_files (list of str): paths to the files we want to instrument, relative to the checkout path.

@property
— def merge_scripts_location(self):

@property
— def metadata_dir(self):

A temporary directory for the metadata. Created on first access.

— def process_coverage_data(self, tests):

Processes the coverage data for html report or metadata.

Args: tests (list of self.m.chromium_tests.stepsl.Test): A list of test objects whose binaries we are to create a coverage report for.

— def profdata_dir(self, step_name=None):

Ensures a directory exists for writing the step-level merged profdata.

Args: step_name (str): The name of the step for the target whose profile we'll save in in this dir. None for getting the parent directory to contain the dirs for all steps.

@property
— def profdata_executable(self):

Returns the path to the llvm-profdata executable.

@property
— def raw_profile_merge_script(self):

Returns the location of a script that merges raw profiles from shards.

This is intended to be passed to the swarming recipe module to be called upon completion of the shards.

@property
— def report_dir(self):

A temporary directory to save a report to. Created on first access.

— def shard_merge(self, step_name, additional_merge=None):

Returns a merge object understood by the swarming module.

See the docstring for the merge parameter of api.chromium_swarming.task.

|additional_merge| is an additional merge script. This will be invoked from the clang coverage merge script.

@property
— def step_merge_script(self):

Returns the script that merges indexed profiles from multiple targets.

@property
— def using_coverage(self):

Checks if the current build is running coverage-instrumented targets.

recipe_modules / codesearch

DEPS: build, chromium, goma, depot_tools/depot_tools, depot_tools/git, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

class CodesearchApi(RecipeApi):

— def add_kythe_metadata(self):

Adds inline Kythe metadata to Mojom generated files.

This metadata is used to connect things in the generated file to the thing in the Mojom file which generated it. This is made possible by annotations added to the generated file by the Mojo compiler.

— def checkout_generated_files_repo_and_sync(self):

Check out the generated files repo and sync the generated files into this checkout.

— def cleanup_old_generated(self, age_days=7):

Clean up generated files older than the specified number of days.

Args: age_days: Minimum age in days for files to delete (integer).

— def create_and_upload_kythe_index_pack(self, commit_timestamp):

Create the kythe index pack and upload it to google storage.

Args: commit_timestamp: Timestamp of the commit at which we're creating the index pack, in integer seconds since the UNIX epoch.

— def generate_compilation_database(self, targets, mastername, buildername, output_file=None, mb_config_path=None):

— def generate_gn_target_list(self, output_file=None):

— def get_config_defaults(self):

— def run_clang_tool(self):

Download and run the clang tool.

recipe_modules / cronet

DEPS: build, chromium, chromium_android, perf_dashboard, puppet_service_account, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

Common steps for recipes that sync/build Cronet sources.

class CronetApi(RecipeApi):

— def build(self, mastername=None, buildername=None, targets=None, use_goma=True):

— def get_bucket(self, platform):

— def get_version(self):

— def init_and_sync(self, recipe_config, kwargs, chromium_apply_config=None):

— def run_perf_tests(self, perf_id):

— def run_tests(self, unit_tests=None, instrumentation_tests=INSTRUMENTATION_TESTS):

— def sizes(self, perf_id):

— def upload_package(self, build_config, cronetdir=None, platform=‘android’):

recipe_modules / crrev

DEPS: build, recipe_engine/commit_position, recipe_engine/json, recipe_engine/python, recipe_engine/step

class CrrevApi(RecipeApi):

Recipe module for making requests to crrev.com.

— def to_commit_hash(self, commit_position, project=‘chromium’, repo=‘chromium/src’, attempts=3, step_name=None, **kwargs):

Fetches the corresponding commit hash for a commit position.

— def to_commit_position(self, commit_hash, attempts=3, step_name=None):

Fetches a commit position string given a commit hash.

recipe_modules / dart

DEPS: chromium_swarming, goma, swarming_client, depot_tools/bot_update, depot_tools/cipd, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gerrit, depot_tools/git, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class DartApi(RecipeApi):

Recipe module for code commonly used in dart recipes.

Shouldn't be used elsewhere.

— def build(self, build_args=None, name=‘build dart’):

Builds dart using the specified build_args

— def checkout(self, clobber=False):

Checks out the dart code and prepares it for building.

— def collect_all(self, steps):

Collects the results of a sharded test run.

— def dart_executable(self):

Returns the path to the checked-in SDK dart executable.

— def delete_debug_log(self):

Deletes the debug log file

— def download_parent_isolate(self):

— def get_secret(self, name):

Decrypts the specified secret and returns the location of the result

— def kill_tasks(self, ok_ret=‘any’):

Kills leftover tasks from previous runs or steps.

— def read_debug_log(self):

Reads the debug log file

— def shard(self, title, isolate_hash, test_args, os, cpu=‘x86-64’, pool=‘dart.tests’, num_shards=0, last_shard_is_local=False, cipd_packages=None, ignore_failure=False):

Runs test.py in the given isolate, sharded over several swarming tasks. Returns the created tasks, which can be collected with collect_all().

— def test(self, test_data):

Reads the test-matrix.json file in checkout and runs each step listed in the file.

Raises StepFailure.

— def upload_isolate(self, isolate_fileset):

Builds an isolate

recipe_modules / disk

DEPS: traceback, depot_tools/infra_paths, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step

class DiskApi(RecipeApi):

DiskApi contains helper functions for reading disk info.

— def space_usage(self, path=None, warning_level=None, previous_result=None, can_fail_build=False, name=None, **kwargs):

Displays disk space usage.

Does not support Windows yet, does not emit steps.

Prints disk space usage in step log and step text.

Args: path (str): path mapped to the disk. Defaults to [START_DIR]. warning_level (float): a value from 0 to 1.0. If usage reaches this level, mark the step as WARNING. Defaults to 0.9. previous_result (dict): previous result of space_usage call. If passed, delta is displayed in step text. name (str): step name. Defaults to “disk space usage”.

Returns: A dict with disk usage info or None if step fails. Dict keys: * capacity (float): disk capacity, in MiB. * used (float): disk usage, in MiB.

recipe_modules / docker

DEPS: recipe_engine/path, recipe_engine/python, recipe_engine/raw_io, recipe_engine/service_account, recipe_engine/step

class DockerApi(RecipeApi):

Provides steps to connect and run Docker images.

— def __call__(self, *args, **kwargs):

Executes specified docker command.

Please make sure to use api.docker.login method before if specified command requires authentication.

Args: args: arguments passed to the ‘docker’ command including subcommand name, e.g. api.docker(‘push’, ‘my_image:latest’). kwargs: arguments passed down to api.step module.

— def ensure_installed(self, **kwargs):

Checks that the docker binary is in the PATH.

Raises StepFailure if binary is not found.

— def get_version(self, **kwargs):

Returns Docker version installed or None if failed to detect.

— def login(self, server=‘gcr.io’, project=‘chromium-container-registry’, service_account=None, step_name=None, **kwargs):

Connect to a Docker registry.

This step must be executed before any other step in this module that requires authentication.

Args: server: GCP container registry to pull images from. Defaults to ‘gcr.io’. project: Name of the Cloud project where Docker images are hosted. service_account: service_account.api.ServiceAccount used for authenticating with the container registry. Defaults to the task's associated service account. step_name: Override step name. Default is ‘docker login’.

— def run(self, image, step_name=None, cmd_args=None, dir_mapping=None, env=None, inherit_luci_context=False, **kwargs):

Run a command in a Docker image as the current user:group.

Args: image: Name of the image to run. step_name: Override step name. Default is ‘docker run’. cmd_args: Used to specify command to run in an image as a list of arguments. If not specified, then the default command embedded into the image is executed. dir_mapping: List of tuples (host_dir, docker_dir) mapping host directories to directories in a Docker container. Directories are mapped as read-write. env: dict of env variables. inherit_luci_context: Inherit current LUCI Context (including auth). CAUTION: removes network isolation between the container and the docker host. Read more https://docs.docker.com/network/host/.

recipe_modules / emulator

DEPS: build, chromium_android, recipe_engine/path, recipe_engine/python, recipe_engine/step

class EmulatorApi(RecipeApi):

— def get_config_defaults(self):

— def install_emulator_deps(self, api_level, **kwargs):

@contextlib.contextmanager
— def launch_emulator(self, abi, api_level, amount=1, partition_size=None, sdcard_size=None, **kwargs):

— def wait_for_emulator(self, num, **kwargs):

recipe_modules / filter

DEPS: chromite, chromium, goma, depot_tools/git, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class FilterApi(RecipeApi):

— def analyze(self, affected_files, test_targets, additional_compile_targets, config_file_name, mb_mastername=None, mb_buildername=None, mb_config_path=None, build_output_dir=None, additional_names=None):

Runs “analyze” step to determine targets affected by the patch.

Returns a tuple of:

list of targets that are needed to run tests (see filter recipe module)
list of targets that need to be compiled (see filter recipe module)

@property
— def compile_targets(self):

Returns the set of targets that need to be compiled based on the set of files that have changed.

— def does_patch_require_compile(self, affected_files, test_targets=None, additional_compile_targets=None, additional_names=None, config_file_name=‘trybot_analyze_config.json’, use_mb=False, mb_mastername=None, mb_buildername=None, mb_config_path=None, build_output_dir=None, cros_board=None, **kwargs):

Check to see if the affected files require a compile or tests.

Args: affected_files: list of files affected by the current patch; paths should only use forward slashes (“/”) on all platforms test_targets: the possible set of executables that are desired to run. When done, test_targets() returns the subsetset of targets that are affected by the files that have changed. additional_compile_targets: any targets to compile in addition to the test_targets. additional_names: additional top level keys to look up exclusions in, see |config_file_name|. conconfig_file_name: the config file to look up exclusions in. mb_mastername: the mastername to pass over to run MB. mb_buildername: the buildername to pass over to run MB. mb_config_path: the path to the MB config file.

Within the file we concatenate “base.exclusions” and “|additional_names|.exclusions” (if |additional_names| is not none) to get the full list of exclusions.

The exclusions should be a list of Python regular expressions (as strings).

If any of the files in the current patch match one of the values in we assume everything needs to be compiled and tested.

If an error occurs, an exception is raised. Otherwise, after the call completes the results can be obtained from self.compile_targets() and self.test_targets().

To run MB, we need to use the actual mastername and buildername we're running on, and not those of the continuous builder the trybot may be configured to match, because a trybot may be configured with different MB settings. However, recipes used by Findit for culprit finding may override the defaults with mb_mastername and mb_buildername to exactly match a given continuous builder.

@property
— def paths(self):

Returns the paths that have changed in this patch.

@property
— def test_targets(self):

Returns the set of targets passed to does_patch_require_compile() that are affected by the set of files that have changed.

recipe_modules / findit

DEPS: chromium, chromium_checkout, test_utils, depot_tools/depot_tools, depot_tools/gclient, depot_tools/git, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class FinditApi(RecipeApi):

— def compile_and_test_at_revision(self, api, target_mastername, target_buildername, target_testername, revision, requested_tests, use_analyze, test_repeat_count=None, skip_tests=False):

Compile the targets needed to execute the specified tests and run them.

Args: api (RecipeApi): With the dependencies injected by the calling recipe. target_mastername (str): Which master to derive the configuration off of. target_buildername (str): likewise target_testername (str): likewise revision (str): A string representing the commit hash of the revision to test. requested_tests (dict): maps the test name (step name) to the names of the subtest to run. i.e. for GTests these are built into a test filter string, and passed in the command line. E.g. {‘browser_tests’: [‘suite.test1’, ‘suite.test2’]} use_analyze (bool): Whether to trim the list of tests to perform based on which files are actually affected by the revision. test_repeat_count (int or None): Repeat count to pass to the test. Note that we don‘t call the test this many times, we call the test once but pass this repeat factor as a parameter, and it’s up to the test implementation to perform the repeats. Either 1 or None imply that no special flag for repeat will be passed. skip_tests (bool): If True, do not actually run the tests. Useful when we only want to isolate the targets for running elsewhere.

— def configure_and_sync(self, api, target_mastername, target_testername, revision, builders=None):

Applies compile/test configs & syncs code.

These are common tasks done in preparation ahead of building and testing chromium revisions, extracted as code share between the test and flake recipes.

Args: api (RecipeApi): With the dependencies injected by the calling recipe. target_mastername (str): Which master to derive the configuration off of. target_testername (str): likewise revision (str): A string representing the commit hash of the revision to test. builders (dict): A dict of the same format as api.chromium_tests.builders. Returns: (target_buildername, checked_out_revision, cached_revision)

— def existing_targets(self, targets, mb_mastername, mb_buildername):

Returns a sublist of the given targets that exist in the build graph.

We test whether a target exists or not by ninja.

A “target” here is actually a node in ninja's build graph. For example:

An executable target like browser_tests
An object file like obj/path/to/Source.o
An action like build/linux:gio_loader
An generated header file like gen/library_loaders/libgio.h
and so on

Args: targets (list): A list of targets to be tested for existence. mb_mastername (str): The mastername to run MB with. mb_buildername (str): The buildername to run MB with.

— def files_changed_by_revision(self, revision, solution_name=‘src’):

Returns the files changed by the given revision.

Args: revision (str): the git hash of a commit. solution_name (str): the gclient solution name, eg: “src” for chromium, “src/third_party/pdfium” for pdfium.

— def record_previous_revision(self, api, bot_config):

Records the latest checked out and cached revisions.

Examines the checkout and records the latest available revision for the first gclient solution.

This also records the latest revision available in the local git cache.

Args: api (RecipeApi): With the dependencies injected by the calling recipe Returns: A pair of revisions (checked_out_revision, cached_revision), or None, None if the checkout directory does not exist.

— def revisions_between(self, start_revision, end_revision, solution_name=‘src’):

Returns the git commit hashes between the given range.

Args: start_revision (str): the git hash of a parent commit. end_revision (str): the git hash of a child commit. solution_name (str): the gclient solution name, eg: “src” for chromium, “src/third_party/pdfium” for pdfium.

Returns: A list of git commit hashes between start_revision (excluded) and end_revision (included), ordered from older commits to newer commits.

recipe_modules / gae_sdk

DEPS: depot_tools/cipd, recipe_engine/platform

class GaeSdkApi(RecipeApi):

@property
— def all_packages(self):

— def fetch(self, plat, dst):

Fetch the AppEngine SDK for the specified platform.

Args: plat (str): platform string, one of the PLAT_ local variables. dst (path.Path): The destination directory to extract it.

@property
— def latest_ref(self):

— def package(self, plat, arch=None):

— def package_spec(self, plat, arch):

@property
— def platforms(self):

recipe_modules / gatekeeper

DEPS: puppet_service_account, depot_tools/gitiles, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

class Gatekeeper(RecipeApi):

Module for Gatekeeper NG.

recipe_modules / git_clone_bundler

DEPS: repo, depot_tools/git, depot_tools/gsutil, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/step

The ‘git_clone_bundler’ creates and uploads ‘clone.bundle’ packages to Google Storage for specified Git repositories.

class GitCloneBundlerApi(RecipeApi):

Provides methods to encapsulate repo operations.

@property
— def base_path(self):

@property
— def bundle_dir(self):

— def create(self, git_path, gs_bucket, gs_subpath=None, name=None, unauthenticated_url=False):

Args: git_path: (Path) The path of the Git repository to bundle. gs_bucket: (str) The name of the Google Storage bucket. gs_subpath: (str) The path within the Google Storage bucket. name (str): If not None, the name of this bundle operation (to differentiate steps). unauthenticated_url: (bool) If True, request an unauthenticated URL from Google Storage.

Returns: (str) The Google Storage URL where the bundle was uploaded.

— def create_repo(self, repo_manifest_url, gs_bucket, gs_subpath=None, remote_name=None, unauthenticated_url=False):

Traverses a ‘repo’ checkout and creates and uploads a Git bundle for each repository in the checkout.

Args: repo_manifest_url: (str) The URL of the manifest to check out. gs_bucket: (str) The name of the Google Storage bucket. gs_subpath: (str) The path within the Google Storage bucket. remote_name: (str) If not None, the name of the remote to query. This is used to build the returned repository-to-GS mapping. unauthenticated_url: (bool) If True, request an unauthenticated URL from Google Storage.

Returns: (dict) If ‘remote_name’ is supplied, a dictionary mapping the remote repository URL (key) to the Google Storage path (value) where that repository's bundle was uploaded.

recipe_modules / gn

DEPS: depot_tools/depot_tools, recipe_engine/context, recipe_engine/file, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class GnApi(RecipeApi):

— def clean(self, build_dir, step_name=‘clean outdir’):

Cleans the output directory except for args.gn and needed ninja files.

See https://gn.googlesource.com/gn/+/master/docs/reference.md#cmd_clean for more documentation of the command. The build_dir must exist and contain files from previous gn run.

Args: build_dir: Path to build output directory. step_name: Optional recipe step name to give to the “gn clean” command.

— def get_args(self, build_dir, location=None, max_text_lines=None, step_name=None):

Get the GN args for the build.

A step will be executed that fetches the args.gn file and adds the contents to the presentation for the step.

Args: build_dir: The Path to the build output directory. The args.gn file will be extracted from this location. The args.gn file must already exist (gn or mb should have already been run before calling this method). location: Controls where the GN args for the build should be presented. By default or if gn.DEFAULT, the args will be in step_text if the count of lines is less than max_text_lines or the logs otherwise. To force the presentation to the step_text or logs, use gn.TEXT or gn.LOGS, respectively. max_text_lines: The maximum number of lines of GN args to display in the step_text when using the default behavior for displaying GN args. step_name: The name of the step for reading the args.

Returns: The content of the args.gn file.

— def ls(self, build_dir, inputs, output_type=None, output_format=‘label’, step_name=‘list gn targets’):

List targets for a given set of inputs.

See https://gn.googlesource.com/gn/+/master/docs/reference.md#ls for more documentation of the command.

Args: build_dir: Path to build output directory. inputs: List of patterns to find matching targets for. output_type: Type of target (eg: “executable”, “shared_library”, etc.) to restrict outputs to. If None (default), no filtering is preformed. output_format: How to display targets. See GN docs for valid options. Default is “label”. step_name: Optional recipe step name to give to the “gn ls” command. Returns: The set of targets found.

— def present_args(self, result, args, location=None, max_text_lines=None):

Present the GN args.

Args: result: The step result to present the GN args on. args: A string containing the args.gn content to present. location: Controls where the GN args for the build should be presented. By default or if gn.DEFAULT, the args will be in step_text if the count of lines is less than max_text_lines or the logs otherwise. To force the presentation to the step_text or logs, use gn.TEXT or gn.LOGS, respectively. max_text_lines: The maximum number of lines of GN args to display in the step_text when using the default behavior for displaying GN args.

— def read_args(self, build_dir, step_name=None):

Read the GN args.

Returns: A tuple containing the contents of the args.gn file as a single string and the step result of reading the file.

— def reformat_args(self, args):

Reformat the GN args to be more useful for local repros.

Some of the arguments have values that are specific to individual runs and so a developer would likely not want to copy them to their own checkout when attempting to reproduce an issue locally. To enable ease of use, these arguments are moved to the end of the arguments so that a single contiguous region can be copied in order to create a build directory with the appropriate arguments.

Args: args: A string containing the args.gn content to reformat.

Returns: The reformatted args.gn content as a single string.

— def refs(self, build_dir, inputs, all_deps=True, output_type=None, output_format=‘label’, step_name=‘calculate gn refs’, **kwargs):

Find reverse dependencies for a given set of inputs.

See https://gn.googlesource.com/gn/+/master/docs/reference.md#refs for more documentation of the command.

Args: build_dir: Path to build output directory. inputs: List of label or files to find dependencies of. all_deps: Boolean indicating wether or not to include indirect dependencies. output_type: Type of target (eg: “executable”, “shared_library”, etc.) to restrict outputs to. If None (default), no filtering is preformed. output_format: How to display targets. See GN docs for valid options. Default is “label”. step_name: Optional recipe step name to give to the “gn refs” command. kwargs: Other arguments passed to the underlying python step. Returns: The set of dependencies found.

recipe_modules / goma

DEPS: build, puppet_service_account, depot_tools/cipd, depot_tools/depot_tools, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/time

class GomaApi(RecipeApi):

GomaApi contains helper functions for using goma.

For local running of goma recipe module, add the $build/goma[‘local’] property when running the recipe with the full path. e.g.

.../recipes.py run --properties-file - recipe_name <<EOF { ..., “$build/goma”: { “local”: “/path/to/workdir/goma/client” } } EOF

Note that the goma client directory must exist inside the recipe workdir. A symlink (on mac/linux) is enough, though.

@property
— def bigquery_service_account_json_path(self):

— def build_with_goma(self, ninja_command, name=None, ninja_log_outdir=None, ninja_log_compiler=None, goma_env=None, ninja_env=None, **kwargs):

Build with ninja_command using goma

Args: ninja_command: Command used for build. This is sent as part of log. (e.g. [‘ninja’, ‘-C’, ‘out/Release’]) name: Name of compile step. ninja_log_outdir: Directory of ninja log. (e.g. “out/Release”) ninja_log_compiler: Compiler used in ninja. (e.g. “clang”) goma_env: Environment controlling goma behavior. ninja_env: Environment for ninja.

Returns: TODO(tikuta): return step_result

Raises: StepFailure or InfraFailure if it fails to build or occurs something failure on goma steps.

@property
— def cloudtail_exe(self):

@property
— def cloudtail_pid_file(self):

@property
— def cloudtail_service_account_json_path(self):

@property
— def counterz_path(self):

@property
— def debug(self):

Returns true if debug mode is turned on.

Uses value from property “$build/goma:{“debug”:true}” if configured (typically in cr-buildbucket.cfg). Defaults to False.

@property
— def default_cache_path(self):

@property
— def default_cache_path_per_slave(self):

@property
— def default_client_path(self):

— def ensure_goma(self, client_type=None):

@property
— def goma_ctl(self):

@property
— def goma_dir(self):

— def initialize(self):

@property
— def jobs(self):

Returns number of jobs for parallel build using Goma.

Uses value from property “$build/goma:{“jobs”: JOBS}” if configured (typically in cr-buildbucket.cfg), else defaults to recommended_goma_jobs.

@property
— def json_path(self):

@property
— def jsonstatus(self):

@property
— def recommended_goma_jobs(self):

Return the recommended number of jobs for parallel build using Goma.

Prefer to use just goma.jobs and configure it through default builder properties in cr-buildbucket.cfg.

This function caches the _recommended_jobs.

@property
— def service_account_json_path(self):

— def start(self, env=None, **kwargs):

Start goma compiler_proxy.

A user MUST execute ensure_goma beforehand. It is user's responsibility to handle failure of starting compiler_proxy.

— def stop(self, build_exit_status, ninja_log_outdir=None, ninja_log_compiler=None, ninja_log_command=None, build_step_name='', **kwargs):

Stop goma compiler_proxy.

A user is expected to execute start beforehand. It is user's responsibility to handle failure of stopping compiler_proxy.

Args: build_exit_status: Exit status of ninja or other build commands like make. (e.g. 0) ninja_log_outdir: Directory of ninja log. (e.g. “out/Release”) ninja_log_compiler: Compiler used in ninja. (e.g. “clang”) ninja_log_command: Command used for build. (e.g. [‘ninja’, ‘-C’, ‘out/Release’])

Raises: StepFailure if it fails to stop goma or upload logs.

recipe_modules / halt

DEPS: recipe_engine/python

class HaltApi(RecipeApi):

— def __call__(self, reason=‘Unknown’, **kwargs):

Return a failing step with the given message.

recipe_modules / ios

DEPS: chromium, chromium_checkout, chromium_swarming, chromium_tests, filter, goma, isolate, perf_dashboard, swarming_client, test_results, test_utils, depot_tools/bot_update, depot_tools/cipd, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/time

class iOSApi(RecipeApi):

— def bootstrap_swarming(self):

Bootstraps Swarming.

@property
— def bucket(self):

— def build(self, analyze=False, mb_path=None, suffix=None, use_mb=True):

Builds from this bot's build config.

Args: analyze: Whether to use the gyp_chromium analyzer to only build affected targets and filter out unaffected tests. mb_path: Custom path to MB. Uses the default if unspecified. suffix: Suffix to use at the end of step names. use_mb: Whether or not to use mb to generate build files.

— def checkout(self, gclient_apply_config=None, **kwargs):

Checks out Chromium.

— def collect(self, triggered_tests):

@property
— def configuration(self):

— def ensure_xcode(self, xcode_build_version):

— def generate_test_from_task(self, task, upload_test_results=True, result_callback=None):

Generates a Test subclass that can run tests and parse/store results.

Returns: None if no tests should be run, otherwise an instance of SwarmingIosTest.

— def get_mac_toolchain_cmd(self):

@staticmethod
— def get_step_name(test):

— def isolate(self, scripts_dir=‘src/ios/build/bots/scripts’):

Isolates the tests specified in this bot's build config.

— def isolate_earlgrey_test(self, test, shard_size, tmp_dir, isolate_template, bot=None):

Isolate earlgrey test into small shards

— def isolate_test(self, test, tmp_dir, isolate_template, test_cases=None, shard_num=None):

Isolates a single test.

@property
— def most_recent_app_dir(self):

Returns the path (relative to checkout working dir) of the most recently compiled apps.

@property
— def most_recent_app_path(self):

Returns the Path to the directory of the most recently compiled apps.

@property
— def most_recent_iossim(self):

Returns the path to the most recently compiled iossim.

— def parse_tests(self, tests, include_dir, start_index=0):

Parses the tests dict, reading necessary includes.

Args: tests: A list of test dicts.

@property
— def platform(self):

— def read_build_config(self, master_name=None, build_config_base_dir=None, buildername=None):

Reads the iOS build config for this bot.

Args: master_name: Name of a master to read the build config from, or None to read from buildbot properties at run-time. build_config_base_dir: Directory to search for build config master and test include directories.

@swarming_service_account.setter
— def swarming_service_account(self, val):

— def symupload(self, artifact, url):

Uploads the given symbols file.

Args: artifact: Name of the artifact to upload. Will be found relative to the out directory, so must have already been compiled. url: URL of the symbol server to upload to.

— def test_swarming(self, scripts_dir=‘src/ios/build/bots/scripts’, upload_test_results=True, retry_failed_shards=False):

Runs tests on Swarming as instructed by this bot's build config.

— def upload(self, base_path=None):

Uploads built artifacts as instructed by this bot's build config.

— def upload_tgz(self, artifact, bucket, path):

Tar gzips and uploads the given artifact to Google Storage.

Args: artifact: Name of the artifact to upload. Will be found relative to the out directory, so must have already been compiled. bucket: Name of the Google Storage bucket to upload to. path: Path to upload the artifact to relative to the bucket.

@property
— def use_goma(self):

@property
— def xcode_build_version(self):

recipe_modules / isolate

DEPS: chromium, swarming_client, depot_tools/depot_tools, depot_tools/git, depot_tools/gsutil, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile

class IsolateApi(RecipeApi):

APIs for interacting with isolates.

— def check_swarm_hashes(self, targets):

Asserts that all the targets in the passed list are present as keys in the ‘swarm_hashes’ property.

This is just an optional early check that all the isolated targets that are needed throughout the build are present. Without this step, the build would just fail later, on a ‘trigger’ step. But the main usefulness of this is that it automatically populates the ‘swarm_hashes’ property in testing context, so that it doesn't need to be manually specified through test_api.

— def clean_isolated_files(self, build_dir):

Cleans out all *.isolated files from the build directory in preparation for the compile. Needed in order to ensure isolates are rebuilt properly because their dependencies are currently not completely described to gyp.

— def compare_build_artifacts(self, first_dir, second_dir):

Compare the artifacts from 2 builds.

— def compose(self, isolate_hashes, step_name=None, **kwargs):

Creates and uploads a new isolate composing multiple existing isolates.

In case of a conflict, where a given file is present in multiple isolates, the version of the file from an earlier isolate (as ordered in the isolate_hashes argument) is checked out on the bot.

Args: isolate_hashes: List of hashes of existing uploaded isolates.

Returns: Hash of the uploaded composite isolate.

— def find_isolated_tests(self, build_dir, targets=None, **kwargs):

Returns a step which finds all *.isolated files in a build directory.

Useful only with ‘archive’ isolation mode. In ‘prepare’ mode use ‘isolate_tests’ instead.

Assigns the dict {target name -> *.isolated file hash} to the swarm_hashes build property. This implies this step can currently only be run once per recipe.

If |targets| is None, the step will use all *.isolated files it finds. Otherwise, it will verify that all |targets| are found and will use only them. If some expected targets are missing, will abort the build.

@isolate_server.setter
— def isolate_server(self, value):

Changes URL of Isolate server to use.

— def isolate_tests(self, build_dir, targets=None, verbose=False, swarm_hashes_property_name=‘swarm_hashes’, step_name=None, suffix='', **kwargs):

Archives prepared tests in |build_dir| to isolate server.

src/tools/mb/mb.py is invoked to produce *.isolated.gen.json files that describe how to archive tests.

This step then uses *.isolated.gen.json files to actually performs the archival. By archiving all tests at once it is able to reduce the total amount of work. Tests share many common files, and such files are processed only once.

Args: targets: List of targets to use instead of finding .isolated.gen.json files. verbose (bool): Isolate command should be verbose in output. swarm_hashes_property_name (str): If set, assigns the dict {target name -> *.isolated file hash} to the named build property. If this needs to be run more than once per recipe run, make sure to pass different propery names for each invocation. suffix: suffix of isolate_tests step. e.g. ' (with patch)‘, ' (without patch)’.

@property
— def isolated_tests(self):

The dictionary of ‘target name -> isolated hash’ for this run.

These come either from the incoming swarm_hashes build property, or from calling find_isolated_tests, above, at some point during the run.

@namespace.setter
— def namespace(self, value):

The namespace determines the algorithms used.

— def run_isolated(self, name, isolate_hash, args=None, **kwargs):

Runs an isolated test.

@service_account_json.setter
— def service_account_json(self, value):

Service account json to use.

recipe_modules / legion

DEPS: chromium_swarming, isolate, swarming_client, recipe_engine/path, recipe_engine/python, recipe_engine/raw_io

class LegionApi(RecipeApi):

Provides a recipes interface for the Legion framework.

— def add_task_to_controller(self, controller, name, path, config_vars=None):

Adds a task config to a controller config.

Args: controller: A controller config returnd by create_controller. name: The name of the task. This corresponds to the command line flag defined in the controller code. path: The path to the .isolate or .isolated file for the task. config_vars: Config variables passed when isolating a task .isolate file. This is ignored if passing a .isolated file.

— def create_controller(self, name, path, os, config_vars=None, controller_vars=None, dimensions=None):

Returns a controller config dictionary.

Args: name: The name of the controller. path: The path to the .isolate or .isolated file for the controller. os: The os to run the controller on. config_vars: A dictionary of config vars to pass when isolating the controller .isolate file. This is ignored if passing a .isolated file. controller_vars: A dictionary of command line vars passed to the controller. dimensions: A dictionary of dimensions to pass when isolating the controller .isolate file. This is ignored if passing a .isolated file.

— def execute(self, config):

Executes a Legion-based swarming test.

config: The configuration returned by create_controller.

@property
— def legion_path(self):

Returns the path to legion.py.

recipe_modules / libyuv

DEPS: archive, chromium, chromium_android, chromium_checkout, trigger, depot_tools/bot_update, depot_tools/gclient, depot_tools/osx_sdk, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/scheduler, recipe_engine/step

class LibyuvApi(RecipeApi):

— def apply_bot_config(self, builders, recipe_configs):

— def checkout(self):

@contextlib.contextmanager
— def ensure_sdk(self):

— def extract_build(self):

— def maybe_trigger(self):

— def package_build(self):

— def runtests(self):

Add a suite of test steps.

@property
— def should_build(self):

@property
— def should_download_build(self):

@property
— def should_test(self):

@property
— def should_upload_build(self):

recipe_modules / math_utils

General statistical or mathematical functions.

class MathUtilsApi(RecipeApi):

@staticmethod
— def confidence_score(sample1, sample2, accept_single_bad_or_good=False):

Calculates a confidence score.

This score is a percentage which represents our degree of confidence in the proposition that the good results and bad results are distinct groups, and their differences aren't due to chance alone.

Args: sample1: A flat list of “good” result numbers. sample2: A flat list of “bad” result numbers. accept_single_bad_or_good: If True, computes confidence even if there is just one bad or good revision, otherwise single good or bad revision always returns 0.0 confidence. This flag will probably get away when we will implement expanding the bisect range by one more revision for such case.

Returns: A number in the range [0, 100].

@staticmethod
— def mean(values):

Calculates the arithmetic mean of a list of values.

@staticmethod
— def pooled_standard_error(work_sets):

Calculates the pooled sample standard error for a set of samples.

Args: work_sets: A collection of collections of numbers.

Returns: Pooled sample standard error.

@staticmethod
— def relative_change(before, after):

Returns the relative change of before and after, relative to before.

There are several different ways to define relative difference between two numbers; sometimes it is defined as relative to the smaller number, or to the mean of the two numbers. This version returns the difference relative to the first of the two numbers.

Args: before: A number representing an earlier value. after: Another number, representing a later value.

Returns: A non-negative floating point number; 0.1 represents a 10% change.

@staticmethod
— def standard_deviation(values):

Calculates the sample standard deviation of the given list of values.

@staticmethod
— def standard_error(values):

Calculates the standard error of a list of values.

@staticmethod
— def variance(values):

Calculates the sample variance.

@staticmethod
— def welchs_t_test(sample1, sample2):

Performs Welch's t-test on the two samples.

Welch‘s t-test is an adaptation of Student’s t-test which is used when the two samples may have unequal variances. It is also an independent two-sample t-test.

Args: sample1: A collection of numbers. sample2: Another collection of numbers.

Returns: A 3-tuple (t-statistic, degrees of freedom, p-value).

recipe_modules / ndk

DEPS: chromite

class NdkApi(RecipeApi):

recipe_modules / perf_dashboard

DEPS: depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/service_account, recipe_engine/step

class PerfDashboardApi(RecipeApi):

Provides steps to take a list of perf points and post them to the Chromium Perf Dashboard. Can also use the test url for testing purposes.

— def add_dashboard_link(self, presentation, test, revision, bot=None):

Adds a results-dashboard link to the step presentation.

Must be called from a follow-up function of the step, to which the link should be added. For a working link, the parameters test, revision and bot must match to the parameters used to upload points to the dashboard.

Args: presentation: A step presentation object. Can be obtained by step_result.presentation from a followup_fn of a step. test: Slash-separated test path. revision: The build revision, e.g. got_revision from the update step. bot: The bot name used for the data on the perf dashboard.

— def add_point(self, data, halt_on_failure=False, name=None, **kwargs):

— def get_change_info(self, commits):

— def get_skeleton_point(self, test, revision, value, bot=None):

— def halt(self, step_result, reason):

— def post(self, name, url, data, halt_on_failure, **kwargs):

Send a POST request to a URL with a payload.

Args: name: The name of the step. url: The URL to post to. data: A dict of parameters to send in the body of the request. halt_on_failure: If True, the step turns purple on failure. Otherwise, it turns orange. debug_info (list[str]|None): An optional list of log lines to add to the post step as debugging information.

— def set_default_config(self):

— def upload_isolate(self, builder_name, change, isolate_server, isolate_map, halt_on_failure=False, **kwargs):

— def warning(self, step_result, reason):

recipe_modules / puppet_service_account

DEPS: recipe_engine/path, recipe_engine/platform, recipe_engine/service_account

[DEPRECATED] API for generating OAuth2 access tokens from service account keys predeployed to Chrome Ops bots via Puppet.

Depends on ‘luci-auth’ being in PATH.

This module exists only to support Buildbot code. On LUCI use default account exposed through ‘recipe_engine/service_account’ module.

class PuppetServiceAccountApi(RecipeApi):

— def get(self, account):

Returns a recipe_module.service_account.ServiceAccount for the account.

Assumes a service account key for the given account is available at self.keys_path.

Args: account: a name of the service account, as defined in Puppet config. Returns A recipe_module.service_account.ServiceAccount instance.

— def get_access_token(self, account, scopes=None):

Returns an access token for a service account.

Token's lifetime is guaranteed to be at least 3 minutes and at most 45.

Args: account: a name of the service account, as defined in Puppet config. scopes: list of OAuth scopes for new token, default is [userinfo.email].

— def get_key_path(self, account):

Path to a particular JSON key (as str).

@property
— def keys_path(self):

Path to a directory where ChromeOps Puppet drops service account keys.

recipe_modules / repo

DEPS: depot_tools/depot_tools, depot_tools/infra_paths, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step

Common steps for recipes that use repo for source control.

class RepoApi(RecipeApi):

Provides methods to encapsulate repo operations.

— def __call__(self, args, name=None, **kwargs):

Executes ‘repo’ with the supplied arguments.

Args: args: (list): A list of arguments to supply to ‘repo’. name (str): The name of the step. If None, generate a name from the args. kwargs: Keyword arguments to pass to the ‘step’ call. Returns: See ‘step.call’.

— def clean(self, *args, **kwargs):

Clean an already-init'd repo.

— def init(self, url, *args, **kwargs):

Perform a ‘repo init’ step with the given manifest url.

— def list(self, **kwargs):

Return (list): A list of (path, name) project tuples. path (str): The relative path to the project's checkout. name (str): The name of the project in the manifst.

@property
— def repo_path(self):

— def reset(self, **kwargs):

Reset to HEAD an already-init'd repo.

— def sync(self, *args, **kwargs):

Sync an already-init'd repo.

recipe_modules / swarming_client

DEPS: depot_tools/git, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/swarming

class SwarmingClientApi(RecipeApi):

Code that both isolate and swarming recipe modules depend on.

Both swarming and isolate scripts live in a single repository called ‘swarming client’. This module include common functionality like finding existing swarming client checkout, fetching a new one, getting version of a swarming script, etc.

— def checkout(self, revision=None, curl_trace_file=None, can_fail_build=True):

Returns a step to checkout swarming client into a separate directory.

Ordinarily swarming client is checked out via Chromium DEPS into src/tools/swarming_client. This step configures recipe module to use a separate checkout.

If |revision| is None, this requires the build property ‘parent_got_swarming_client_revision’ to be present, and raises an exception otherwise. Fail-fast behavior is used because if machines silently fell back to checking out the entire workspace, that would cause dramatic increases in cycle time if a misconfiguration were made and it were no longer possible for the bot to check out swarming_client separately.

— def ensure_script_version(self, script, min_version, step_test_data=None):

Yields steps to ensure a script version is not older than |min_version|.

Will abort recipe execution if it is.

— def get_script_version(self, script):

Returns a version of some swarming script as a tuple (Major, Minor, Rev).

It should have been queried by ‘query_script_version’ step before. Raises AssertionError if it wasn't.

@property
— def path(self):

Returns path to a swarming client checkout.

It's subdirectory of Chromium src/ checkout or a separate directory if ‘checkout_swarming_client’ step was used.

— def query_script_version(self, script, step_test_data=None):

Yields a step to query a swarming script for its version.

Version tuple is later accessible via ‘get_script_version’ method. If |step_test_data| is given, it is a tuple with version to use in expectation tests by default.

Does nothing if script's version is already known.

recipe_modules / swarming_heartbeat

DEPS: test_utils, recipe_engine/properties, recipe_engine/python

class SwarmingHeartbeatApi(RecipeApi):

Defines support to run automated heartbeat.

— def run(self):

Runs job_runs_fine.py to run an heartbeat check.

recipe_modules / tar

DEPS: recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python

class TarApi(RecipeApi):

Provides steps to tar and untar files.

— def directory(self, step_name, directory, output):

Step to compress a single directory.

Args: step_name: display name of the step. directory: path to a directory to compress, it would become the root of an archive, i.e. |directory|/file.txt would be named ‘file.txt’ in the archive. output: path to a tar file to create.

— def make_package(self, root, output, compression=None):

Returns TarPackage object that can be used to compress a set of files.

Usage: pkg = api.tar.make_package(root, output, ‘bz2’) pkg.add_file(root.join(‘file’)) pkg.add_directory(root.join(‘directory’)) yield pkg.tar(‘taring step’)

Args: root: a directory that would become root of a package, all files added to an archive will have archive paths relative to this directory. output: path to a tar file to create.

Returns: TarPackage object.

— def untar(self, step_name, tar_file, output, quiet=False):

Step to uncompress |tar_file| into |output| directory.

Step will FAIL if |output| already exists.

Args: step_name: display name of a step. tar_file: path to a tar file to uncompress, should exist. output: path to a directory to unpack to, it should NOT exist. quiet (bool): If True, print terse output instead of the name of each untared file.

recipe_modules / test_results

DEPS: recipe_engine/buildbucket, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

class TestResultsApi(RecipeApi):

Recipe module to upload gtest json results to test-results server.

— def initialize(self):

Set the default config.

— def upload(self, results_file, test_type, chrome_revision, test_results_server=None, downgrade_error_to_warning=True, builder_name_suffix=''):

Upload results json to test-results.

Args: results_file: Path to file containing result json. Supported format are: ttest format & full json results format ( http://www.chromium.org/developers/the-json-test-results-format). test_type: Test type string, e.g. blink_web_tests. test_results_server: Server where results should be uploaded. downgrade_error_to_warning: If True, treat a failure to upload as a warning. builder_name_suffix: Optional suffix to append to the builder name.

Returns: The step result.

recipe_modules / test_utils

DEPS: build, chromium, traceback, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class TestUtilsApi(RecipeApi):

This class helps run tests and parse results.

Tests are run in [up to] three stages:

‘with patch’
‘retry shards with patch’
‘without patch’

The first stage applies the patch and runs the tests. If this passes, we're finished. Assuming that tests fail or return invalid results, then we deapply the patch and try running the tests again. If the failures are the same, then this is an issue with tip of tree and we ignore the failures.

Finally, we roll the checkout and reapply the patch, and then rerun the failing tests. This helps confirm whether the failures were flakes or deterministic errors.

@property
— def canonical(self):

— def create_results_from_json(self, data):

@staticmethod
— def format_step_text(data):

Returns string suitable for use in a followup function‘s step result’s presentation step text.

Args: data - iterable of sections, where each section is one of: a) tuple/list with one element for a single-line section (always displayed) b) tuple/list with two elements where first one is the header, and the second one is an iterable of content lines; if there are no contents, the whole section is not displayed

@returns_placeholder
— def gtest_results(self, add_json_log=True):

A placeholder which will expand to ‘--test-launcher-summary-output=/tmp/file’.

Provides the --test-launcher-summary-output flag since --flag=value (i.e. a single token in the command line) is the required format.

The test_results will be an instance of the GTestResults class.

— def limit_failures(self, failures, limit=None):

Limit failures of a step to prevent large results JSON.

Args: failures - An iterable containing the failures that resulted from some step. limit - The maxmium number of failures to display in the results. Returns: A tuple containing 2 elements: 1. The list of the subset of at most limit elements of failures, suitable for iterating over and indexing into structures that are indexed by the failure names. 2. The list of failures suitable for including in the step's text. If failures contains more elements than limit, it will contain an element indicating the number of additional failures.

— def present_gtest_failures(self, step_result, presentation=None):

Update a step result's presentation with details of gtest failures.

If the provided step result contains valid gtest results, then the presentation will be updated to include information about the failing tests, including logs for the individual failures.

The max_reported_failures property modifies this behavior by limiting the number of tests that will appear in the step text and have their logs included. If the limit is exceeded the step text will indicate the number of additional failures.

Args: step_result - The step result that potentially contains gtest results. presentation - The presentation to update. If not provided or None, the presentation of step_result will be updated. Returns: The gtest_results object if it is present in the step result, otherwise None.

— def run_tests(self, caller_api, tests, suffix, sort_by_shard=False):

Utility function for running a list of tests and returning the failed tests.

Args: caller_api - caller‘s recipe API; this is needed because self.m here is different than in the caller (different recipe modules get injected depending on caller’s DEPS vs. this module‘s DEPS) This must include the ‘swarming’ recipe module, in order to use the grouping logic in this method. Unfortunately we can’t import this module in the test_utils module, as it would cause a circular dependency. tests - iterable of objects implementing the Test interface above suffix - custom suffix, e.g. “with patch”, “without patch” indicating context of the test run sort_by_shard - sort the order of triggering depends on the number of shards. Returns: A tuple of (list of tests with invalid results, list of tests which failed)

— def run_tests_with_patch(self, caller_api, tests, retry_failed_shards=False):

Run tests and returns failures.

Args: caller_api: The api object given by the caller of this module. tests: A list of test suites to run with the patch. retry_failed_shards: If true, attempts to retry failed shards of swarming tests.

Returns: A tuple (invalid_test_suites, all_failing_test_suites). invalid_test_suites: Test suites that do not have valid test results. all_failing_test_suites: This includes test suites than ran but have failing tests, test suites that do not have valid test results, and test suites that failed with otherwise unspecified reasons. This is a superset of invalid_test_suites.

— def summarize_failing_test_with_no_retries(self, caller_api, test_suite):

Summarizes a failing test suite that is not going to be retried.

— def summarize_findit_flakiness(self, caller_api, test_suites):

Exports a summary of flakiness for post-processing by FindIt.

There are several types of test flakiness. FindIt categories these by the layer at which the flakiness is discovered. One of these categories is for a test that fails, but when retried in a separate step, succeeds. This currently applies to ‘retry shards with patch’. These are labeled ‘Step Layer Flakiness’.

FindIt also wants to know about ‘with patch’ tests that caused the build to fail. If a future build with the same CL succeeds, then the tests are potential flakes. Although it's also possible that rolling tip of tree caused the results to change. These are labeled ‘Failing With Patch Tests That Caused Build Failure’.

This function emits a step with a fixed name, and metadata for FindIt. Before making changes to this function, check with the FindIt team to ensure that their post-processing will still work correctly.

— def summarize_test_with_patch_deapplied(self, caller_api, test_suite):

Summarizes test results after a CL has been retried with patch deapplied.

Args: If there are no new failures, this method will emit a passing step. If there are new failures, this method will emit a step whose presentation status is ‘FAILURE’.

Returns: A Boolean describing whether the retry succeeded. Which is to say, all tests that failed in the original run also failed in the retry, which suggests that the error is due to an issue with top of tree, and should not cause the CL to fail.

@returns_placeholder
— def test_results(self, add_json_log=True):

A placeholder which will expand to ‘/tmp/file’.

The recipe must provide the expected --json-test-results flag.

The test_results will be an instance of the TestResults class.

recipe_modules / traceback

DEPS: recipe_engine/path, recipe_engine/step

class TracebackApi(RecipeApi):

— def format_exc(self):

Returns a string containing an exception traceback.

Calls traceback.format_exc but during testing removes absolute paths.

recipe_modules / trigger

DEPS: recipe_engine/step

This recipe module allows triggering builds within the same master.

See README.md.

class TriggerApi(RecipeApi):

APIs for triggering new builds.

— def __call__(self, *trigger_specs, **kwargs):

Triggers new builds by builder names.

Args: trigger_specs: a list of trigger dicts, where each dict specifies a build to trigger. Supported keys: builder_name (str): in BuildBot context, builder name bucket (str): buildbucket bucket where the triggered builds will be placed. critical (bool): if True (default) and triggering fails on the buildbot master side, mark entire build as failed (exception). properties (dict): build properties for a new build. buildbot_changes (list of dict): list of Buildbot changes to create. See below. tags (dict or list of str): if the trigger build is scheduled through buildbucket, |tags| will be appended to the buildbucket build. name: name of the step. If not specified, it is generated automatically. Its format may change in future.

Buildbot changes: buildbot_changes (a list of dicts) is a list of changes for the triggered builds. Each change is a dict with keys (all optional): author (str) revision revlink (str): link to a web view of the revision. comment when_timestamp (int): timestamp of the change, in seconds since Unix Epoch. branch category (str): Buildbot change category files (list of str): list of changed filenames The first change is used to populate source stamp properties.

Examples: Basic: api.trigger({ ‘builder_name’: ‘Release’, ‘properties’: { ‘my_prop’: 123, }, })

Using BuildBucket: api.trigger({ ‘builder_name’: ‘Release’, ‘bucket’: ‘master.tryserver.chromium.linux’, ‘properties’: { ‘my_prop’: 123, }, }) WARNING: on buildbot, this requires certain configuration on the master prior first use. See ./README.md.

Create Buildbot changes: api.trigger({ ‘builder_name’: ‘Release’, ‘buildbot_changes’: [{ ‘author’: ‘someone@chromium.org’, ‘branch’: ‘master’, ‘files’: [‘a.txt.’], ‘comments’: ‘Refactoring’, ‘revision’: ‘deadbeef’, ‘revlink’: ‘http://chromium.googlesource.com/chromium/src/+/deadbeef’, ‘when_timestamp’: 1416859562, }] })

recipe_modules / v8

DEPS: archive, build, chromium, chromium_swarming, docker, gn, isolate, perf_dashboard, test_utils, trigger, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, depot_tools/gitiles, depot_tools/gsutil, depot_tools/osx_sdk, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/cq, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/scheduler, recipe_engine/step, recipe_engine/swarming, recipe_engine/tempfile, recipe_engine/time, recipe_engine/url

class V8Api(RecipeApi):

— def apply_bot_config(self, bot_config):

Entry method for using the v8 api.

— def bot_config_by_buildername(self, builders=None, use_goma=True):

@property
— def bot_type(self):

@property
— def build_output_dir(self):

Absolute path to the build product based on the ‘checkout’ path.

— def buildbucket_trigger(self, requests, project=None, bucket=None, step_name=‘trigger’):

Triggers builds via buildbucket.

Args: requests: List of 2-tuples (builder_name, properties). project: Project to trigger builds in (defaults to same as parent). bucket: Bucket to trigger builds in (defaults to same as parent). step_name: Name of the triggering step that appear on the build.

Returns: List of api.buildbucket.build_pb2.Build messages.

@property
— def builderset(self):

Returns a list of names of this builder and all its triggered testers.

— def calc_missing_values_in_sequence(self, sequence, subsequence, value):

Calculate a list of missing values from a subsequence.

Args: sequence: The complete sequence including all values. subsequence: A subsequence from the sequence above. value: An element from subsequence. Returns: A subsequence from sequence [a..b], where b is the value and for all x in a..b-1 holds x not in subsequence. Also a-1 is either in subsequence or value was the first element in subsequence.

— def calculate_patch_base_gerrit(self):

Calculates the commit hash a gerrit patch was branched off.

— def checkout(self, revision=None, **kwargs):

— def collect_post_compile_metrics(self):

— def compile(self, test_spec=v8_builders.EmptyTestSpec, mb_config_path=None, out_dir=None, **kwargs):

Compile all desired targets and isolate tests.

Args: test_spec: Optional TestSpec object as returned by read_test_spec(). Expected to contain only specifications for the current builder and all triggered builders. All corrensponding extra targets will also be isolated. mb_config_path: Path to the MB config file. Defaults to infra/mb/mb_config.py in the checkout. out_dir: Name of the build output directory, e.g. ‘out-ref’. Defaults to ‘out’. Note that it is not a path, but just the name of the directory.

— def create_coverage_context(self):

— def create_test(self, test):

Wrapper that allows to shortcut common tests with their names.

Returns: A runnable test instance.

— def create_tests(self):

— def dedupe_tests(self, high_prec_tests, low_prec_tests):

Dedupe tests with lower precedence.

@property
— def depot_tools_path(self):

Returns path to depot_tools pinned in the V8 checkout.

— def download_isolated_json(self, revision):

@property
— def extra_flags(self):

— def extra_tests_from_properties(self):

Returns runnable testing.BaseTest objects for each extra test specified by parent_test_spec property.

— def extra_tests_from_test_spec(self, test_spec):

Returns runnable testing.BaseTest objects for each extra test specified in the test spec of the current builder.

@staticmethod
— def format_duration(duration_in_seconds):

@property
— def generate_gcov_coverage(self):

@property
— def generate_sanitizer_coverage(self):

— def get_available_range(self, bisect_range):

— def get_change_range(self):

— def get_test_roots(self):

Returns the list of default and extensible test root directories.

A test root is a directory with the following layout: /infra/testing/config.pyl (optional) /infra/testing/builders.pyl /test/ (optional)

By default, the V8 checkout is a test root, and all matching directories under v8/custom_deps.

Returns: List of paths to test roots.

— def init_gcov_coverage(self):

Delete all gcov counter files.

@property
— def is_pure_swarming_tester(self):

— def isolate_targets_from_tests(self, tests):

Returns the isolated targets associated with a list of tests.

Args: tests: A list of test names used as keys in the V8 API's test config.

— def isolate_tests(self, isolate_targets, out_dir=None):

Upload isolated tests to isolate server.

Args: isolate_targets: Targets to isolate. out_dir: Name of the build output directory, e.g. ‘out-ref’. Defaults to ‘out’. Note that it is not a path, but just the name of the directory.

@property
— def isolated_archive_path(self):

— def load_dynamic_test_configs(self, root):

Add test configs from configured location.

The test configs in /infra/testing/config.pyl are expected to follow the same structure as the TEST_CONFIGS dict in testing.py.

Args: test_checkout: Path to test root, can either be the V8 checkout or an additional test checkout. Returns: Test config dict.

— def load_static_test_configs(self):

Set predifined test configs from build repository.

— def log_available_range(self, available_bisect_range):

— def maybe_bisect(self, test_results):

Build-local bisection for one failure.

— def maybe_create_clusterfuzz_archive(self, update_step):

@contextlib.contextmanager
— def maybe_nest(self, condition, parent_step_name):

— def maybe_trigger(self, test_spec=v8_builders.EmptyTestSpec, **additional_properties):

— def parse_revision_props(self, got_revision, got_revision_cp=None):

Parses got_revision and got_revision_cp properties.

Sets self.revision, self.revision_cp and self.revision_number.

Normally this is called from self.checkout above, but it may also be useful on bots where we do not have a checkout but have these properties (e.g. set by the parent builder when triggering child) and need to parse them.

Args: got_revision: Full git hash of the commit. got_revision_cp: Value of the Cr-Commit-Position commit footer, e.g. “refs/heads/master@”.

— def read_test_spec(self, root):

Reads a test specification file under /infra/testing/builders.pyl.

Args: root: Path to checkout root with configurations. Returns: v8_builders.TestSpec object, filtered by interesting builders (current builder and all its triggered testers).

— def read_version_file(self, ref, step_name_desc):

Read and return the version-file content at a paricular ref.

— def read_version_from_ref(self, ref, step_name_desc):

Read and return the version at a paricular ref.

@property
— def relative_path_to_d8(self):

— def report_culprits(self, culprit_range):

— def runhooks(self, **kwargs):

— def runtests(self, tests):

— def set_chromium_configs(self, clobber, default_targets):

— def set_gclient_custom_deps(self, custom_deps):

Configures additional gclient custom_deps to be synced.

— def set_gclient_custom_var(self, var_name):

Sets the gclient custom var var_name if given.

This customizes gclient sync, based on conditions on the variable in the V8 DEPS file.

— def set_up_swarming(self):

@property
— def should_build(self):

@property
— def should_collect_post_compile_metrics(self):

@property
— def should_test(self):

@property
— def should_upload_build(self):

@property
— def target_bits(self):

Returns target bits (as int) inferred from gn arguments from MB.

@property
— def test_filter(self):

— def testing_random_seed(self):

Return a random seed suitable for v8 testing.

If there are isolate hashes, build a random seed based on the hashes. Otherwise use the system's PRNG. This uses a deterministic seed for recipe simulation.

— def ui_test_label(self, full_test_name):

— def update_bot_config(self, bot_config, binary_size_tracking, build_config, clusterfuzz_archive, coverage, enable_swarming, target_arch, target_platform, track_build_dependencies, triggers, triggers_proxy):

Update bot_config dict with src-side properties.

Args: bot_config: The bot_config dict to update. binary_size_tracking: Additional configurations to enable binary size tracking. build_config: Config value for BUILD_CONFIG in chromium recipe module. clusterfuzz_archive: Additional configurations set for archiving builds to GS buckets for clusterfuzz. coverage: Optional coverage setting. enable_swarming: Switch to enable/disable swarming. target_arch: Config value for TARGET_ARCH in chromium recipe module. target_platform: Config value for TARGET_PLATFORM in chromium recipe module. track_build_dependencies: Weather to track and upload build-dependencies. triggers: List of tester names to trigger on success. triggers_proxy: Weather to trigger the internal trigger proxy.

Returns: An updated copy of the bot_config dict.

— def update_test_configs(self, test_configs):

Update test configs without mutating previous copy.

— def upload_build(self, name_suffix='', archive=None):

— def upload_gcov_coverage_report(self):

Capture coverage data and upload a report.

— def upload_isolated_json(self):

@staticmethod
— def version_from_file(blob):

recipe_modules / webrtc

DEPS: adb, archive, build, chromium, chromium_android, chromium_checkout, chromium_swarming, chromium_tests, goma, ios, isolate, test_results, test_utils, trigger, zip, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/git, depot_tools/gitiles, depot_tools/gsutil, depot_tools/osx_sdk, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/scheduler, recipe_engine/step, recipe_engine/tempfile

class WebRTCApi(RecipeApi):

— def apply_bot_config(self, builders, recipe_configs):

— def apply_ios_config(self):

Generate a JSON config from bot config, apply it to ios recipe module.

@property
— def bucketname(self):

— def build_android_archive(self):

@property
— def build_url(self):

@property
— def buildername(self):

— def check_swarming_version(self):

— def checkout(self, **kwargs):

— def compile(self, phase=None):

— def configure_isolate(self, phase=None):

— def configure_swarming(self):

— def download_audio_quality_tools(self):

— def download_video_quality_tools(self):

@contextlib.contextmanager
— def ensure_sdk(self):

— def get_binary_sizes(self, files=None, base_dir=None):

— def get_bot(self, bucketname, buildername):

— def isolate(self):

@property
— def master_config(self):

@property
— def mastername(self):

— def maybe_trigger(self):

— def package_apprtcmobile(self):

— def related_bots(self):

@property
— def revision_number(self):

— def run_mb(self, phase=None):

— def run_mb_ios(self):

— def runtests(self, phase=None):

Add a suite of test steps.

Args: test_suite=The name of the test suite.

@property
— def should_download_audio_quality_tools(self):

@property
— def should_download_video_quality_tools(self):

— def upload_to_perf_dashboard(self, name, step_result):

recipe_modules / zip

DEPS: recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python

class ZipApi(RecipeApi):

Provides steps to zip and unzip files.

— def directory(self, step_name, directory, output):

Step to compress a single directory.

— def make_package(self, root, output):

Returns ZipPackage object that can be used to compress a set of files.

Usage: pkg = api.zip.make_package(root, output) pkg.add_file(root.join(‘file’)) pkg.add_directory(root.join(‘directory’)) yield pkg.zip(‘zipping step’)

Args: root: a directory that would become root of a package, all files added to an archive will have archive paths relative to this directory. output: path to a zip file to create.

Returns: ZipPackage object.

— def unzip(self, step_name, zip_file, output, quiet=False):

Step to uncompress |zip_file| into |output| directory.

Step will FAIL if |output| already exists.

Args: step_name: display name of a step. zip_file: path to a zip file to uncompress, should exist. output: path to a directory to unpack to, it should NOT exist. quiet (bool): If True, print terse output instead of the name of each unzipped file.

Recipes

recipes / adb:examples/full

DEPS: adb, recipe_engine/path, recipe_engine/step

— def RunSteps(api):

recipes / android/builder

DEPS: chromium, chromium_android, chromium_tests, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, mastername, buildername, revision):

recipes / angle

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gsutil, depot_tools/osx_sdk, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step, recipe_engine/time

— def RunSteps(api, target_cpu, debug, clang):

recipes / annotated_run_test

DEPS: recipe_engine/properties, recipe_engine/step

Checks that properties get to recipes from annotated_run properly

— def RunSteps(api, true_prop, num_prop, string_prop, dict_prop):

recipes / archive:examples/full

DEPS: archive, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / archive:tests/download_and_unzip_build

DEPS: archive, recipe_engine/path, recipe_engine/properties

— def RunSteps(api):

recipes / archive:tests/zip_and_upload_build

DEPS: archive, depot_tools/bot_update, depot_tools/gclient, recipe_engine/path, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / art

DEPS: repo, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

— def checkout(api):

— def clobber(api):

— def full_checkout(api):

— def setup_aosp_builder(api, read_barrier):

— def setup_host_x86(api, debug, bitness, concurrent_collector=True, generational_cc=True, heap_poisoning=False, gcstress=False, cdex_level=‘none’):

— def setup_target(api, device, debug, concurrent_collector=True, generational_cc=True, heap_poisoning=False, gcstress=False):

recipes / binary_size_trybot

DEPS: chromium, chromium_android, chromium_checkout, chromium_tests, filter, depot_tools/bot_update, depot_tools/gclient, depot_tools/gerrit, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step, recipe_engine/tempfile, recipe_engine/time

— def RunSteps(api, analyze_targets, compile_targets, apk_name):

recipes / blink_downstream

DEPS: build, chromium, chromium_checkout, chromium_swarming, chromium_tests, isolate, test_utils, depot_tools/bot_update, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties

This recipe can be used by components like v8 to verify blink tests with a low false positive rate. Similar to a trybot, this recipe compares test failures from a build with a current component revision with test failures from a build with a pinned component revision.

Summary of the recipe flow:

Sync chromium to HEAD
Sync blink to HEAD
Sync component X to revision Y
Run blink tests -> In case of failures:
Sync chromium to same revision as 1
Sync blink to same revision as 2
Sync component X to pinned revision from DEPS file
Run blink tests -> If failures in 4 don't happen in 8, then revision Y reveals a problem not present in the pinned revision

Revision Y will be the revision property as provided by buildbot or HEAD (i.e. in a forced build with no revision provided).

— def RunSteps(api):

— def V8Builder(config, bits, platform, swarming_shards, swarming_priority=35):

— def build(api, suffix):

Compiles and isolates the checked-out code.

Args: api: Recipe module api. suffix: Step name suffix to disambiguate repeated calls.

recipes / boringssl

DEPS: chromium, test_utils, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / boringssl_docs

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

Generates BoringSSL documentation and uploads it to Cloud Storage.

— def RunSteps(api):

recipes / build:examples/full

DEPS: build

— def RunSteps(api):

recipes / catapult

DEPS: chromium, gae_sdk, depot_tools/bot_update, depot_tools/cipd, depot_tools/gclient, depot_tools/gitiles, depot_tools/osx_sdk, recipe_engine/context, recipe_engine/generator_script, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime

— def RunSteps(api, platform):

recipes / celab

DEPS: zip, depot_tools/bot_update, depot_tools/cipd, depot_tools/gclient, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step, recipe_engine/time

— def RunSteps(api):

recipes / chromite:examples/full

DEPS: chromite, depot_tools/gitiles, recipe_engine/properties

— def RunSteps(api):

recipes / chromium

DEPS: adb, build, chromium, chromium_android, chromium_swarming, chromium_tests, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step, recipe_engine/tempfile

— def RunSteps(api):

— def ignore_undumpable(obj):

recipes / chromium:examples/full

DEPS: chromium, chromium_tests, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / chromium:tests/analyze

DEPS: chromium, filter

— def RunSteps(api):

recipes / chromium:tests/archive_build

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium:tests/clean_outdir

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient

— def RunSteps(api):

recipes / chromium:tests/compile

DEPS: chromium, goma, recipe_engine/path, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium:tests/configs

DEPS: chromium, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/configure_bot

DEPS: chromium, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/gclient

DEPS: depot_tools/gclient, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/get_env

DEPS: chromium, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium:tests/get_version

DEPS: chromium, recipe_engine/assertions

— def RunSteps(api):

recipes / chromium:tests/mb_analyze

DEPS: chromium, chromium_checkout, depot_tools/gclient, recipe_engine/context, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium:tests/mb_gen

DEPS: chromium, recipe_engine/path, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/mb_isolate_everything

DEPS: chromium

— def RunSteps(api):

recipes / chromium:tests/mb_lookup

DEPS: chromium, recipe_engine/assertions, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api):

recipes / chromium:tests/process_dumps

DEPS: chromium

— def RunSteps(api):

recipes / chromium:tests/run_gn

DEPS: chromium, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/runhooks

DEPS: chromium, recipe_engine/properties

— def RunSteps(api):

recipes / chromium:tests/runtest

DEPS: chromium, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium:tests/sizes

DEPS: chromium, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium:tests/taskkill

DEPS: chromium

— def RunSteps(api):

recipes / chromium:tests/version

DEPS: chromium, recipe_engine/assertions

— def RunSteps(api):

recipes / chromium_afl

DEPS: archive, chromium, depot_tools/bot_update, depot_tools/depot_tools, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

— def gn_refs(api, step_name, args):

Runs gn refs with given additional arguments. Returns: the list of matched targets.

recipes / chromium_android:examples/full

DEPS: adb, build, chromium, chromium_android, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, buildername):

recipes / chromium_android:tests/android_build_wrapper

DEPS: chromium, chromium_android, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_android:tests/configs

DEPS: chromium, chromium_android, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/download_build

DEPS: chromium_android, depot_tools/bot_update, depot_tools/gclient, recipe_engine/context, recipe_engine/path, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/get_changed_lines_for_revision

DEPS: chromium_android

— def RunSteps(api):

recipes / chromium_android:tests/git_number

DEPS: chromium_android, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/incremental_coverage_report

DEPS: chromium_android, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/init_and_sync

DEPS: chromium_android

— def RunSteps(api):

recipes / chromium_android:tests/logcat_dump

DEPS: chromium, chromium_android, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/provision_devices

DEPS: chromium, chromium_android

— def RunSteps(api):

recipes / chromium_android:tests/provision_devices_test

DEPS: chromium_android

— def RunSteps(api):

recipes / chromium_android:tests/read_blacklist

DEPS: chromium_android, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_android:tests/resource_sizes

DEPS: chromium, chromium_android, recipe_engine/path, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/run_instrumentation_suite

DEPS: chromium, chromium_android, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/run_java_unit_test_suite

DEPS: chromium, chromium_android, recipe_engine/json

— def RunSteps(api):

recipes / chromium_android:tests/run_test_suite

DEPS: chromium, chromium_android, recipe_engine/json

— def RunSteps(api):

recipes / chromium_android:tests/run_webview_cts_test

DEPS: chromium, chromium_android, test_utils, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_android:tests/stackwalker

DEPS: chromium, chromium_android, recipe_engine/path

— def RunSteps(api):

recipes / chromium_android:tests/upload_apks_for_bisect

DEPS: build, chromium, chromium_android, depot_tools/bot_update, depot_tools/gclient

— def RunSteps(api):

recipes / chromium_android:tests/upload_build

DEPS: build, chromium, chromium_android

— def RunSteps(api):

recipes / chromium_android:tests/zip_and_upload_build

DEPS: chromium, chromium_android

— def RunSteps(api):

recipes / chromium_checkout:tests/full

DEPS: chromium_checkout, chromium_tests, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / chromium_clang_coverage_reports

DEPS: chromium, depot_tools/bot_update, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_clang_coverage_tot

DEPS: chromium, chromium_checkout, depot_tools/bot_update, depot_tools/depot_tools, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_codesearch

DEPS: build, chromium, codesearch, goma, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/time

— def RunSteps(api, root_solution_revision, root_solution_revision_timestamp):

recipes / chromium_codesearch_initiator

DEPS: recipe_engine/buildbucket, recipe_engine/json, recipe_engine/properties, recipe_engine/runtime, recipe_engine/scheduler, recipe_engine/step, recipe_engine/time, recipe_engine/url

A recipe for picking a stable HEAD revision for chromium/src.

This recipe picks a commit at HEAD, and then triggers the other codesearch recipes with the chosen commit hash as a parameter. This ensures that codesearch index packs (used to generate xrefs) are all generated from the same revision.

— def RunSteps(api):

recipes / chromium_gsutil:examples/full

DEPS: chromium_gsutil, recipe_engine/path

— def RunSteps(api):

recipes / chromium_libfuzzer

DEPS: archive, chromium, chromium_checkout, gn, depot_tools/bot_update, depot_tools/depot_tools, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_libfuzzer_clang_tot

DEPS: archive, chromium, depot_tools/bot_update, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_libfuzzer_trybot

DEPS: chromium, chromium_checkout, chromium_tests, filter, gn, depot_tools/bot_update, depot_tools/gclient, depot_tools/gerrit, depot_tools/tryserver, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_swarming:examples/full

DEPS: chromium_checkout, chromium_swarming, isolate, swarming_client, test_utils, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api, platforms, custom_trigger_script, show_outputs_ref_in_collect_step, show_shards_in_collect_step, gtest_task, isolated_script_task, merge, trigger_script, named_caches, service_account, wait_for_tasks):

recipes / chromium_swarming:tests/configure_swarming

DEPS: chromium, chromium_swarming, recipe_engine/path, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium_swarming:tests/shards_to_retry

DEPS: chromium_swarming, recipe_engine/assertions, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, task_to_retry, expected_value):

recipes / chromium_swarming:tests/task

DEPS: chromium_swarming, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_swarming:tests/trigger_output

DEPS: chromium_swarming, recipe_engine/assertions, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, task_to_retry, expected_value):

recipes / chromium_tests:examples/full

DEPS: chromium, chromium_tests, filter, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/json, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/archive_build

DEPS: chromium_tests, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python

— def RunSteps(api):

recipes / chromium_tests:tests/api/bot_id

DEPS: chromium_tests, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/compile_specific_targets

DEPS: chromium, chromium_tests, depot_tools/tryserver, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python

— def RunSteps(api):

recipes / chromium_tests:tests/api/configure_build

DEPS: chromium_tests, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/create_bot_config_object

DEPS: chromium_tests, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/create_bot_db_from_master_dict

DEPS: chromium_tests

— def RunSteps(api):

recipes / chromium_tests:tests/api/create_test_runner

DEPS: chromium, chromium_tests, test_results, depot_tools/tryserver, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/deapply_patch

DEPS: chromium_tests, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/download_and_unzip_build

DEPS: chromium_tests, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/get_common_args_for_scripts

DEPS: chromium_tests, recipe_engine/path, recipe_engine/properties, recipe_engine/python

— def RunSteps(api):

recipes / chromium_tests:tests/api/main_waterfall_steps

DEPS: chromium_swarming, chromium_tests, recipe_engine/json, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium_tests:tests/api/package_build

DEPS: chromium_tests, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python

— def RunSteps(api, builders):

recipes / chromium_tests:tests/api/prepare_checkout

DEPS: chromium, chromium_tests, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/runhooks

DEPS: chromium_checkout, chromium_tests, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/api/set_up_swarming

DEPS: chromium_swarming, chromium_tests, isolate, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium_tests:tests/api/trigger_child_builds

DEPS: chromium_tests, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python

— def RunSteps(api):

recipes / chromium_tests:tests/api/trybot_steps

DEPS: chromium_swarming, chromium_tests, filter, test_utils, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime

— def RunSteps(api):

recipes / chromium_tests:tests/api/trybot_steps_with_specific_tests

DEPS: chromium_swarming, chromium_tests, test_utils, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, mastername, buildername):

recipes / chromium_tests:tests/api/wrap_chromium_tests

DEPS: chromium_tests, depot_tools/tryserver, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/builders

DEPS: chromium_tests, recipe_engine/platform, recipe_engine/properties

Test to ensure the validity of the entries within BUILDERS.

Each entry in the BUILDERS dict will be checked to ensure chromium_tests.configure_build can be called with a BotConfig for that builder without error.

— def RunSteps(api, mastername, buildername):

— def validate_tester_config(api, mastername, buildername, bot_config):

recipes / chromium_tests:tests/configs

DEPS: chromium, chromium_tests, depot_tools/gclient, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/steps/android_junit_test

DEPS: chromium, chromium_android, chromium_tests, test_results, test_utils, depot_tools/bot_update, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/blink_test

DEPS: build, chromium, chromium_tests, test_results, test_utils, depot_tools/bot_update, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/experimental_test

DEPS: chromium_tests, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/find_annotated_test

DEPS: chromium, chromium_tests, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step, recipe_engine/tempfile

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_cts_test

DEPS: chromium, chromium_android, chromium_tests, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_fuchsia_test

DEPS: chromium, chromium_tests, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_gtest

DEPS: chromium, chromium_swarming, chromium_tests, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_isolated_script

DEPS: build, chromium, chromium_swarming, chromium_tests, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_junit_test

— def RunSteps(api):

recipes / chromium_tests:tests/steps/generate_script

DEPS: chromium, chromium_tests, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/get_args_for_test

DEPS: chromium, chromium_tests, test_results, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/properties

— def RunSteps(api):

recipes / chromium_tests:tests/steps/gtest_test

DEPS: chromium, chromium_tests, test_results, test_utils, depot_tools/bot_update, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/incremental_coverage_test

DEPS: chromium_android, chromium_tests, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/json_results_handler

DEPS: chromium_tests, test_utils, traceback, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/local_gtest_test

DEPS: chromium, chromium_android, chromium_tests, test_results, test_utils, depot_tools/bot_update, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/local_isolated_script_test

DEPS: chromium_tests, isolate, test_utils, recipe_engine/json, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/mini_installer_test

DEPS: chromium, chromium_tests, test_utils, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/mock_test

DEPS: chromium_tests, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/script_test

DEPS: chromium, chromium_tests, test_utils, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/sizes_step

DEPS: chromium, chromium_tests, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/swarming_gtest_test

DEPS: chromium, chromium_swarming, chromium_tests, clang_coverage, isolate, test_results, test_utils, depot_tools/bot_update, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/swarming_ios_test

DEPS: chromium, chromium_swarming, chromium_tests, ios, perf_dashboard, test_results, test_utils, depot_tools/bot_update, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time

— def RunSteps(api):

recipes / chromium_tests:tests/steps/swarming_isolated_script_test

DEPS: build, chromium, chromium_checkout, chromium_swarming, chromium_tests, clang_coverage, isolate, perf_dashboard, puppet_service_account, test_results, test_utils, depot_tools/bot_update, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/web_view_cts_test

DEPS: chromium, chromium_android, chromium_tests, test_results, test_utils, depot_tools/bot_update, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/steps/webrtc_perf_test

DEPS: chromium, chromium_checkout, chromium_tests, test_results, test_utils, zip, depot_tools/bot_update, depot_tools/gclient, depot_tools/gsutil, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / chromium_tests:tests/trybots

DEPS: chromium_tests, recipe_engine/properties, recipe_engine/step

Test to ensure the validity of the entries within TRYBOTS.

Each entry in the TRYBOTS dict will be checked to ensure chromium_tests.trybot_steps can be called with the mastername and buildername properties set for the entry.

— def RunSteps(api):

recipes / chromium_trybot

DEPS: build, chromium, chromium_android, chromium_checkout, chromium_swarming, chromium_tests, filter, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/gerrit, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / chromium_upload_clang

DEPS: chromium, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gsutil, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / clang_coverage:tests/full

DEPS: chromium_tests, clang_coverage, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / client.nacl.sdk.recipe_autogen

DEPS: build, chromium, depot_tools/bot_update, depot_tools/gclient, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

— def sdk_multi_steps(api):

— def sdk_multirel_steps(api):

recipes / closure_compilation

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / codesearch:examples/full

DEPS: chromium, codesearch, goma, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / codesearch:tests/checkout_generated_files_repo_and_sync

DEPS: chromium, codesearch, depot_tools/bot_update, depot_tools/gclient, recipe_engine/properties

— def RunSteps(api):

recipes / codesearch:tests/configs

DEPS: chromium, codesearch, recipe_engine/properties

— def RunSteps(api):

recipes / codesearch:tests/create_and_upload_kythe_index_pack

DEPS: chromium, codesearch, depot_tools/bot_update, depot_tools/gclient, recipe_engine/properties

— def RunSteps(api):

recipes / codesearch:tests/generate_compilation_database

DEPS: chromium, codesearch, recipe_engine/properties

— def RunSteps(api):

recipes / codesearch:tests/run_clang_tool

DEPS: codesearch

— def RunSteps(api):

recipes / crashpad/build

DEPS: depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/osx_sdk, depot_tools/windows_sdk, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

Buildbot recipe definition for the various Crashpad continuous builders.

— def RunSteps(api, buildername, config, target_os, target_cpu):

Generates the sequence of steps that will be run by the slave.

recipes / cronet

DEPS: cronet, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/properties

— def RunSteps(api):

recipes / cronet:examples/full

DEPS: chromium, cronet, recipe_engine/buildbucket, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / cronet:tests/run_tests

DEPS: chromium, chromium_android, cronet, recipe_engine/properties

— def RunSteps(api):

recipes / cros/cbuildbot

DEPS: chromite, depot_tools/gitiles, recipe_engine/properties

— def RunSteps(api):

recipes / cros/cbuildbot_tryjob

DEPS: chromite, depot_tools/gitiles, recipe_engine/json, recipe_engine/properties

— def RunSteps(api):

recipes / cros/swarming

DEPS: chromite, depot_tools/gitiles, recipe_engine/properties

— def RunSteps(api):

recipes / crrev:examples/full

DEPS: crrev, recipe_engine/json, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / custom_tabs_client

DEPS: depot_tools/bot_update, depot_tools/gclient, recipe_engine/properties

— def RunSteps(api):

recipes / dart/chocolatey

DEPS: dart, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step, recipe_engine/url

— def RunSteps(api):

recipes / dart/docker

DEPS: dart, zip, depot_tools/git, depot_tools/gsutil, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / dart/external

DEPS: recipe_engine/buildbucket, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / dart/flutter_engine

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gitiles, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

— def AnalyzeDartUI(api, checkout_dir):

— def Build(api, checkout_dir, config, *targets):

— def BuildAndTest(api, start_dir, checkout_dir, flutter_rev):

— def BuildLinux(api, checkout_dir):

— def BuildLinuxAndroidArm(api, checkout_dir):

— def BuildLinuxAndroidx86(api, checkout_dir):

— def CopyArtifacts(api, engine_src, cached_dest, file_paths):

— def GetCheckout(api):

— def KillTasks(api, checkout_dir, ok_ret=‘any’):

Kills leftover tasks from previous runs or steps.

— def RunGN(api, checkout_dir, *args):

— def RunSteps(api):

— def TestEngine(api, checkout_dir):

— def TestFlutter(api, start_dir, just_built_dart_sdk):

— def TestObservatory(api, checkout_dir):

— def UpdateCachedEngineArtifacts(api, flutter, engine_src):

recipes / dart/forward_branch

DEPS: dart, depot_tools/git, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / dart/gclient

DEPS: depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/properties

— def RunSteps(api):

recipes / dart/linearize

DEPS: dart, depot_tools/git, depot_tools/gitiles, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties

— def RunSteps(api):

recipes / dart/neo

DEPS: dart, swarming_client, test_utils, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / dart/package_co19

DEPS: depot_tools/git, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / dart:examples/example

DEPS: dart, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / dart:examples/example-get_secret

DEPS: dart, recipe_engine/platform

— def RunSteps(api):

recipes / dawn

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gsutil, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step, recipe_engine/time

— def RunSteps(api, target_cpu, debug, clang):

recipes / disk:examples/full

DEPS: disk, recipe_engine/json, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / docker:examples/full

DEPS: docker, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / docker_run

DEPS: docker, recipe_engine/properties

A generic recipe that runs a given docker container and exits.

— def RunSteps(api, server, project, image, cmd_args, env):

recipes / emscripten_releases

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / emulator

DEPS: chromium, chromium_android, emulator, depot_tools/bot_update, depot_tools/gclient, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, mastername, buildername):

recipes / emulator:examples/full

DEPS: chromium, chromium_android, emulator, depot_tools/bot_update, depot_tools/gclient, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, mastername, buildername):

recipes / example/defer_no_fail

DEPS: recipe_engine/step

— def RunSteps(api):

recipes / example/hierarchical

DEPS: recipe_engine/context, recipe_engine/step

— def RunSteps(api):

recipes / filter:examples/full

DEPS: chromium, filter, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / filter:tests/analyze

DEPS: chromium, filter, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / filter:tests/analyze_mac

DEPS: chromium, filter, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / filter:tests/suppress_analyze

DEPS: chromium, filter, recipe_engine/properties

— def RunSteps(api):

recipes / findit/chromium/compile

DEPS: chromium, chromium_checkout, chromium_tests, filter, findit, depot_tools/gclient, depot_tools/git, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api, target_mastername, target_buildername, good_revision, bad_revision, compile_targets, use_analyze, suspected_revisions, use_bisect, compile_on_good_revision):

recipes / findit/chromium/compile_isolate

DEPS: chromium, chromium_android, chromium_checkout, chromium_swarming, chromium_tests, findit, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, recipe_engine/commit_position, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

This recipe is to compile and isolate the given isolated targets.

If multiple test targets match the same isolated targets, we default to the first one after ordering test target names alphabetically.

— def RunSteps(api, target_mastername, target_testername, revision, isolated_targets):

recipes / findit/chromium/export_bot_db

DEPS: chromium_tests, depot_tools/gsutil, recipe_engine/json, recipe_engine/properties

Export the bot db to cloud storage.

— def RunSteps(api):

— def thaw_and_remove_unserializable(v):

recipes / findit/chromium/flake

DEPS: adb, chromium, chromium_android, chromium_checkout, chromium_swarming, chromium_tests, filter, findit, isolate, test_results, test_utils, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api, target_mastername, target_testername, test_revision, tests, test_repeat_count, skip_tests):

recipes / findit/chromium/preemptive_bot_update

DEPS: chromium_checkout, chromium_tests, depot_tools/gclient, depot_tools/git, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time

Checkout the chromium src tree at the latest revision.

This recipe is meant to be run on idle bots to keep their local checkouts up to date as much as possible so that when they are required to run culprit-finding jobs they have as low latency as possible in their bot_update steps.

— def RunSteps(api):

— def TargetMasterAndBuilder(api):

recipes / findit/chromium/test

— def RunSteps(api, target_mastername, target_testername, good_revision, bad_revision, tests, use_analyze, suspected_revisions, test_on_good_revision, test_repeat_count):

recipes / findit/chromium/update_components

DEPS: chromium_tests, depot_tools/git, depot_tools/gsutil, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

Traverse the source tree and update a cloud file with component mappings.

In each OWNERS file, a component and a team may be defined. This recipe calls a script that gathers all the information onto a json file and refreshes a world-readable cloud location.

— def RunSteps(api):

— def RunStepsForFile(api, filename, extra_arguments, step_suffix):

recipes / findit:examples/full

DEPS: findit, recipe_engine/path, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api):

recipes / flutter/engine

DEPS: build, goma, zip, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gsutil, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

— def AddFiles(api, pkg, relative_paths):

— def AnalyzeDartUI(api):

— def Build(api, config, *targets):

— def BuildIOS(api):

— def BuildJavadoc(api):

— def BuildLinux(api):

— def BuildLinuxAndroid(api):

— def BuildMac(api):

— def BuildObjcDoc(api):

Builds documentation for the Objective-C variant of engine.

— def BuildWindows(api):

— def GetCheckout(api):

— def GetCloudPath(api, path):

— def GetMacSDKDir(api):

@contextmanager
— def InstallJazzy(api):

@contextlib.contextmanager
— def MakeTempDir(api, label):

— def PackageIOSVariant(api, label, arm64_out, armv7_out, sim_out, bucket_name):

— def RunGN(api, *args):

— def RunHostTests(api, out_dir, exe_extension=''):

— def RunSteps(api):

@contextmanager
— def SetupXcode(api):

— def TestObservatory(api):

— def UploadArtifacts(api, platform, file_paths, archive_name=‘artifacts.zip’):

— def UploadDartPackage(api, package_name):

— def UploadDartSdk(api, archive_name):

— def UploadFlutterPatchedSdk(api):

— def UploadFolder(api, dir_label, parent_dir, folder_name, zip_name, platform=None):

— def UploadTreeMap(api, upload_dir, lib_flutter_path, android_triple):

— def UploadWebSdk(api, archive_name):

— def VerifyExportedSymbols(api):

recipes / flutter/flutter

DEPS: build, zip, depot_tools/depot_tools, depot_tools/git, depot_tools/gsutil, depot_tools/osx_sdk, depot_tools/windows_sdk, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/url

— def BuildExamples(api, git_hash, flutter_executable):

— def CreateAndUploadFlutterPackage(api, git_hash, branch):

Prepares, builds, and uploads an all-inclusive archive package.

— def DecryptKMS(api, step_name, crypto_key_path, ciphertext_file, plaintext_file):

— def EnsureCloudKMS(api, version=None):

— def GetCloudPath(api, git_hash, path):

— def GetGradleDirName(api):

— def GetGradleDistributionUrl(api):

— def GetGradleZipFileName(api):

@contextmanager
— def Install7za(api):

@contextmanager
— def InstallGem(api, gem_name, gem_version):

— def InstallGradle(api, checkout):

— def InstallOpenJDK(api):

— def RunSteps(api):

— def UploadFlutterCoverage(api):

Uploads the Flutter coverage output to cloud storage and Coveralls.

recipes / gae_sdk:examples/full

DEPS: gae_sdk, depot_tools/cipd, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / gatekeeper

DEPS: gatekeeper, depot_tools/infra_paths, recipe_engine/path, recipe_engine/properties

Launches the gatekeeper.

— def RunSteps(api):

recipes / gatekeeper-failure

DEPS: recipe_engine/platform, recipe_engine/step

Repeatedly fails as a way to ensure the gatekeeper is alive and well.

— def RunSteps(api):

recipes / gatekeeper:tests/call

DEPS: gatekeeper, depot_tools/infra_paths, recipe_engine/properties

— def RunSteps(api):

recipes / git_clone_bundler:examples/full

DEPS: git_clone_bundler, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api):

recipes / gn:tests/clean

DEPS: gn, recipe_engine/path

— def RunSteps(api):

recipes / gn:tests/get_args

DEPS: gn, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api):

recipes / gn:tests/ls

DEPS: gn, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api, targets, output_type):

recipes / gn:tests/refs

DEPS: gn, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api, targets, output_type):

recipes / goma:examples/full

DEPS: goma, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / goma:tests/build_with_goma

DEPS: goma, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / goma:tests/ensure

DEPS: goma, recipe_engine/properties

— def RunSteps(api):

recipes / goma:tests/jobs

DEPS: goma, recipe_engine/platform

— def RunSteps(api):

recipes / goma:tests/recommended_goma_jobs

DEPS: goma

— def RunSteps(api):

recipes / goma:tests/start

DEPS: goma, recipe_engine/buildbucket, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api):

recipes / goma_client

DEPS: depot_tools/bot_update, depot_tools/cipd, depot_tools/depot_tools, depot_tools/gclient, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / goma_server

DEPS: depot_tools/cipd, depot_tools/git, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

— def SetupExecutables(api, pkg_dir):

Set up go and protoc to run the script.

Args: pkg_dir: a root directory to install cipd packages.

Returns: a list of paths.

recipes / halt:examples/full

DEPS: halt

— def RunSteps(api):

recipes / ios/try

DEPS: chromium_swarming, ios, depot_tools/gclient, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / ios/unified_builder_tester

DEPS: ios, recipe_engine/buildbucket, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / ios:examples/full

DEPS: chromium_swarming, ios, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime

— def RunSteps(api):

recipes / ios:tests/isolate

DEPS: ios, recipe_engine/assertions, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime

— def RunSteps(api, expected_partial_tasks):

recipes / ios:tests/optional_dimensions

DEPS: ios, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api):

recipes / ios:tests/swarming_service_acount

DEPS: ios, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io

— def RunSteps(api):

recipes / isolate:examples/compose

DEPS: isolate, swarming_client

— def RunSteps(api):

recipes / isolate:examples/full

DEPS: isolate, swarming_client, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / isolate:tests/check_swarm_hashes

DEPS: isolate, recipe_engine/properties

— def RunSteps(api):

recipes / isolate:tests/clean_isolated_files

DEPS: isolate, recipe_engine/path

— def RunSteps(api):

recipes / isolate:tests/compare_build_artifacts

DEPS: chromium, isolate, recipe_engine/properties

— def RunSteps(api):

recipes / isolate:tests/isolate_tests

DEPS: isolate, recipe_engine/path

— def RunSteps(api):

recipes / isolate:tests/isolated_tests

DEPS: isolate, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / isolate:tests/run_isolated

DEPS: isolate

— def RunSteps(api):

recipes / led_recipes_tester

DEPS: depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/led, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

Tests a recipe CL by running a chromium builder.

— def RunSteps(api, repo_name):

— def trigger_cl(api, recipe, repo_path, recipes_py_path, builder):

Calculates the chromium testing CL for the current CL.

Returns None if we shouldn't trigger anything. Returns a key in the CHROMIUM_SRC_TEST_CLS dictionary.

recipes / legion/legion

DEPS: chromium_swarming, isolate, legion, depot_tools/bot_update, depot_tools/gclient, recipe_engine/path, recipe_engine/properties

Launches Legion tests.

— def RunSteps(api):

recipes / legion:examples/full

DEPS: legion, depot_tools/bot_update, depot_tools/gclient, recipe_engine/properties

Specifies how to launch chromoting integration test on build_internal.

— def RunSteps(api):

recipes / libyuv/libyuv

DEPS: chromium, chromium_android, libyuv, depot_tools/bot_update, depot_tools/gclient, depot_tools/tryserver, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

Recipe for building and running tests for Libyuv stand-alone.

— def RunSteps(api):

recipes / libyuv/roll_deps

DEPS: libyuv, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gerrit, depot_tools/git, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / math_utils:examples/full

DEPS: math_utils, recipe_engine/step

— def RunSteps(api):

recipes / math_utils:tests/mean_error

DEPS: math_utils

— def RunSteps(api):

recipes / nacl

DEPS: goma, trigger, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/osx_sdk, depot_tools/windows_sdk, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / nacl_ports

DEPS: chromium, depot_tools/bot_update, depot_tools/gclient, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api):

recipes / ndk/ndk_buildbot

DEPS: chromite, ndk, recipe_engine/properties

— def RunSteps(api):

recipes / ndk:tests/cover

DEPS: chromite, ndk

— def RunSteps(api):

recipes / openscreen

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/git, depot_tools/osx_sdk, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

Recipe for building and running tests for Open Screen stand-alone.

— def RunSteps(api):

recipes / pdfium

DEPS: goma, depot_tools/bot_update, depot_tools/depot_tools, depot_tools/gclient, depot_tools/gsutil, depot_tools/osx_sdk, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step, recipe_engine/time

— def RunSteps(api, memory_tool, skia, skia_paths, xfa, v8, target_cpu, clang, msvc, rel, jumbo, skip_test, target_os):

— def dict_to_str(props):

Returns the given dictionary as a string of space separated key/value pairs sorted by keys.

— def gen_ci_build(api, builder):

— def get_gold_ignore_hashes(api, out_dir):

Downloads a list of MD5 hashes from Gold and writes them to a file. That file is then used by the test runner in the pdfium repository to ignore already known hashes.

— def get_gold_params(api, build_config, revision):

Get the parameters to be passed to the testing call to generate the dm.json file expected by Gold and to upload the generated images. Returns: (properties_str, key_str) These strings can be passed directly into run_corpus_tests.py.

— def gold_build_config(args):

Extracts key value pairs from the arguments handed to ‘gn gen’ and returns them as a dictionary. Since these are used as parameters in Gold we strip common prefixes and disregard some arguments. i.e. ‘use_goma’ since we don't care about how a binary was built. Only arguments that follow the ‘key=value’ pattern are considered.

— def gs_cp(api, name, src, dst, multithreaded=False, extra_args=None):

Copy the src to dst in Google storage.

— def upload_dm_results(api, results_dir, revision, test_type):

Uploads results of the tests to Gold. This assumes that results_dir contains a JSON file and a set of PNGs. Adapted from: https://skia.googlesource.com/skia/+/master/infra/bots/recipes/upload_dm_results.py test_type is expected to be ‘corpus’, ‘javascript’, or ‘pixel’

recipes / perf_dashboard:examples/full

DEPS: perf_dashboard, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / pinpoint/builder

DEPS: chromium, chromium_tests, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/properties

— def RunSteps(api):

recipes / puppet_service_account:examples/full

DEPS: puppet_service_account, recipe_engine/platform

Small example of using the puppet_service_account api.

— def RunSteps(api):

recipes / remote_run_test

DEPS: recipe_engine/properties, recipe_engine/step

Checks that properties get to recipes from remote_run properly

— def RunSteps(api, true_prop, num_prop, string_prop, dict_prop):

recipes / repo:examples/full

DEPS: repo, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / run_presubmit

DEPS: v8, webrtc, depot_tools/bot_update, depot_tools/gclient, depot_tools/git, depot_tools/presubmit, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/cq, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/runtime, recipe_engine/step

— def RunSteps(api):

recipes / swarming/deterministic_build

DEPS: chromium, chromium_android, isolate, depot_tools/bot_update, depot_tools/gclient, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

Recipe to test the deterministic build.

Waterfall page: https://build.chromium.org/p/chromium.swarm/waterfall

— def ConfigureChromiumBuilder(api, recipe_config):

— def MoveBuildDirectory(api, src_dir, dst_dir):

— def RunSteps(api, buildername):

recipes / swarming/heartbeat

DEPS: chromium_swarming, swarming_client, swarming_heartbeat, recipe_engine/properties

Swarming heart beat recipe: runs a dummy job on the prod Swarming instance to ensure it is working properly.

Waterfall page: https://build.chromium.org/p/chromium.swarm/waterfall

— def RunSteps(api):

recipes / swarming/staging

DEPS: chromium, chromium_checkout, chromium_swarming, chromium_tests, isolate, swarming_client, test_results, test_utils, depot_tools/gclient, recipe_engine/commit_position, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

Swarming staging recipe: runs tests for HEAD of chromium using HEAD of swarming_client toolset on Swarming staging server instances (*-dev.appspot.com).

Intended to catch bugs in swarming_client and Swarming servers early on, before full roll out.

Waterfall page: https://build.chromium.org/p/chromium.swarm/waterfall

— def RunSteps(api, buildername, mastername):

recipes / swarming_client:examples/full

DEPS: swarming_client, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / swarming_heartbeat:examples/full

DEPS: swarming_heartbeat, recipe_engine/properties

— def RunSteps(api):

recipes / tar:examples/full

DEPS: tar, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/step

— def RunSteps(api):

recipes / test_results:examples/full

DEPS: test_results, recipe_engine/json, recipe_engine/properties, recipe_engine/runtime

— def RunSteps(api, warning, server_config):

recipes / test_utils:tests/gtest_results

DEPS: test_utils, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / test_utils:tests/present_gtest_failures

DEPS: test_utils, recipe_engine/step

— def RunSteps(api):

recipes / test_utils:tests/run_tests

DEPS: chromium, chromium_swarming, chromium_tests, test_results, test_utils, depot_tools/tryserver, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api, test_swarming, test_name, abort_on_failure):

recipes / test_utils:tests/run_tests_include_swarming

DEPS: test_utils

— def RunSteps(api):

recipes / test_utils:tests/run_tests_with_patch

DEPS: chromium_swarming, chromium_tests, test_utils, recipe_engine/properties, recipe_engine/python, recipe_engine/step

— def RunSteps(api, has_valid_results, retry_failed_shards, per_suffix_failures, per_suffix_valid):

recipes / test_utils:tests/test_results

DEPS: test_utils, recipe_engine/properties

— def RunSteps(api):

recipes / traceback:examples/full

DEPS: traceback, recipe_engine/path, recipe_engine/platform, recipe_engine/step

— def RunSteps(api):

recipes / trigger:examples/full

DEPS: trigger, recipe_engine/properties

— def RunSteps(api):

recipes / v8

DEPS: archive, chromium, chromium_swarming, swarming_client, test_utils, v8, depot_tools/gclient, depot_tools/infra_paths, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/swarming, recipe_engine/time, recipe_engine/url

— def RunSteps(api, binary_size_tracking, build_config, clobber, clobber_all, clusterfuzz_archive, coverage, custom_deps, default_targets, enable_swarming, mb_config_path, set_gclient_var, target_arch, target_platform, track_build_dependencies, triggers, triggers_proxy, use_goma):

recipes / v8/archive

DEPS: chromium, gn, v8, zip, depot_tools/gclient, depot_tools/git, depot_tools/gsutil, recipe_engine/buildbucket, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

Recipe for archiving officially tagged v8 builds.

— def RunSteps(api, build_config, target_arch, target_bits, target_platform):

— def make_archive(api, bot_config, ref, version, archive_type, step_suffix=‘‘, archive_suffix=’’):

recipes / v8/auto_roll_deps

DEPS: chromium, v8, depot_tools/bot_update, depot_tools/gclient, depot_tools/gerrit, depot_tools/git, depot_tools/gitiles, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/service_account, recipe_engine/step, recipe_engine/url

— def RunSteps(api):

— def V8RevisionFrom(deps):

recipes / v8/auto_roll_push

DEPS: chromium, v8, depot_tools/bot_update, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/service_account, recipe_engine/step, recipe_engine/url

— def RunSteps(api):

recipes / v8/auto_roll_release_process

DEPS: v8, depot_tools/bot_update, depot_tools/gclient, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step

— def ClusterfuzzHasIssues(api):

— def RunSteps(api):

recipes / v8/auto_roll_v8_deps

DEPS: v8, depot_tools/bot_update, depot_tools/gclient, depot_tools/gerrit, depot_tools/git, depot_tools/gitiles, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/service_account, recipe_engine/step, recipe_engine/url

— def GetDEPS(api, name, repo):

— def RunSteps(api):

— def commit_messages_log_entries(api, repo, from_commit, to_commit):

Returns list of log entries to be added to commit message.

Args: api: Recipes api. repo: Gitiles url to rolled repository. from_commit: Parent of first rolled commit. to_commit: Newest rolled commit.

recipes / v8/auto_tag

DEPS: v8, depot_tools/gclient, depot_tools/git, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/service_account, recipe_engine/step

This recipe checks if a version update on branch is necessary, where ‘version’ refers to the contents of the v8 version file (part of the v8 sources).

The recipe will:

Commit a v8 version change to with an incremented patch level if the latest two commits point to the same version.
Make sure that the actual HEAD of is tagged with its v8 version (as specified in the v8 version file at HEAD).
Update a ref called -lkgr to point to the latest commit that has a unique, incremented version and that is tagged with that version.

— def GetCommitForRef(api, repo, ref):

— def Git(api, *args, **kwargs):

Convenience wrapper.

— def IncrementVersion(api, ref, latest_version, latest_version_file):

Increment the version on branch ‘ref’ to the next patch level and wait for the committed ref to be gnumbd-ed or time out.

Args: api: The recipe api. ref: Ref name where to change the version, e.g. refs/remotes/branch-heads/1.2. latest_version: The currently latest version to be incremented. latest_version_file: The content of the current version file.

— def InitClean(api):

Ensures a clean state of the git checkout.

— def LogStep(api, text):

— def PushRef(api, repo, ref, hsh):

— def RunSteps(api):

— def UpdateRef(api, repo, head, lkgr_ref):

recipes / v8/flako

DEPS: chromium_swarming, swarming_client, depot_tools/gitiles, depot_tools/gsutil, recipe_engine/json, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/runtime, recipe_engine/step, recipe_engine/tempfile

Recipe to bisect flaky tests in V8.

Bisection will start at a known bad to_revision and:

Calibrate the number of repetitions until enough confidence is reached.
Bisect backwards exponentially, doubling the offset in each step.
After finding a good from_revision, bisect into the range from_revision..to_revision and report the suspect.

Tests are only run on existing isolated files, looked up on Google Storage.

All revisions during bisections are represented as offsets to the start revision which has offset 0.

See PROPERTIES for documentation on the recipe's interface.

— def RunSteps(api, bisect_mastername, bisect_buildername, build_config, extra_args, initial_commit_offset, max_calibration_attempts, isolated_name, num_shards, repetitions, repro_only, swarming_dimensions, swarming_priority, swarming_expiration, test_name, timeout_sec, total_timeout_sec, to_revision, variant):

— def bisect(api, depot, initial_commit_offset, is_bad_func, offset):

Exercises the bisection control flow.

Args: api: Recipe api. depot: Helper for accessing storage and git. initial_commit_offset: Number of commits, backwards bisection will initially leap over. is_bad_func: Function (revision->bool) determining if a given revision is bad. offset: Offset at which to start bisection.

— def setup_swarming(api, swarming_dimensions, swarming_priority, swarming_expiration):

recipes / v8/node_integration

DEPS: chromium, goma, trigger, v8, zip, depot_tools/gclient, depot_tools/gsutil, depot_tools/tryserver, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime, recipe_engine/step

Recipe to test v8/node.js integration.

— def RunSteps(api):

@contextmanager
— def goma_wrapper(api):

recipes / v8/node_integration_ng

DEPS: chromium, goma, v8, zip, depot_tools/bot_update, depot_tools/gclient, depot_tools/gsutil, depot_tools/osx_sdk, recipe_engine/buildbucket, recipe_engine/commit_position, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step

Recipe to test v8/node.js integration.

— def RunSteps(api, triggers, v8_tot):

— def run_with_retry(api, step_name, step_fun):

Runs step_fun and retries once on failure.

Returns: True if a flake has been detected.

recipes / v8/presubmit

DEPS: chromium, v8, depot_tools/gclient, depot_tools/infra_paths, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/runtime

Recipe for running presubmit in V8 CI.

— def RunSteps(api):

recipes / v8/verify_flakes

DEPS: v8, depot_tools/gitiles, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/runtime, recipe_engine/step

Recipe to verify flaky tests (aka progression testing).

The recipe reads flake config and for each entry triggers flako recipe in reproduce_only mode. If any flake fails to reproduce, the build is marked as failed, which can then be used to alert sheriffs via a gatekeeper rule.

— def RunSteps(api):

— def update_step_presentation(api, presentation, build, flake_config):