Scripts and files in this directory describe how to build CIPD packages from the source code in infra.git repo.
There are two flavors of packages:
A package is defined in a *.yaml file with the following structure:
# Name of the package in CIPD repository. package: infra/example/package # Human readable description of the package. description: Example package # Optional list of Buildbot CI builders to build this package on. If not # specified the package will be build on all CI builders. When build.py script # is invoked manually (without --builder flag), this property is ignored. builders: - infra-continuous-precise-64 - ... # Path to the root of the package source files on the system we're building # the package from. Can be absolute or relative to the path of the *.yaml # file itself. root: ../.. data: # 'dir' section adds a subdirectory of 'root' to the package. In this case # it will scan directory <yaml_path>/../../a/b/c and put files into a/b/c # directory of the package. - dir: a/b/c # A list of regular expressions for files to exclude from the package. # Syntax is defined at http://golang.org/pkg/regexp/syntax/. Each expression # is implicitly wrapped into ^...$. The tests are applied to paths relative # to 'dir', e.g. 'bin/active' regexp matches only single file # <yaml_path>/../../a/b/c/bin/active. exclude: - bin/activate - .*\.pyc # 'file' section adds a single file to the package. - file: run.py
Any string in package definition can reference a variable via ${var_name}, for example:
package: infra/tools/cipd/${platform}
Available variables are defined in build.py in get_package_vars
:
${exe_suffix}
is ‘.exe’ on Windows and empty string on other platforms.${platform}
defines where build.py is running, as ‘(flavor)-(bitness)’ string. It is suitable for packages that do not depend much on the exact version of the OS, for example packages with statically linked binaries. Example values:${os_ver}
defines major and minor version of the OS/Linux distribution. It is useful if package depends on .dll/.so libraries provided by the OS. Example values:${python_version}
defines python version as ‘(major)(minor)’ string, e.g ‘27’.See packages for examples of package definitions.
build.py script does the following:
--upload
option is passed) uploads CIPD packages to the repository.Package definition files can assume that Go infra code is built and all artifacts are installed in GOBIN
(which is go/bin).
You can also pass one or more *.yaml file names to build only specific packages:
build.py infra_python cipd_client
To install a built package locally use cipd client binary (it is built by build.py as well). For example, to rebuild and install infra_python.cipd into ./install_dir, run:
cd infra.git/ rm -rf install_dir ./build/build.py infra_python ./go/bin/cipd pkg-deploy -root=install_dir build/out/infra_python.cipd cd install_dir
test_package.py script can be used to run simple package integrity tests to verify a built package looks good after deploy.
For each .yaml in packages/ there can be corresponding .py file in tests/ that is invoked by test_package.py to check that deployed package looks good.
Basically test_package.py does the following:
--work-dir
is used).python test/<name>.py
with cwd == installation directory.Thus to test that infra_python.cipd package works, one can do the following:
./build/build.py infra_python ./build/test_packages.py infra_python
test_packages.py is used on CI builders to verify packages look good before uploading them.