Add test project pool management scripts (for E2E tests)

generate_host_files.py:
  This script generates host files for different projects that share the same
  storage location. It will be used by our LUCI bot at the beginning of a test
  run to generate candidate host files to which to deploy.

Pool creation scripts:
- create_test_pool.py:
  It invokes the following 3 scripts to build a project pool.
- create_gcp_project.py:
  This creates a project and associates it to a billing account.
- create_gcp_storage.py:
  This script creates the shared storage on a given project (ex: for test assets).
- configure_test_project.py:
  This script configures a gcp_project for E2E tests (Enable APIs & set ACLs)
- configure_cron_stop_instances.py:
  This script schedules a job that stops instances older than 24h.
  We normally clean-up the deployment at the end of each test so we shouldn't
  need this, but this provides a safety net in case of unexpected failures.
  The deployment would get cleaned up automatically by cel_ctl next time we
  run a test on that project.

Usage examples:
  python generate_host_files.py --template examples/schema/host/example.host.textpb
       --projects="p1;p2;p3" --storage_bucket celab-ci-pool-assets
       --storage_prefix=tests-1234 --destination_dir /tmp/host_files
  python scripts/tests/create_test_pool.py --name celab-ci-pool
       --folder <XYZ> --billing <XYZ>
       --service_accounts="celab-ci-builder@...com"
       --size 5


Change-Id: I8ca5d499521092dddec94d3dd3c75eca57360003
Reviewed-on: https://chromium-review.googlesource.com/c/1387809
Reviewed-by: Fei Ling <feiling@chromium.org>
7 files changed
tree: d89e9a7f718dd6ecb909cd29f34de18bb4370585
  1. build/
  2. docs/
  3. examples/
  4. go/
  5. infra/
  6. resources/
  7. schema/
  8. scripts/
  9. test/
  10. .gitignore
  11. .gitmodules
  12. .style.yapf
  13. .vpython
  14. build.py
  15. CODE_OF_CONDUCT.md
  16. codereview.settings
  17. CONTRIBUTING.md
  18. Gopkg.lock
  19. Gopkg.toml
  20. LICENSE
  21. navbar.md
  22. OWNERS
  23. PRESUBMIT.py
  24. README.md
  25. run_tests.py
  26. test.py
  27. VERSION
README.md

Chrome Enterprise Lab

Chrome Enterprise Lab is an inaccurately named set of tools for building enterprise labs quickly and easily. The labs so built can be used for system level end-to-end testing of Google Chrome/Chromium.

Have a peek at the Design document.

Also have a peek at the Code of Conduct.

Most of the code is in Go. See Guide to code to get a head start on the code in this repository.

Building

Prerequisites

  • Go: Download Go from the Go download page. This project requires Go 1.9.4 or later. The build will fail with prior versions.

  • Depot Tools: Used for managing the checkout and the contributor workflow.

  • Protocol Buffers Compiler: Protocol buffers are used extensively for wranging all the data that needs to be shuttled around. Can be installed automatically via ./build.py deps --install.

  • Dep : Used for Go depedency management. This can be installed automatically by running ./build.py deps --install from the root of the source tree.

  • Go support for Protocol Buffers : This can be installed automatically by running ./build.py deps --install from the root of the source tree.

Get The Source

There are two ways to get the source. One is to use managed deps, and the other is to use plain go get. The latter workflow doesn't quite work yet due to this repository not being integrated with go.chromium.org. So this page only mentions the managed dependency workflow.

  1. Clone this repository:

    Assumes that $GOPATH is a single path and not a : delimited list.

    mkdir -p ${GOPATH}/src/chromium.googlesource.com/enterprise
    cd ${GOPATH}/src/chromium.googlesource.com/enterprise
    git clone https://chromium.googlesource.com/enterprise/cel
    cd cel
    
  2. Get the dependencies:

    python build.py deps --install
    

Build It

  1. Use the build script:

    python build.py build
    
  2. Also make sure the tests pass.

    python build.py test
    

Contributing

See CONTRIBUTING.