Clone this repo:
  1. 80d9f9f Automatic config update by chromeos-ci-prod · 14 hours ago main
  2. a32ea06 Automatic config update by chromeos-ci-prod · 14 hours ago
  3. 315c0aa Automatic config update by chromeos-ci-prod · 16 hours ago
  4. 91037e4 Automatic config update by chromeos-ci-prod · 22 hours ago
  5. e66b567 Automatic config update by chromeos-ci-prod · 4 days ago

Suite-Scheduler README

Developer Setup

SDK

suite-scheduler is an AppEngine Standard Python Environment V1 application. You need the Google Cloud SDK's AppEngine Python component to develop and deploy this application:

  • Install Google Cloud SDK:
    • Follow the instructions to install the Python App Engine component.
  • Log in to gcloud: gcloud auth login <username>@google.com
  • Install App Engine Python extensions
    • gcloud components install app-engine-python
    • gcloud components install app-engine-python-extras

Developer environment

Suite-scheduler development must be done in the standard Chrome OS source checkout but entirely outside the Chrome OS chroot environment.

suite-scheduler uses infra_virtualenv to provide a stable environment for development and release.

First, make sure you install virtualenv with version at least 20.0.

Then, to (re)initialize developer environment, run

  bin/setup_environment

For testing changes beyond to the configs/ directory, and for deploying suite-scheduler, you must also obtain certain service credentials used by suite-scheduler.

  bin/setup_environment --load-creds

If you get failures when trying to download credentials,

  • Double check you're logged into the Google Cloud SDK by running gcloud auth list.
  • Contact Test Platform team or one of the OWNERS to be allowlisted.

Testing your changes

Changes to configs/* directory can be tested via sanity tests alone:

  bin/run_sanity_tests
  bin/run_sanity_tests --debug  # More verbose

Other changes must be validated with the full test suite:

  bin/run_tests
  bin/run_tests --debug  # More verbose

These tests include some integration tests that can take over 5 minutes to run.

** WARNING: suite-scheduler unittests do not currently run in presubmit. You MUST ensure that unit-tests pass locally for your change. **

Making configuration changes

  • Add or update your job configuration in configs/suite_scheduler.ini.
  • Update configs/lab_config.ini, if the suite test relies on new board or model. Note, suite scheduler does not automatically catch the new boards or models added to the lab.
  • Test your changes.
  • Submit the change and ping/file bug to infra team. Test platform deputy will rollout your change to the production. Once this is done, you may find the scheduling status of the new suite via our dashboard.

Running an instance locally

If you want to verify any APIs locally, you can run suite scheduler v2 as a local development server (https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server):

a. Finally, start dev_appserver.py in a window:

    dev_appserver.py app.yaml \
    --port=8888 \
    --admin_port=8001 \
    --log_level=debug

b. Test it:

    curl 'http://localhost:8888/cron/trigger_event', or open it in chrome.

Please note that once a dev_appserver is started, it simulates data_store & task_queue, which means:

a. The datastore doesn‘t contain any previous cron run’s information, so the first round of cron/trigger_event won't trigger any event.

b. The task queue is newly created with 0 tasks in it unless cron/trigger_event adds tasks in it. Dev_appserver restart will cause the local task queue to get purged.

Releasing to production

There are two instances of suite-scheduler:

All releases must follow the following process:

  • Release to staging instance
      bin/deploy
    
  • Go the to cron page for the staging instance and manually trigger cron/test_push. Verify there are no obvious failures by looking at service logs.
  • Note the currently deployed prod version, in case you need to rollback.
  • Release to prod instance
      bin/deploy -p
    
  • Send out an email to g/chromeos-infra-releases. Include in the email:
    • The version of the newly deployed app-engine service.
    • The version of the old app-engine service (required for rollback)
    • The changelist, generated with
      git log --pretty=format:'%h %s (%cr) <%an>'
      

Run bin/deploy -h for more usage information.

If there are issues discovered in prod after release, rollback to the previous version.

  gcloud app versions migrate --project=google.com:suite-scheduler ${VERSION}