Google Git
Sign in
chromium / chromiumos / infra / suite_scheduler / factory-nocturne-10984.B
commit87d471d22d62bcb3eb0b9d1c53a7454b45ca8175[log] [tgz]
authorAadit Modi <modia@google.com>Thu Aug 16 18:10:44 2018
committerchrome-bot <chrome-bot@chromium.org>Fri Aug 17 18:18:00 2018
treedd8551e5ecf785cc6b4f892384162c4c2897efa9
parent2b62f760d9f226be90ff93d2bac5b93020bb686f [diff]
Informing scheduler to add new board as nocturne

board_list: nocturne
model_list: nocturne_nocturne

Bug:  None

Change-Id: I2e929b7db29834fccb1bd8f3ec4d7a292882cf95
Reviewed-on: https://chromium-review.googlesource.com/1178464
Commit-Ready: Aadit Maheshkumar Modi <modia@chromium.org>
Tested-by: Aadit Maheshkumar Modi <modia@chromium.org>
Reviewed-by: Mary Ruthven <mruthven@chromium.org>
1 file changed
tree: dd8551e5ecf785cc6b4f892384162c4c2897efa9
  1. bin/
  2. configs/
  3. www/
  4. .gitignore
  5. app.yaml
  6. appengine_config.py
  7. base_event.py
  8. base_event_unittest.py
  9. build_event.py
  10. build_event_unittest.py
  11. build_lib.py
  12. cloud_sql_client.py
  13. config_reader.py
  14. config_reader_unittest.py
  15. constants.py
  16. cron.py
  17. cron.yaml
  18. datastore_client.py
  19. datastore_client_unittest.py
  20. file_getter.py
  21. gae_import.py
  22. gae_test.py
  23. global_config.py
  24. local_import.py
  25. local_integration_test.py
  26. main.py
  27. PRESUBMIT.cfg
  28. queue.yaml
  29. README.md
  30. requirements.txt
  31. rest_client.py
  32. runner.py
  33. sanity_test.py
  34. setup_environment.py
  35. stackdriver_lib.py
  36. swarming_lib.py
  37. swarming_lib_unittest.py
  38. task.py
  39. task_executor.py
  40. task_executor_unittest.py
  41. task_unittest.py
  42. test.yaml
  43. time_converter.py
  44. time_converter_unittest.py
  45. timed_event.py
  46. timed_event_unittest.py
  47. tot_manager.py
  48. tot_manager_unittest.py
  49. trigger_receiver.py
  50. trigger_receiver_unittest.py
  51. utils.py
  52. utils_unittest.py
README.md

Setup

  1. Put your Google gCloud SDK in ~/google-cloud-sdk.

    SDK download page

  2. Install App Engine SDK in ~/google-cloud-sdk:

     ~/google-cloud-sdk/bin/gcloud components install app-engine-python

  3. Run python setup_environment.py to download third-party libraries and credentials.

Known Issues

  1. Failed to install third-packages in requirement.txt:

    Workround: Run pip install -t lib -r requirements.txt locally on your workstation to verify whether it could work.

  2. Failed to download credentials from ‘gs://suite-scheduler.google.com.a.appspot.com/credentials/’:

    Workround: Run gsutil ls gs://suite-scheduler.google.com.a.appspot.com/credentials/ locally on your workstation to verify whether you can fetch credentials with gsutil.

    If there's any auth issue, it may be because

    a. You‘re not using your corp account to login gcloud. Please run gcloud auth list to check what’s the used account for gcloud.

    b. Your corp account is not whitelisted by suite-scheduler project. Please contact @xixuan to whitelist your ldap.

How to test:

  1. Presubmit will kick off sanity test for verifying suite_scheduler.ini:

     python runner.py --test_type sanity

  2. Run unittest:

     python runner.py' or 'python runner.py --test_type unittest

  3. Run integration test:

     python runner.py --test_type integration

  4. Run tests with debug mode:

     python runner.py --debug

  5. Run suite scheduler v2 as a local development server:

    If you want to verify any APIs locally, you can run suite scheduler v2 as a local development server (https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server):

    Assuming you already got credentials/cloud_sql_credentials.txt by running ‘setup_environment.py’, set up some variables based on the database instance you want to connect to:

       # Set up CONNECTION_NAME based on the cloudsql project/instance
   # name.  For more details, see:
   # https://docs.google.com/document/d/1e_3mcsxdRKECYuYaIfpHcqf_TjnxxfHDKtidn900dEg/edit
   CONNECTION_NAME='cosmic-strategy-646:us-central1:cidb-gen2-replica2'

   # Set up USER based on the user you want to log in.  If you use
   # readonly_nopw, grab the password from (as of April, 2018):
   # https://valentine.corp.google.com/#/show/1493156766394660?tab=metadata
   USER='readonly_nopw' # [CIDB username]

    a. Test locally on your workstation to make sure you're whitelisted:

       /google/data/ro/projects/speckle/mysql \
       -S /cloudsql/${CONNECTION_NAME} \
       -u ${USER} \
       || echo "You may need to run prodaccess"

    b. Start the cloud_sql_proxy in a new window to let your service talk to cloud sql. Note that cloud_sql_proxy needs to be downloaded from Google's public cloud website: https://cloud.google.com/sql/docs/mysql/sql-proxy

       # Create the /cloudsql directory, since the suite scheduler assumes
   # that this is where to find it's socket.
   sudo mkdir /cloudsql
   sudo chmod 1777 /cloudsql

   # Start the cloud sql proxy.
   ./cloud_sql_proxy \
       -dir=/cloudsql \
       -instances=${CONNECTION_NAME} \
       -credential_file=./credentials/suite-scheduler_client_secret_service_account.json &

    c. Finally, start dev_appserver.py in a window:

       dev_appserver.py app.yaml \
       --port=8888 \
       --admin_port=8001 \
       --log_level=debug

    d. Test it:

       curl 'http://localhost:8888/cron/trigger_event', or open it in chrome.

    Please note that once a dev_appserver is started, it simulates data_store & task_queue, which means:

    a. The datastore doesn‘t contain any previous cron run’s information, so the first round of cron/trigger_event won't trigger any event.

    b. The task queue is newly created with 0 tasks in it unless cron/trigger_event adds tasks in it. Dev_appserver restart will cause the local task queue to get purged.