Merge "[suite_scheduler] Update lab_config.ini to include new models."
tree: fdfd5b349983ba416b31d9169682bb4ff6841885
  1. bin/
  2. configs/
  3. venv/
  4. www/
  5. .gitignore
  6. __init__.py
  7. app.yaml
  8. appengine_config.py
  9. base_event.py
  10. base_event_unittest.py
  11. build_event.py
  12. build_event_unittest.py
  13. build_lib.py
  14. build_utils.py
  15. check_migration_configs.py
  16. config_reader.py
  17. config_reader_unittest.py
  18. constants.py
  19. cron.py
  20. cron.yaml
  21. datastore_client.py
  22. datastore_client_unittest.py
  23. file_getter.py
  24. gae_import.py
  25. gae_test.py
  26. global_config.py
  27. local_import.py
  28. local_integration_test.py
  29. main.py
  30. PRESUBMIT.cfg
  31. queue.yaml
  32. README.md
  33. requirements.txt
  34. rest_client.py
  35. runner.py
  36. sanity_test.py
  37. setup_environment.py
  38. skylab.py
  39. stackdriver_lib.py
  40. swarming_lib.py
  41. swarming_lib_unittest.py
  42. task.py
  43. task_config_reader.py
  44. task_config_reader_unittest.py
  45. task_executor.py
  46. task_executor_unittest.py
  47. task_unittest.py
  48. test.yaml
  49. time_converter.py
  50. time_converter_unittest.py
  51. timed_event.py
  52. timed_event_unittest.py
  53. tot_manager.py
  54. tot_manager_unittest.py
  55. trigger_receiver.py
  56. trigger_receiver_unittest.py
  57. update_migration_configs.py
  58. upload_migration_config.py
  59. utils.py
  60. utils_unittest.py
README.md

Setup

Due to https://crbug.com/792690, this must be run outside the chroot.

  1. Install Google Cloud SDK:

    1. Follow the instructions.

    2. Install the Python App Engine component:

       sudo apt-get install google-cloud-sdk-app-engine-python
      
  2. Log in to gcloud: gcloud auth login LDAP@google.com

  3. Install pip:

     sudo apt-get install python-pip
    
  4. Run python setup_environment.py to download third-party libraries and credentials.

Known Issues

  1. Failed to install third-packages in requirement.txt:

    Workround: Run pip install -t lib -r requirements.txt locally on your workstation to verify whether it could work.

  2. Failed to download credentials from ‘gs://suite-scheduler.google.com.a.appspot.com/credentials/’:

    Workround: Run gsutil ls gs://suite-scheduler.google.com.a.appspot.com/credentials/ locally on your workstation to verify whether you can fetch credentials with gsutil.

    If there's any auth issue, it may be because

    a. You‘re not using your corp account to login gcloud. Please run gcloud auth list to check what’s the used account for gcloud.

    b. Your corp account is not whitelisted by suite-scheduler project. Please contact @xixuan to whitelist your ldap.

How to test:

  1. Presubmit will kick off sanity test for verifying suite_scheduler.ini:

     python runner.py --test_type sanity
    
  2. Run unittest:

     python runner.py' or 'python runner.py --test_type unittest
    
  3. Run integration test:

     python runner.py --test_type integration
    
  4. Run tests with debug mode:

     python runner.py --debug
    
  5. Run suite scheduler v2 as a local development server:

    If you want to verify any APIs locally, you can run suite scheduler v2 as a local development server (https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server):

    a. Finally, start dev_appserver.py in a window:

       dev_appserver.py app.yaml \
           --port=8888 \
           --admin_port=8001 \
           --log_level=debug
    

    b. Test it:

       curl 'http://localhost:8888/cron/trigger_event', or open it in chrome.
    

    Please note that once a dev_appserver is started, it simulates data_store & task_queue, which means:

    a. The datastore doesn‘t contain any previous cron run’s information, so the first round of cron/trigger_event won't trigger any event.

    b. The task queue is newly created with 0 tasks in it unless cron/trigger_event adds tasks in it. Dev_appserver restart will cause the local task queue to get purged.