tree: 737cbe2691fa10405027bc246ac280c0d2072dc9 [path history] [tgz]
  1. .coveragerc
  2. .eslintrc.json
  3. .expect_tests.cfg
  4. .expect_tests_pretest.py
  5. .gitignore
  6. .style.yapf
  7. .testcoveragerc
  8. Makefile
  9. OWNERS
  10. PRESUBMIT.py
  11. README.md
  12. api/
  13. app-cloud.yaml.m4
  14. app.yaml.m4
  15. appengine_config.py
  16. businesslogic/
  17. codereview.settings
  18. cron.yaml
  19. dispatch.yaml
  20. doc/
  21. dos.yaml
  22. features/
  23. framework/
  24. google_proto/
  25. jsconfig.json
  26. karma.conf.js
  27. module-api-cloud.yaml.m4
  28. module-api.yaml.m4
  29. module-besearch-cloud.yaml.m4
  30. module-besearch.yaml.m4
  31. module-latency-insensitive-cloud.yaml.m4
  32. module-latency-insensitive.yaml.m4
  33. monorailapp.py
  34. package-lock.json
  35. package.json
  36. project/
  37. proto/
  38. queue.yaml
  39. registerpages.py
  40. schema/
  41. search/
  42. services/
  43. settings.py
  44. sitewide/
  45. static/
  46. static_src/
  47. templates/
  48. testing/
  49. third_party/
  50. tools/
  51. tracker/
  52. webpack.config.js
appengine/monorail/README.md

Monorail Issue Tracker

Monorail is the Issue Tracker used by the Chromium project and other related projects. It is hosted at bugs.chromium.org.

If you wish to file a bug against Monorail itself, please do so in our self-hosting tracker. We also discuss development of Monorail at infra-dev@chromium.org.

Getting started with Monorail development

Here's how to run Monorail locally for development on MacOS and Debian stretch/buster or its derivatives.

  1. You need to get the Chrome Infra depot_tools commands to check out the source code and all its related dependencies and to be able to send changes for review.
  2. Check out the Monorail source code
    1. cd /path/to/empty/workdir
    2. fetch infra
    3. cd infra/appengine/monorail
  3. Make sure you have the AppEngine SDK:
    1. It should be fetched for you by step 1 above (during runhooks)
    2. Otherwise, you can download it from https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python
  4. Install MySQL v5.6.
    1. On a Debian derivative, download and unpack this bundle:
      1. tar -xf mysql-server_5.6.40-1ubuntu14.04_amd64.deb-bundle.tar
    2. Install the packages in the order of mysql-common,mysql-community-client, mysql-client, then mysql-community-server.
    3. As of March 2020, MacOS MySQL setup will look something like this:
      1. Install docker
      2. Run MySQL@5.6 in docker
      3. Install MySQL locally (only needed for MySQL-python)
      4. Set MySQL user passwords
      5. Install MySQL-python
      6. Initialize database with tables
      7. Run the app locally
      
      1. Install docker for mac.
      2. Start your docker container named mysql for MySQL v5.6.
        1. docker run --name mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -p 3306:3306 -d mysql:5.6
      3. Use homebrew to install MySQL v5.6 on your system for its configs to install MySQLdb later. We will be using the docker container's MySQL instance.
        1. brew install mysql@5.6
    4. Otherwise, download from the official page.
      1. Do not download v5.7 (as of April 2016)
  5. Get the database backend configured and running:
    1. On Debian
      1. Allow passwordless authentication from non-root users:
        1. sudo mysql -uroot mysql -e "UPDATE user SET host='%', plugin='' WHERE user='root'; FLUSH PRIVILEGES;"
      2. Disable STRICT_TRANS_TABLES
        1. echo -e "[mysqld]\nsql_mode = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" | sudo tee /etc/mysql/conf.d/99-sql-mode.cnf
      3. sudo /etc/init.d/mysql restart
    2. On MacOS
      1. Enter the docker container and update its authentication.
        1. docker exec -it mysql bash
        2. mysqladmin -u root -p'my-secret-pw' password ''
        3. mysql -uroot mysql -e "UPDATE user SET host='172.17.0.1', plugin='' WHERE user='root'; FLUSH PRIVILEGES; exit;"
          1. We set the host to 172.17.0.1 to match the docker container. Your docker container may be at a different IP, you use docker network inspect bridge from another terminal to verify your container's IP.
      2. Exit the docker container
        1. exit
  6. Install Python MySQLdb.
    1. On Debian, use apt-get:
      1. sudo apt-get install python-mysqldb
    2. On MacOS, use pip
      1. export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
      2. Check if you have pip, if you don't, install it first
        1. sudo easy_install pip
      3. pip install MYSQL-python
    3. Otherwise, download from http://sourceforge.net/projects/mysql-python/
      1. Follow instructions to install.
      2. If needed, add these lines to your ~/.profile file and restart on MacOS 10.8:
        1. setenv DYLD_LIBRARY_PATH /usr/local/mysql/lib/
        2. setenv VERSIONER_PYTHON_PREFER_64_BIT no
        3. setenv VERSIONER_PYTHON_PREFER_32_BIT yes
      3. In Mac OSX 10.11.1, if you see errors about failing to import MySQLdb or that _mysql.so references an untrusted relative path, then run: sudo install_name_tool -change libmysqlclient.18.dylib \ /usr/local/mysql/lib/libmysqlclient.18.dylib \ /Library/Python/2.7/site-packages/_mysql.so
  7. Set up one master SQL database. (You can keep the same sharding options in settings.py that you have configured for production.).
    1. On Debian
      1. mysql --user=root -e 'CREATE DATABASE monorail;'
      2. mysql --user=root monorail < schema/framework.sql
      3. mysql --user=root monorail < schema/project.sql
      4. mysql --user=root monorail < schema/tracker.sql
    2. On MacOS
      1. Install your schema into the docker container
        1. docker cp schema/. mysql:/schema
        2. docker exec -it mysql bash
        3. mysql --user=root monorail < schema/framework.sql
        4. mysql --user=root monorail < schema/project.sql
        5. mysql --user=root monorail < schema/tracker.sql
  8. Configure the site defaults in settings.py. You can leave it as-is for now.
  9. Set up the front-end development environment:
    1. On Debian
      1. eval `../../go/env.py` -- you'll need to run this in any shell you wish to use for developing Monorail. It will add some key directories to your $PATH.
      2. npm install
      3. Install build requirements:
        1. sudo apt-get install build-essential automake
    2. On MacOS
      1. Install npm
        1. brew install nvm
        2. See the brew instructions on updating your shell's configuration
        3. nvm install 12.13.0
        4. npm install
  10. Run the app:
    1. make serve
  11. Browse the app at localhost:8080 your browser.
  12. Optional: Create/modify your Monorail User row in the database and make that user a site admin. You will need to be a site admin to gain access to create projects through the UI.
    1. mysql --user=root monorail -e "UPDATE User SET is_site_admin = TRUE WHERE email = 'test@example.com';"
    2. If the admin change isn't immediately apparent, you may need to restart your local dev appserver.

Instructions for deploying Monorail to an existing instance or setting up a new instance are here.

Here's how to run unit tests from the command-line:

Testing

To run all Python unit tests, in the appengine/monorail directory run:

make test

For quick debugging, if you need to run just one test you can do the following. For instance for the test IssueServiceTest.testUpdateIssues_Normal in services/test/issue_svc_test.py:

../../test.py test appengine/monorail:services.test.issue_svc_test.IssueServiceTest.testUpdateIssues_Normal --no-coverage

Frontend testing

To run the frontend tests for Monorail, you first need to set up your Go environment. From the Monorail directory, run:

eval `../../go/env.py`

Then, to run the frontend tests, run:

make karma

To run only one test or a subset of tests, you can add .only to the test function you want to isolate:

// Run one test.
it.only(() => {
  ...
});

// Run a subset of tests.
describe.only(() => {
  ...
});

Just remember to remove them before you upload your CL.

Troubleshooting

  • TypeError: connect() got an unexpected keyword argument 'charset'

This error occurs when dev_appserver cannot find the MySQLdb library. Try installing it via sudo apt-get install python-mysqldb.

  • TypeError: connect() argument 6 must be string, not None

This occurs when your mysql server is not running. Check if it is running with ps aux | grep mysqld. Start it up with /etc/init.d/mysqld start on linux, or just mysqld.

  • dev_appserver says OSError: [Errno 24] Too many open files and then lists out all source files

dev_appserver wants to reload source files that you have changed in the editor, however that feature does not seem to work well with multiple GAE modules and instances running in different processes. The workaround is to control-C or kill the dev_appserver processes and restart them.

  • IntegrityError: (1364, "Field 'comment_id' doesn't have a default value") happens when trying to file or update an issue

In some versions of SQL, the STRICT_TRANS_TABLES option is set by default. You'll have to disable this option to stop this error.

  • ImportError: No module named six.moves

You may not have six.moves in your virtual environment and you may need to install it.

  1. Determine that python and pip versions are possibly in vpython-root
    1. which python
    2. which pip
  2. If your python and pip are in vpython-root
    1. sudo `which python` `which pip` install six

Development resources

Supported browsers

Monorail supports all browsers defined in the Chrome Ops guidelines.

File a browser compatability bug here.

Frontend code practices

See: Monorail Frontend Code Practices

Monorail's design

Triage process

See: Monorail Triage Guide

Release process

See: Monorail Deployment

User guide

For information on how to user Monorail, see the Monorail User Guide.