| # Building UnicodeJsp | |
| - Note 2: there are some notes on updated processes for using GCP at [gcp-run.md](./gcp-run.md) - at present, automated deployment is being worked on. | |
| ## Compiling | |
| ### Prerequisites | |
| - Java | |
| - Maven (download instructions [here](http://cldr.unicode.org/development/maven#TOC-Installing-Maven) | |
| - GitHub account | |
| - UnicodeJsp source downloaded | |
| ### Maven Environment setup | |
| Setup your maven environment’s `.m2/settings.xml` as per | |
| [CLDR instructions](http://cldr.unicode.org/development/maven#TOC-Getting-Started---GitHub-token). You don’t | |
| need to actually need to download or build CLDR itself. | |
| ### Building from the command line | |
| Run `mvn -am -pl UnicodeJsps package -DskipTests` at the root directory of the repository to build the JSPs. | |
| ### Building from Eclipse | |
| UnicodeJsps can be imported into eclipse using the _Import->existing maven project_ menu option. | |
| If you already have `UnicodeJsps` in eclipse, it might be better to remove it from eclipse before the import. | |
| ## Running | |
| ### Command Line | |
| ```shell | |
| mvn -DCLDR_DIR=/path/to/cldr -DUNICODETOOLS_REPO_DIR=/path/to/unicodetools -DUNICODETOOLS_GEN_DIR=../Generated org.eclipse.jetty:jetty-maven-plugin:run | |
| ``` | |
| If port 8080 is in use, another port can be specified with `-Djetty.port=⟨port number⟩`. | |
| The following system properties described in [Building Unicode Tools](../build.md#java-system-properties-used-in-unicode-tools) must be set in order for the JSPs to function properly: `CLDR_DIR`, `UNICODETOOLS_REPO_DIR`; this can be done with `-DCLDR_DIR=⟨some path⟩ -DUNICODETOOLS_REPO_DIR=. -DUNICODETOOLS_GEN_DIR=../Generated`. | |
| You can now connect to <http://127.0.0.1:8080> as suggested from the command line. | |
| Use Control-C to stop the server. | |
| > Note: this used to be `mvn jetty:run` but a more verbose command is needed at present. | |
| ### Running from within Eclipse | |
| This screenshot shows creating a `org.eclipse.jetty:jetty-maven-plugin:run` maven run configuration. | |
|  | |
| > Note: Please change the Goal to `org.eclipse.jetty:jetty-maven-plugin:run` instead of `jetty:run` | |
| ### Debugging from within eclipse | |
| Jetty provides instructions [here](https://www.eclipse.org/jetty/documentation/jetty-9/index.html#debugging-with-eclipse) for debugging | |
| within eclipse. It is unknown whether this has been attempted with the UnicodeJsps. | |
| ## Upgrading the UnicodeJsps | |
| ### Run `org.unicode.tools.UpdateJspFiles` | |
| This will copy a number of files from the unicodetools to the Jsp directory. | |
| The items it doesn't cover are discussed below. | |
| ### Other | |
| `{$workspace}/unicodetools/data/emoji/\<VERSION\>` | |
| * *\<emoji-variants\>* :construction: **TODO** | |
| Other files: | |
| * alpha2_3.txt — mapping from 3 letter to 2 letter. No change needed. | |
| * fixCodes.txt — *dump of language/territory/region alias from CLDR. Fix to use | |
| ICU or CLDR directly instead.* | |
| Files to investigate | |
| * annotations.txt - hieroglyphs | |
| * Categories.txt | |
| * Transforms? | |
| * Deva-IPA.txt | |
| * en-IPA.txt | |
| * Globe.txt | |
| * nameprep.txt | |
| * nfm.txt — *not sure what this is; investigate and use ICU instead.* | |
| * tables.txt | |
| * temp.html | |
| * test.htm | |
| ## Adding/Updating New Properties | |
| Go to `XPropertyFactory.java` to add new properties other than the ones in /props/ | |
| ### Using Beta Properties | |
| In [org.unicode.text.Settings](/unicodetools/src/main/java/org/unicode/text/utility/Settings.java), set `latestVersionPhase` to `BETA`. | |
| Build & Test | |
| 1. Run `mvn test` | |
| :construction: **TODO**: These need a lot of work; they mostly print out a lot of gorp that you | |
| need to scan over. | |
| Run the server (see above) | |
| Look at <http://localhost:8080/UnicodeJsps/properties.jsp>, and make sure that | |
| there aren't any Z-Other props at the bottom (you'll need to update via Adding | |
| New Properties if there are). | |
| ### Running a Docker-based build | |
| Download the Last Resort font. | |
| ``` | |
| wget https://github.com/unicode-org/last-resort-font/releases/latest/download/LastResort-Regular.ttf | |
| mv ./LastResort-Regular.ttf ./UnicodeJsps/src/main/webapp/ | |
| ``` | |
| compile java stuff | |
| - `mvn -B package -am -pl UnicodeJsps -DskipTests=true` | |
| - make a copy of CLDR - lots of ways to do this. `--reference-if-able ~/src/cldr` is another directory on my disk which has a copy of CLDR, to save copying. The CLDR_REF calculation is to make sure you have the same CLDR version as the build. | |
| ``` | |
| git clone https://github.com/unicode-org/cldr.git --reference-if-able ~/src/cldr | |
| CLDR_REF=$(mvn help:evaluate -Dexpression=cldr.version -q -DforceStdout | cut -d- -f3) | |
| (cd cldr ; git reset --hard ${CLDR_REF}) | |
| mkdir -p UnicodeJsps/target && tar -cpz --exclude=.git -f UnicodeJsps/target/cldr-unicodetools.tgz ./cldr/ ./unicodetools/ | |
| ``` | |
| - Regenerate the property cache files: | |
| ``` | |
| mvn compile exec:java '-Dexec.mainClass="org.unicode.jsp.RebuildPropertyCache"' -am -pl unicodetools "-DUNICODETOOLS_GEN_DIR=Generated" "-DUNICODETOOLS_REPO_DIR=." "-DCLDR_DIR=<somewhere>" | |
| tar -cpz -f UnicodeJsps/target/generated.tgz ./Generated/ | |
| ``` | |
| Now, finally build. | |
| - `docker build -t unicode/unicode-jsp:latest UnicodeJsps/` | |
| … And run. Control-C to cancel it, otherwise visit <http://127.0.0.1:8080> | |
| ``` | |
| docker run --rm -p 8080:8080 unicode/unicode-jsp:latest | |
| ``` | |
| ## Commit/PR | |
| 1. Commit the code to your own branch, create a PR on GitHub | |
| 2. Verify that all of the checks (build steps) succeeded, for example | |
| <https://github.com/unicode-org/unicodetools/pull/22/checks> in particular | |
| "build JSP.". Request reviews | |
| 3. Merge PR into master when reviewed. | |
| ## Make a release / Deploy | |
| When it is time to push a new version of the JSPs and tools, it is time to make | |
| a "release". A release is a snapshot of the entire repository, including all | |
| Unicode Tools. | |
| 1. Go to the UnicodeTools repository release page at | |
| <https://github.com/unicode-org/unicodetools/releases> | |
| 2. *Draft a New Release*Click the button, | |
| 1. Set the "Tag version" field to a release tag with the current date, such | |
| as "**release-2020-09-15**" | |
| 2. The *Target* is what the tag refers to. Leave this at **master** to tag | |
| the current master. | |
| 3. Enter a descriptive title (such as "2020-09-15 Release" | |
| 4. Optionally fill in some content (see image at right). PRs and issues | |
| will autocomplete if you want to mention a specific PR or issue.\ | |
|  | |
| 5. click Publish Release (Green button at the bottom) | |
| 3. Now, go to the (A) [ Actions tab in GitHub, and click on the (B) "Push to | |
| GCR Github Action" | |
| workflow](https://github.com/unicode-org/unicodetools/actions?query=workflow%3A%22Push+to+GCR+Github+Action%22). | |
| (see illustration) | |
|  | |
| 1. Every tagged release creates a new run of this workflow. Look for your | |
| new tag to show up and wait for it to finish building (should only take | |
| a few minutes) and show a green checkmark. ( Example at (C) : "test3" | |
| and "test4" ) | |
| 4. Now, login to https://console.cloud.google.com/ with your unicode.org | |
| account | |
| 1. At the top of the page, switch the project to [`goog-unicode-dev`](https://console.cloud.google.com/run?project=goog-unicode-dev) | |
|  | |
| 2. From the left nav / hamburger menu, find "Cloud Run". | |
| Click on the link for the service "**unicode-jsps-staging**". This way | |
| you can update the staging release before going to production. | |
| 3. Now, choose "Edit and Deploy a New Revision" | |
| 4. You will get a page that prompts you to choose a docker image to be | |
| deployed. See the illustration below. | |
|  | |
| 2. Click Select | |
| 3. expand the "us-central1-docker.pkg.dev/goog-unicode-dev/unicode-jsps" images | |
| 4. You should see the release tags (test3, test4) show up as "images". | |
| 5. Choose the image that corresponds with your deployment (such as | |
| **2020-09-15-release**) - the full title may not show up. | |
| 6. Click Select below the image. | |
| 5. Make sure "**√ Serve this revision immediately**" at the bottom is | |
| checked. | |
| 6. Finally, click the Deploy button (way at the bottom) to schedule the | |
| deployment | |
| 7. Allow 15 seconds or so to update (you will be given visual progress as | |
| the new revision takes over traffic. | |
| 5. Check | |
| 1. Test the new deployment at | |
| <https://unicode-jsps-staging-o2ookmn2oq-uc.a.run.app> | |
| 2. To check new characters (for example): | |
| <https://unicode-jsps-staging-o2ookmn2oq-uc.a.run.app/UnicodeJsps/list-unicodeset.jsp?a=\p{age=8.0}-\p{age=7.0}> | |
| 3. Check at the bottom for the right Unicode and ICU versions. | |
| 4. Click once on each top link to do simple sanity check. | |
| 5. Once that's successful then: | |
| 6. **Deploy to Production** | |
| 1. Repeat step §4.2 above, but choose the service **unicode-jsps** instead | |
| of **unicode-jsps-staging**. | |
| 7. Check | |
| 1. Go to <https://util.unicode.org/UnicodeJsps/> to check that that works, using | |
| the same steps as #3, that is: | |
| 2. To check new characters (for example): | |
| <https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=\p{age=8.0}-\p{age=7.0}> | |
| 3. Check at the bottom for the right Unicode and ICU versions. | |
| 4. Click once on each top link to do simple sanity check. | |
| 8. Revert? | |
| 1. You can choose a back-level image tag (Step 4.4 above) in order to | |
| revert to previous versions. |