| # Forklift Utility |
| |
| ## Overview |
| |
| The forklift utility aids in backporting large numbers of commits from |
| upstream to a local branch. To accomplish this, it uses a report file to keep |
| track of the commits which have been backported and which are outstanding. |
| |
| The report file contains the list of commits to be cherry picked as well as the |
| current progress of the forklift. The utility uses it to keep track of progress |
| across invocations. The file is human-readable json, and can be read/altered if |
| necessary. It should not be necessary to inspect/modify the report fie, if you |
| find yourself doing this, please send patches :-). |
| |
| |
| ## Operation |
| The order of operations for a forklift is as follows: |
| |
| ### Find the target pull request |
| Seach lore.kernel.org for the pull request you would like to forklift. In this |
| example we will use the [drm 5.11 pull request](https://lore.kernel.org/dri-devel/CAPM=9tyNrbap4FG6qstkC5YTznqVebD=ye+4+Z+t42yQnL325A@mail.gmail.com/). |
| |
| From here, you will need the `Message-Id` as well as the mailing list/project to |
| gather it from. In our case, they are: |
| ``` |
| List/Project: dri-devel |
| Message-ID: CAPM=9tyNrbap4FG6qstkC5YTznqVebD=ye+4+Z+t42yQnL325A@mail.gmail.com |
| ``` |
| |
| ### Generate the report |
| Now that we have the pull request, we start a new branch in our target kernel |
| directory (using `repo sync` or `git`) and generate the report. |
| |
| ``` |
| $ forklift.py generate-report \ |
| --report-path forklift.json \ |
| --list dri-devel \ |
| --msg-id 'CAPM=9tyNrbap4FG6qstkC5YTznqVebD=ye+4+Z+t42yQnL325A@mail.gmail.com' \ |
| --common-ancestor v5.9 \ |
| --bug 'b:12345678' |
| --test 'Trust me.' |
| ``` |
| |
| In this case we've also specified a common ancestor between the 2 trees. This |
| is a commit which is common to both trees. In our case we've chose the 5.9 |
| release tag. While this is not necessary, it allows the forklift utility to |
| skip common history when looking for commits. |
| |
| ### Cherry-pick the commits |
| Now that the report has been generated, the utility knows which commits it |
| should try to backport. You can use the cherry-pick subcommand to start |
| pulling back patches. |
| |
| ``` |
| $ forklift.py cherry-pick --report-path forklift.json |
| ``` |
| |
| The tool will pause when it reaches a merge conflict. At this point you must |
| manually fix the conflict and choose a resolution to continue. The menu will |
| present three options to continue: |
| ``` |
| c - Conflict resolved, patch is HEAD. Continue |
| s - Patch was not needed. Skip it. |
| q - Quit |
| Please resolve the conflict and choose an option: |
| ``` |
| |
| Choice 'c' will add a BACKPORT prefix to the subject of the HEAD commit as |
| well as the BUG and TEST entries at the bottom of the commit message. Choice |
| 's' will mark the commit as complete in the report and move to the next |
| commit. Choice 'q' will exit immediately, leaving the conflicting commit |
| unfinished. |
| |
| If you leave the conflicting commit unfinished, you can use the |
| `complete-cherry-pick` subcommand later to mark it complete. |
| |
| ### Resolving conflicts |
| Conflict resolution is the most time consuming part of forklifting. This |
| utility can help disect where the conflicts are coming from to expedite |
| the process. |
| |
| One will typically want to resolve a conflict in the following order of |
| operation. |
| |
| 1. Review the entire conflict by using `Print conflict`. |
| 2. Compare the output in HEAD (`Print head`) with the output in remote |
| (`Print remote`). |
| 3. Once the difference has been identified, use `Blame head` and `Blame remote` |
| to identify the commit causing the conflict. |
| 4. Resolve the conflict and record the resolution in the commit message. |
| |
| See the `resolve-conflict` subcommand documentation below for detailed information |
| and output on the above choices. |
| |
| |
| ## Subcommands |
| ### generate-report |
| |
| This subcommand creates the report file from a pull request on lore.kernel.org. |
| |
| Usage: |
| ``` |
| usage: forklift generate-report [-h] [--git-path GIT_PATH] --report-path |
| REPORT_PATH --list LIST --msg-id MSG_ID |
| [--bug BUG] [--test TEST] |
| [--common-ancestor COMMON_ANCESTOR] |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| --git-path GIT_PATH, -g GIT_PATH |
| Path to git repository (if not current dir). |
| --report-path REPORT_PATH, -m REPORT_PATH |
| Path to store forklift report file. |
| --list LIST Mailing list from lore.kernel.org/lists.html. |
| --msg-id MSG_ID Message-Id for the pull request to process. |
| --bug BUG Value to use for BUG= in commit descriptions. |
| --test TEST Value to use for TEST= in commit descriptions. |
| --common-ancestor COMMON_ANCESTOR |
| Optional common ancestor between the local and remote |
| trees. Improves execution time if provided. |
| ``` |
| |
| To generate your report, provide the subcommand with the project on lore (find |
| this on the homepage) and the Message-Id value of the pull request message. Finally, choose a location for the report file and pass this in as report-path. |
| |
| Once invoked, forklift will fetch the remote tag/branch and compare it with the |
| contents of the local branch. It will write the patches into the report file for |
| later use. |
| |
| |
| ### cherry-pick |
| This subcommand cherry-picks the outstanding patches from the report to the |
| local branch. |
| |
| Usage: |
| ``` |
| usage: forklift.py cherry-pick [-h] [--git-path GIT_PATH] --report-path |
| REPORT_PATH |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| --git-path GIT_PATH Path to git repository (if not current dir). |
| --report-path REPORT_PATH |
| Path to store forklift report file. |
| ``` |
| |
| To start cherry-picking, invoke the subcommand with an already generated |
| report. For each commit, the utility will try to determine if the patch exists |
| in the local branch and will invoke `git cherry-pick` if not. |
| |
| If the cherry-pick results in an empty commit, this will skip the commit and |
| continue. |
| |
| If there is a conflict which needs intervention, the subcommand will stop |
| cherry-picking and wait. There are typically 2 resolutions: |
| |
| 1. Manually resolve the commit and apply it. |
| 2. If this patch is not needed, discard it. |
| |
| ### complete-cherry-pick |
| This marks a commit in the report file as backported. |
| |
| Usage: |
| ``` |
| usage: forklift.py complete-cherry-pick [-h] --report-path REPORT_PATH |
| [--git-path GIT_PATH] |
| [--commit COMMIT] |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| --report-path REPORT_PATH |
| Path to store forklift report file. |
| --git-path GIT_PATH Path to git repository (if not current dir). |
| --commit COMMIT Git hash to identify a commit. |
| ``` |
| |
| This can be useful when cherry-picking a file outside of the cherry-pick |
| subcommand and wants the utility to skip a commit. |
| |
| ### resolve |
| Aids in resolving a conflict. |
| |
| _Note:_ this is not tied to the report and can be used for any conflict. |
| |
| Usage: |
| ``` |
| usage: forklift.py resolve [-h] [--git-path GIT_PATH] |
| |
| optional arguments: |
| -h, --help show this help message and exit |
| --git-path GIT_PATH Path to git repository (if not current dir). |
| ``` |
| |
| This subcommand has 3 different sub-menus with different options: |
| |
| #### File Selection menu |
| ``` |
| $ forklift.py resolve |
| |
| 1 - drivers/gpu/drm/mediatek/mtk_drm_drv.c |
| 2 - drivers/gpu/drm/rockchip/rockchip_drm_drv.c |
| q - Exit |
| Choose a file to resolve: |
| ``` |
| |
| This menu allows you to choose which file you'd like to inspect. Enter the |
| number in the prompt. |
| |
| #### Conflict Selection Menu |
| ``` |
| Attempting to resolve conflicts in drivers/gpu/drm/mediatek/mtk_drm_drv.c |
| |
| 1 - Line 334 |
| bh - Blame file HEAD |
| br - Blame file remote |
| b - Back |
| q - Quit |
| Choose a conflict to resolve: |
| ``` |
| |
| Each conflict in the selected file will show up as a menu item. In the example |
| above, there is only one conflict at line 334. |
| |
| Additionally, if more context is required, the other options in this menu |
| allow one to view the `git blame` output for the entire file. `Blame file |
| HEAD` can be used to view the blame output for the file on disk in the |
| local branch. `Blame file remote` shows the blame output for the remote file |
| _before_ the remote commit was applied. This allows one to determine if |
| there are missing commits causing the conflict. |
| |
| #### Conflict Inspection Menu |
| ``` |
| c - Print conflict |
| h - Print head |
| r - Print remote |
| rc - Print remote commit |
| bh - Blame head |
| br - Blame remote |
| b - Back |
| q - Quit |
| Choose an action: |
| ``` |
| This menu allows one to inspect each conflict in isolation. The options |
| are as follows: |
| |
| - `Print conflict` This outputs the entire conflict from the local file. |
| ``` |
| 334 <<<<<<< HEAD |
| 335 static struct drm_driver mtk_drm_driver = { |
| 336 .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC | |
| 337 DRIVER_RENDER, |
| 338 ======= |
| 339 static const struct drm_driver mtk_drm_driver = { |
| 340 .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC, |
| 341 >>>>>>> 70a59dd82959f.. drm/<drivers>: Constify struct drm_driver |
| ``` |
| - `Print head` This outputs the HEAD portion of the conflict from the local |
| file. |
| ``` |
| 334 <<<<<<< HEAD |
| 335 static struct drm_driver mtk_drm_driver = { |
| 336 .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC | |
| 337 DRIVER_RENDER, |
| 338 ======= |
| ``` |
| - `Print remote` This outputs the remote portion of the conflict from the local file. |
| ``` |
| 338 ======= |
| 339 static const struct drm_driver mtk_drm_driver = { |
| 340 .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC, |
| 341 >>>>>>> 70a59dd82959f.. drm/<drivers>: Constify struct drm_driver |
| ``` |
| - `Print remote commit` This outputs the conflicting commit in entirety. |
| ``` |
| |
| commit 70a59dd82959f828220bf3f5f336e1b8fd931d15 |
| Author: Daniel Vetter <daniel.vetter@ffwll.ch> |
| Date: Wed Nov 4 11:04:24 2020 +0100 |
| |
| drm/<drivers>: Constify struct drm_driver |
| |
| Only the following drivers aren't converted: |
| - amdgpu, because of the driver_feature mangling due to virt support. |
| Subsequent patch will address this. |
| - nouveau, because DRIVER_ATOMIC uapi is still not the default on the |
| platforms where it's supported (i.e. again driver_feature mangling) |
| ... |
| |
| diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c |
| index 1f6682032ca49..6b99df6963842 100644 |
| --- a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c |
| +++ b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c |
| @@ -58,7 +58,7 @@ static irqreturn_t komeda_kms_irq_handler(int irq, void *data) |
| return status; |
| } |
| |
| -static struct drm_driver komeda_kms_driver = { |
| +static const struct drm_driver komeda_kms_driver = { |
| .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC, |
| .lastclose = drm_fb_helper_lastclose, |
| DRM_GEM_CMA_DRIVER_OPS |
| |
| ... |
| ``` |
| - `Blame head` This prints the `git blame` output for the HEAD portion of the |
| conflict (with context). |
| ``` |
| 4c6f3196e6ea11 (Alexandre Courbot 2019-07-29 14:33:34 +0900 331) return drm_gem_prime_import_dev(dev, dma_buf, private->dma_dev); |
| 4c6f3196e6ea11 (Alexandre Courbot 2019-07-29 14:33:34 +0900 332) } |
| 4c6f3196e6ea11 (Alexandre Courbot 2019-07-29 14:33:34 +0900 333) |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 335) static struct drm_driver mtk_drm_driver = { |
| a2c4410ab39d64 (CK Hu 2016-01-12 16:15:50 +0100 336) .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC | |
| a2c4410ab39d64 (CK Hu 2016-01-12 16:15:50 +0100 337) DRIVER_RENDER, |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 342) |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 343) .dumb_create = mtk_drm_gem_dumb_create, |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 344) |
| ``` |
| |
| - `Blame remote` This prints the change the remote commit is attempting to make |
| to the code and the `git blame` output for the remote portion of the conflict |
| _before_ the remote commit was applied. |
| |
| ``` |
| >>>>> Possible result 0, score=0 |
| -- diff |
| return drm_gem_prime_import_dev(dev, dma_buf, private->dma_dev); |
| } |
| |
| -static struct drm_driver mtk_drm_driver = { |
| +static const struct drm_driver mtk_drm_driver = { |
| .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC, |
| |
| .dumb_create = mtk_drm_gem_dumb_create, |
| -- blame |
| 4c6f3196e6ea11 (Alexandre Courbot 2019-07-29 14:33:34 +0900 322) } |
| 4c6f3196e6ea11 (Alexandre Courbot 2019-07-29 14:33:34 +0900 323) |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 324) static struct drm_driver mtk_drm_driver = { |
| 0424fdaf883a68 (Daniel Vetter 2019-06-17 17:39:24 +0200 325) .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC, |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 326) |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 327) .dumb_create = mtk_drm_gem_dumb_create, |
| 119f5173628aa7 (CK Hu 2016-01-04 18:36:34 +0100 328) |
| ``` |
| |
| By inspecting the above, we can see the conflict resulted from the different |
| values of `.driver_features` between upstream and downstream. When we inspect |
| those commits, we see that downstream patch `a2c4410ab39d6 ("CHROMIUM: |
| drm/mediatek: Add interface to allocate Mediatek GEM buffer.")` introduced the |
| `DRIVER_RENDER` flag which causes the conflict. |
| |
| In some cases there are multiple results. This is because mapping a conflict in |
| a local file to remote blame is not 1:1. The utility will attempt to show the |
| most relevant results. If all else fails, one can list the entire blame output |
| in the file menu. |