| commit | f1af215f0a2aba1e5c88f22cc1d443cddf2df7d7 | [log] [tgz] |
|---|---|---|
| author | George Burgess IV <gbiv@google.com> | Mon May 22 16:23:16 2023 |
| committer | Chromeos LUCI <chromeos-scoped@luci-project-accounts.iam.gserviceaccount.com> | Wed May 24 19:05:19 2023 |
| tree | 548418be277f0b4c9c1e579650f6ba18c7449c23 | |
| parent | f944ece0a5f5bb11ce0a31c2b906fea577c7cbc2 [diff] |
cargo_update: run vendor.py BUG=b:264463632 TEST=vendor.py works Change-Id: I2a8a9b583d8c3e0f98cd5104199c936b885e0454 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/rust_crates/+/4557785 Tested-by: Allen Webb <allenwebb@google.com> Reviewed-by: Allen Webb <allenwebb@google.com> Commit-Queue: Allen Webb <allenwebb@google.com>
This repository contains vendored Rust libraries for ChromeOS userspace binaries. They are installed by the dev-rust/third-party-crates-src ebuild. Dependencies are fetched using cargo vendor.
Please see the individuals listed in the OWNERS file if you'd like to learn more about this repo.
In order to update or add any package, follow these steps:
projects/ corresponding to the first-party package you'd like to update. If it does not exist, please see the “Adding a first-party package,” section.Cargo.toml to add, remove or upgrade packages.python vendor.py inside chroot.cargo vendor first, which updates Cargo.lock and puts downloaded crates into the vendor directorypatches directory. It also regenerates checksums for packages that were modified.--license-map=<filename> is given, it will dump a json file which is a dictionary with the crate names as keys and another dictionary with the license and license_file as keys.patches/ can either be named after a crate (in which case, they apply to all versions of their corresponding crate), or can have a version in their name, in which case they're meant to apply specifically to a given version of a crate.cargo vendor seems to also pick up dependencies for unused configs (i.e. windows). You will need to make sure these packages are stripped by cargo vendor.vendor.py script with the new license and also update dev-rust/third-party-crates-src with the new license.NOTE: If your patch changes over 30 files, please see the Splitting Patches section below.
When it is necessary to patch a package due to incompatibility, you can create a patch targeting the specific package and store it in patches/. For any given ${crate} at ${version}, if patches/${crate}-${version} exists, the patches from that directory are applied to the crate. If such a directory does not exist, patches/${crate} is checked. Similarly, if this exists, patches are applied; otherwise, the crate is left unpatched.
If ./vendor.py complains about a specific directory in patches/ not having a corresponding vendor/ directory, the most likely fixes are:
Patches can come in two forms. Files with names ending in .patch are always applied with patch -p1 to the vendor directory. Executable files that do not have names ending in patch will be executed in the vendor directory which they should apply to. All other files are ignored.
Updates to this repo will be captured by the CQ. To directly test changes, either build the net-wireless/floss package, or run cros_workon --board=${BOARD} dev-rust/third-party-crates-src and build packages + run tests for your board of choice.
The projects/ subdirectory contains a set of Cargo.toml files that roughly correspond to what exists in the ChromeOS tree. These exist only to provide dependency information with which we can create our vendor/ directory, so everything is removed except for author information, dependencies, and (if necessary) a minimal set of features/annotations. Dependency sections outside of what would build for ChromeOS, like [target.'cfg(windows)'.dependencies], are also removed.
Once your Cargo.toml seems correct and you're ready to test, run projects/populate-workspace.py to add it to the workspace.
Admittedly, it's sort of awkward to have two Cargo.tomls for each first-party project. It may be worth trying to consolidate this in the future, though our future bazel migration potentially influences what the ‘ideal’ setup here is.
NOTE: If your patch changes over 30 files, please see the Splitting Patches section below.
cargo vendor will trim the vendor directory based on the features you require of your dependencies, but not their optionality. For instance, given:
[dependencies]
tokio = { version = "1", optional = true }
cargo vendor will simply vendor tokio as though the optional = true isn‘t present. That said, it’s important to note that cargo vendor will try to trim optional dependencies of crates which Cargo.toml files depend upon. For instance, given:
[dependencies]
tokio = { version = "1", features = [] }
cargo vendor selects a much smaller set of crates to vendor than it would if features = [] were replaced with features = ["full"]. Hence, Cargo.tomls should be sure to either specify all of the features they require explicitly in a dependency's features = [] block, or keep a crate-level [features] block which makes the set of potential features of dependencies we use visible to cargo vendor. For example, both of the following two Cargo.toml bodies are OK:
[dependencies]
tokio = { version = "1", features = ["full"] }
[dependencies]
tokio = { version = "1", features = [] }
[features]
tokio-full = [ "tokio/full" ]
tl;dr: if your crate has optional dependencies, feel free to drop the optional = annotation. If your crate depends on features of other crates, please do not remove those.
The crate names in these Cargo.tomls aren‘t really relevant or used; please add some context that seems meaningful to your package’s name, or do that to the conflicting package.
Due to the nature of rust_crates, patches can get massive (tens-hundreds of thousands of lines of diff over thousands of files). For ease of review, it's requested that large changes are split into a few logically independent pieces:
vendor/.vendor/ as a result of running ./vendor.py.Note that Cargo.lock changes may land in either the first or second CLs. Use your judgment to determine whether Cargo.lock changes are noise (e.g., “I added a new package, and Cargo.lock now reflects that.”) or signal (e.g., “I ran cargo update on a few packages, and Cargo.lock is the only thing outside of vendor/ that can reflect that.”).
These patches should all be landed at the same time. The split is meant to help increase reviewability. If you‘re unfamiliar with Gerrit’s Relation chain feature, you can make a patch stack like the above by making multiple commits in git, and running repo upload as usual.
If you have one large patch that you'd like to split out into three, git checkout may be helpful to you:
## Assuming you currently have the full change you want to commit ## checked out, and `git status` says your repo is clean: $ all_changes=$(git rev-parse HEAD) $ git checkout HEAD~ -- vendor/ projects/Cargo.lock $ git commit --amend --no-edit $ git checkout "${all_changes}" -- vendor projects/Cargo.lock $ git commit -a # This commits vendor changes. ## NOTE: if any part of this fails, you can always get back to your original ## state by running `git reset --hard ${all_changes}`.
To ensure your splitting did the right thing, you can run git diff ${all_changes}. If no changes are printed and git status is clean, you successfully split your CL.
In order to make changes to old CLs, git rebase -i and/or git commit --fixup combined with git rebase -i --autosquash might be helpful to you. Please see git's manual for more info.
vendor.py complaining about cargo-vet audits?Please see the cargo-vet readme.
dev-rust/third-party-crates-src?dev-rust/third-party-crates-src is a cros_workon package. If you‘d like to hack on it for a board, you can use cros-workon-${BOARD} start dev-rust/third-party-crates-src. If you’d like changes to apply to the host, use cros-workon --host start dev-rust/third-party-crates-src instead. After running one of these, any emerges of dev-rust/third-party-crates-src should output that the 9999 version of this package is being emerged, and your local changes should apply.
As an example, for local development on dependencies for the host, the “add a crate and make the change live” cycle looks like:
$ cros-workon --host start dev-rust/third-party-crates-src $ while ! rust_crates_does_what_i_want; do $EDITOR projects/foo/Cargo.toml && \ ./vendor.py && \ sudo emerge dev-rust/third-party-crates-src done
Use the package field:
# Depend on both wasi 0.10.0 and 0.9.0.
[dependencies]
wasi-0-10 = { package = "wasi", version = "0.10.0" }
wasi-0-9 = { package = "wasi", version = "0.9.0" }
vendor.py will automatically clean up packages that aren't used for ChromeOS triples, so if you just need to placate Cargo by having e.g., a Windows-only package available, you can put your dependencies in a cfg(windows) dependency block:
# By putting this in a cfg(windows) block, a package for wasi-0.10.* will be # installed in the registry, but it won't actually contain any code from the # package. # # As a side-effect from the above, this package will be exempt from audit # requirements. [target.'cfg(windows)'.dependencies] wasi = "0.10"