| commit | 25cdcf62bb05fc30ef9b418fc7ff3731263c79cd | [log] [tgz] |
|---|---|---|
| author | yuanchieh <yuanchieh@google.com> | Thu Jan 09 05:39:33 2025 |
| committer | Chromium LUCI CQ <chromium-scoped@luci-project-accounts.iam.gserviceaccount.com> | Thu Jan 09 05:39:33 2025 |
| tree | e8fd77f3ab646a620bfdeae0bf5f9fbf1747606d | |
| parent | e0062bc5cb7b6474714d5a960c1b835cb29576fa [diff] |
recorder: chunk content before input into T&S model To prevent out-of-memory errors when processing long input sequences with the T&S model (observed with inputs exceeding 8k tokens), this change implements content chunking. Instead of processing the entire input at once, the content is divided into smaller chunks, and safety is verified individually for each chunk. This approach mitigates excessive memory consumption by the T&S model, particularly for longer inputs. Testing on Navi with 11k tokens input, safety check execution time after chunking is around 2 seconds. BUG=b:385078550 TEST=manual; test on brya with Xss model & rauru with Xs model Change-Id: I3bd1ebe2031fd8603d0deaa2fab30de1e21445ff Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6108466 Reviewed-by: Pi-Hsun Shih <pihsun@chromium.org> Commit-Queue: Yuan-Chieh Cheng <yuanchieh@google.com> Reviewed-by: Jennifer Ling <hsuanling@google.com> Cr-Commit-Position: refs/heads/main@{#1404001}
ChromiumChromium is an open-source browser project that aims to build a safer, faster, and more stable way for all users to experience the web.
The project's web site is https://www.chromium.org.
To check out the source code locally, don't use git clone! Instead, follow the instructions on how to get the code.
Documentation in the source is rooted in docs/README.md.
Learn how to Get Around the Chromium Source Code Directory Structure.
For historical reasons, there are some small top level directories. Now the guidance is that new top level directories are for product (e.g. Chrome, Android WebView, Ash). Even if these products have multiple executables, the code should be in subdirectories of the product.
If you found a bug, please file it at https://crbug.com/new.