-
54c9c18 Ruff: explicitly set legacy 0.15 default lint rules. by Alex Ovsienko · 5 weeks ago main
-
91b825c Ruff lint: relint //agents with ruff and remove pylint usage. by Alex Ovsienko · 7 weeks ago
-
0cae56b presubmit: Fast-path nested PRESUBMITs in agents/, gpu/, and content/test/gpu/ (part 6 of 8) by James Hawkins · 8 weeks ago
-
ad6a19d Ruff: reformat agents. by Alex Ovsienko · 3 months ago
-
83eae80 Add automatic prompt eval for feature_flags_add by Bryan Kim · 4 months ago
-
1da1da1 Update //agents presubmits to support absolute imports by Brian Sheedy · 5 months ago
-
e31023a Add auto-rater capabilities to auto prompt eval by Bryan Kim · 5 months ago
-
0d84aad doc: Add an instruction about virtualized workspace path for builds on Btrfs by Junji Watanabe · 5 months ago
-
4434ac1 Add support for internal prompts/skills and mock commands in testing. by Nathan Scoglio · 6 months ago
-
6d667a3 Remove strip call on git status result by Bryan Kim · 6 months ago
-
190476c Fix and promote adapt_builder to stable by Bryan Kim · 6 months ago
-
c27f839 Exclude prompt files from format check and update instructions by Joe Mason · 6 months ago
-
8d5abd8 Upgrade Gemini CLI version used for tests by Brian Sheedy · 7 months ago
-
749168e Default to and correctly use hermetic gCLI by Brian Sheedy · 7 months ago
-
7d63684 Use virtual siso paths for prompt evals by Brian Sheedy · 7 months ago
-
644944a Switch prompt eval tests to Gemini 3 Flash by Brian Sheedy · 7 months ago
-
e75f14a Support launching gemini.ps1 on Windows by Greg Thompson · 9 months ago
-
dbd343f Kill promptfoo process group by Brian Sheedy · 9 months ago
-
2f460ee Enable prompt eval fetch error retries by Brian Sheedy · 9 months ago
-
b116256 Switch prompt eval model to 2.5 pro by Brian Sheedy · 9 months ago
-
58141cf Change prompt eval model to flash by Brian Sheedy · 9 months ago
-
5e0808c Update prompt eval model by Brian Sheedy · 9 months ago
-
0b5fbfb Specify model for prompt eval tests by Brian Sheedy · 9 months ago
-
500ecda Update prompt eval dependencies by Brian Sheedy · 9 months ago
-
ca859ac Populate fuzzing skill by Jon Toohill · 10 months ago
-
60caadf Add --list-tests to eval_prompts by Sven Zheng · 10 months ago
-
a7ff07f Add support for extra test paths in eval_prompts.py by Ashwin Verleker · 11 months ago
-
65cda24 Add a new evaluation for class refactoring. by Jie Sheng · 12 months ago
-
5d780c0 Set system prompts correctly by Struan Shrimpton · 12 months ago
-
f3e3b91 Revert "Update promptfoo cipd tags" by Struan Shrimpton · 12 months ago
-
2609ee7 Update promptfoo cipd tags by Struan Shrimpton · 1 year ago
-
607be8b Include the prompt/response in the rdb test result by Struan Shrimpton · 1 year ago
-
e55a048 Support tool call checks in gemini cli eval framework. by Jie Sheng · 1 year ago
-
46d60b6 Make owners explicit by Struan Shrimpton · 1 year ago
-
776b148 Improve the test log output in gemini cli eval framework. by Jie Sheng · 1 year ago
-
3d5bab8 Include the test tags in the rdb tags by Struan Shrimpton · 1 year ago
-
81c4be3 Update prompt eval metric reporting by Brian Sheedy · 1 year ago
-
4fabd70 Create a new gemini cli eval to build a file. by Jie Sheng · 1 year ago
-
e856061 Tag rdb results with their metrics by Struan Shrimpton · 1 year ago
-
f343053 Update run_tests_in_file by Struan Shrimpton · 1 year ago
-
0c04b80 Clean workdirs regardless of forced by Struan Shrimpton · 1 year ago
-
a135030 [agents][eval] Store system prompt in `//GEMINI.md` by Jonathan Lee · 1 year, 1 month ago
-
17cd0aa Create a new gemini cli eval to verify run tests in file. by Jie Sheng · 1 year, 1 month ago
-
c092057 Move perf uploading to handler by Brian Sheedy · 1 year, 1 month ago
-
2930733 eval: Add negative tag filtering for tests by James Woo · 1 year, 1 month ago
-
65da9b6 Move ResultDB reporting to handler by Brian Sheedy · 1 year, 1 month ago
-
2389503 Support user-provided result handlers by Brian Sheedy · 1 year, 1 month ago
-
ba2d83a Separate reporting for each iteration by Struan Shrimpton · 1 year, 1 month ago
-
04cbefc eval: Add test filtering on metadata by James Woo · 1 year, 1 month ago
-
d2778a5 Adjust prompt eval perf upload location by Brian Sheedy · 1 year, 1 month ago
-
3a58f91 Reland "Eval Prompts: Add custom node support and cipd option for gcli" by Struan Shrimpton · 1 year, 1 month ago
-
e634b2f Revert "Eval Prompts: Add custom node support and cipd option for gcli" by Struan Shrimpton · 1 year, 1 month ago
-
4a0111a Fix perf uploading location by Brian Sheedy · 1 year, 1 month ago
-
91b5c41 Eval Prompts: Add custom node support and cipd option for gcli by Struan Shrimpton · 1 year, 1 month ago
-
0e4368b Adjust perf upload format by Brian Sheedy · 1 year, 1 month ago
-
c401587 Add prompt eval metric uploading by Brian Sheedy · 1 year, 1 month ago
-
228eed9 Prompt Evals: specify precompile_targets on the test by Struan Shrimpton · 1 year, 1 month ago
-
934b4d7 Eval Prompts: Remove promptfoo npm/src by Struan Shrimpton · 1 year, 1 month ago
-
18d80c2 Prompt Eval: Improve test reliability by Struan Shrimpton · 1 year, 1 month ago
-
3641d75 Eval Prompts: Improve readability of test results by Struan Shrimpton · 1 year, 1 month ago
-
bbec6df Refactor test/iteration results by Brian Sheedy · 1 year, 1 month ago
-
a09ab25 Add pass@k support to prompt evals by James Woo · 1 year, 1 month ago
-
d2c74f5 Adjust args for perf dashboard uploading by Brian Sheedy · 1 year, 1 month ago
-
2fa6db2 Prevent unit tests from pulling cipd packages by Struan Shrimpton · 1 year, 1 month ago
-
498a5bd Eval prompts: fix default gemini bin for 1P by Struan Shrimpton · 1 year, 1 month ago
-
4dbddcb Eval Prompts: Add trusted folders for temp HOME dir by Struan Shrimpton · 1 year, 1 month ago
-
02bdd58 Add and use by default cipd promptfoo by Struan Shrimpton · 1 year, 1 month ago
-
356a0f3 Use settings.json for gemini-cli telemetry by Brian Sheedy · 1 year, 1 month ago
-
8b4da35 Extract prompt eval scores by Brian Sheedy · 1 year, 1 month ago
-
4220cc7 Update //agents/testing/README by Struan Shrimpton · 1 year, 1 month ago
-
eb5f995 Surface gemini-cli token usage by Brian Sheedy · 1 year, 1 month ago
-
f37d63c Update eval_prompts for structured test ids by Struan Shrimpton · 1 year, 1 month ago
-
f9cc421 agents: Add pass@k configuration parsing by James Woo · 1 year, 1 month ago
-
183dad9 Fix gemini_provider console width parsing by Brian Sheedy · 1 year, 1 month ago
-
cc91996 Refactor gemini_provider's call_api by Brian Sheedy · 1 year, 1 month ago
-
fad9ca4 Docs: Improve btrfs setup instructions by James Woo · 1 year, 1 month ago
-
52c65dc agents: Centralize Gemini helper functions by James Woo · 1 year, 1 month ago
-
9a59afd feat(eval): Set sandbox PATH from container image by James Woo · 1 year, 1 month ago
-
e5537eb Add an unrestricted parallel option by Struan Shrimpton · 1 year, 1 month ago
-
069ef68 [agents][eval] Introduce reusable `check_gtests.py` assert by Jonathan Lee · 1 year, 1 month ago
-
935f99d [agents][eval] Enable `use_remoteexec` to speed up builds by Jonathan Lee · 1 year, 1 month ago
-
f7162c1 feat(eval): Mount depot_tools in sandbox by James Woo · 1 year, 1 month ago
-
583c967 Add missing prompt eval arg by Brian Sheedy · 1 year, 1 month ago
-
cb6d335 refactor(eval): Directly fetch sandbox image by James Woo · 1 year, 1 month ago
-
458560c Add remaining isolated script args + validation by Brian Sheedy · 1 year, 1 month ago
-
e692bd3 Add prompt eval --isolated-script-test-repeat by Brian Sheedy · 1 year, 1 month ago
-
850bc85 feat(eval): Use temp home for test environments by James Woo · 1 year, 1 month ago
-
45403f3 Update prompt eval test filtering by Brian Sheedy · 1 year, 1 month ago
-
858723a Rename example and test extensions with underscores by Struan Shrimpton · 1 year, 1 month ago
-
76e79dc Refactor prompt eval argument parsing by Brian Sheedy · 1 year, 1 month ago
-
ad78f25 Update build_information MCP to build-information by Struan Shrimpton · 1 year, 1 month ago
-
f91b762 Add OWNERS to extensions and testing by Struan Shrimpton · 1 year, 1 month ago
-
74e36b1 Add missing //agents/testing tests by Brian Sheedy · 1 year, 1 month ago
-
cd484d4 feat(eval): Add support for local dev binaries by James Woo · 1 year, 1 month ago
-
8e5c311 Reland "Add presubmit checks for `promptfoo.yaml` files." by Jiamei Liu · 1 year, 1 month ago
-
8f6a2b5 [6/6] Parallel worker cleanup by Brian Sheedy · 1 year, 1 month ago
-
3ced775 [5/6?] Support multiple parallel workers by Brian Sheedy · 1 year, 1 month ago
-
6cba49a feat(testing): Add test retries for flaky tests by James Woo · 1 year, 1 month ago
-
68b0b0d [4/?] Move promptfoo installation code to new file by Brian Sheedy · 1 year, 1 month ago
-
3572449 [3/?] Move WorkDir to new file by Brian Sheedy · 1 year, 1 month ago