Expose API to parse known absl flags. This change makes it easier for consumers such as Chromium to use fuzztest. It changes argument handling in two ways: 1) It exposes a new 'InitAbslFlagsForFuzztest' API. Chromium's test suites otherwise needed to call into absl flags directly to initialize absl's command line; this was problematic in projects where absl flags are not normally used. Integration of fuzztest is easier if we need to call only fuzztest APIs instead of absl flags directly. A key point of this new API is that it does not call absl::ParseCommandLine, but instead absl::ParseAbseilFlagsOnly. This ignores unknown command-line options, which is important in environments where other code may use non-absl parsing of command lines. 2) fuzztest_gtest_main is amended to use the above function instead of absl::ParseCommandLine. This is because, when used with libfuzzer or other non-centipede fuzzing engines, those fuzzing engines themselves absorb command linm arguments, and thus the previous use of absl::ParseCommandLine always failed unless a complex '-undefok=' argument was constructed listing all the possible options that could be passed to libfuzzer. As an indication of the complexity here, in a typical Chromium test, command-line arguments may flow to four places: * FuzzTest's absl flags * libfuzzer, passed there via InitFuzzTest (or some other fuzzing engine) * Chromium's own test frameworks (flags like --single-process-tests) * Chromium's underlying code, e.g. for enabling runtime features or disabling the sandbox. Because of the many areas of code handling command line arguments, it's not practical to declare them all in advance either using absl flags, nor using `-undefok` and that's why it's important that FuzzTest uses absl::ParseAbseilFlagsOnly to avoid erroring in the face of unknown extra command line arguments. This CL does not fix a third problem in this area, which is that the fuzztest engine stores the given argc and argv and assumes it never subsequently changes - this assumption is wrong in projects such as Chromium where other layers of software may absorb and modify the command line later, and results in use-after-return bugs. However this can be easily worked around by storing an extra copy of the original argc and argv for use by fuzztest, disregarding subsequent changes. PiperOrigin-RevId: 625785463
FuzzTest is a C++ testing framework for writing and executing fuzz tests, which are property-based tests executed using coverage-guided fuzzing under the hood. Fuzz tests are like regular unit tests, but more generic and more powerful. Instead of saying: “for this specific input, we expect this specific output”, we can say: “for these types of input, we expect this generic property to be true”. For example:
void MyApiAlwaysSucceedsOnPositiveIntegers(int i) { bool success = MyApi(i); EXPECT_TRUE(success); } FUZZ_TEST(MyApiTest, MyApiAlwaysSucceedsOnPositiveIntegers) .WithDomains(/*i:*/fuzztest::Positive<int>());
It is our latest fuzz testing technology and the successor of previously used fuzzing tools, such as libFuzzer. It allows you to write powerful fuzz tests more easily than with previously used fuzz targets. You can use it together with GoogleTest, or other unit testing frameworks, allowing you to write fuzz test side by side with regular unit tests, and just as easily.
It is a first-of-its-kind tool that bridges the gap between fuzzing and property-based testing, as it is both:
FuzzTest is for everyone who writes C++ code. (Currently, only C++ is supported.) Fuzz testing is a proven testing technique that has found tens of thousands of bugs. With the FuzzTest framework writing these tests becomes a breeze. Because fuzz tests are more generic, they are more powerful than regular unit tests. They can find tricky edge cases automatically for us, edge cases that most likely we would never think of.
You can write fuzz tests as easily as you write unit tests using GoogleTest for example. Simply use the FUZZ_TEST macro like you would use GoogleTest's TEST macro.
At Google, FuzzTest is widely used and software engineers love it. It has replaced the old style of writing fuzz targets.
To get started, read the Quickstart with Bazel or Quickstart with CMake, then take a look at the Overview and the Codelab.
Once you have a high level understanding about fuzz tests, consider reading the rest of the documentation, including the:
If you have a question or encounter a bug, please file an issue on GitHub.