Protocol Buffer Extras

Motivation

Protocol buffers are used extensively throughout Chromium for data serialization. To keep Chromium's binary size as small as possible, most protobufs are compiled with the LITE_RUNTIME option. This uses the google::protobuf::MessageLite interface, which is significantly smaller than the full google::protobuf::Message interface because it strips out reflection and other descriptors.

While LITE_RUNTIME is great for performance and size, the lack of reflection makes implementing otherwise trivial functionality difficult and tedious. This includes:

• Debug serialization (e.g., printing a message to a log).
• Equality comparison.

Historically, this has led to developers writing boilerplate, error-prone, hand-written code to perform these actions. For example, equality was often implemented by comparing the results of SerializeAsString(), which can produce false negatives for messages containing map fields (as map field ordering is not guaranteed). This manual approach is a maintenance burden, as the serialization or equality logic can easily become outdated when the proto definition changes.

The proto_extras library was created to solve these problems. It is a code generator that automatically creates this missing functionality for MessageLite protos, providing robust, efficient, and easy-to-maintain helpers for serialization and testing.

Features

• Serialization of a proto message to base::Value::Dict
• operator<< stream support for printing a proto message.
• operator== and operator!= equality support for proto messages.
• gmock matchers for testing proto messages.

Usage

proto_extras

The proto_extras template is used to generate the serialization, stream operator, and equality support. It is used in a BUILD.gn file as follows:

import("//components/proto_extras/proto_extras.gni")

proto_extras("my_proto_extras") {
  sources = [ "my_proto.proto" ]
  deps = [ ":my_proto_cc_proto" ] # Must be a proto_library
}

By default, all functionality is generated. To disable functionality, the following properties can be set:

• omit_to_value_serialization: Disables serialization to base::Value::Dict.
• omit_stream_operators: Disables operator<< stream support.
• omit_equality: Disables operator== and operator!= equality support.

The generated files can be included in C++ as follows:

• Serialization: #include "path/to/my_proto.to_value.h"
• Stream operator: #include "path/to/my_proto.ostream.h"
• Equality: #include "path/to/my_proto.equal.h"

proto_test_extras

The proto_test_extras template is used to generate gmock matchers for proto messages. It is used in a BUILD.gn file as follows:

import("//components/proto_extras/proto_extras.gni")

proto_test_extras("my_proto_test_extras") {
  testonly = true
  sources = [ "my_proto.proto" ]
  deps = [ ":my_proto_cc_proto" ] # Must be a proto_library
  extras_deps = [ ":my_proto_extras" ]
}

This will generate a <name>.test.h file that can be included in test files.

Generated Code Examples

base::Value::Dict Serialization

Given the following proto:

message TestMessage {
  optional string name = 1;
  optional int32 id = 2;
}

The generated to_value.h header will contain:

// Generated by the proto_extras plugin. DO NOT EDIT!
// source: test.proto

#ifndef TEST_TO_VALUE_H_
#define TEST_TO_VALUE_H_

#include <optional>
#include <string_view>

namespace base {
class DictValue;
}  // namespace base

namespace my_package::proto {
class TestMessage;
}  // namespace my_package::proto

namespace my_package::proto {
base::DictValue Serialize(const TestMessage& message);
void MaybeSerialize(const std::optional<TestMessage>& opt_message,
                    std::string_view output_dictionary_field_name,
                    base::DictValue& output_dictionary);
}  // namespace my_package::proto

#endif  // TEST_TO_VALUE_H_

And can be used as follows:

#include "path/to/test.to_value.h"

my_package::proto::TestMessage message;
message.set_name("test");
message.set_id(123);

// The `Serialize` function is in the same namespace as the message.
base::Value::Dict dict = my_package::proto::Serialize(message);
// dict is now: {"name": "test", "id": 123}

The serialization handles all field types, including repeated fields, maps, and oneofs.

Stream Operator

The generated ostream.h header will contain:

// Generated by the proto_extras plugin. DO NOT EDIT!
// source: test.proto

#ifndef TEST_OSTREAM_H_
#define TEST_OSTREAM_H_

#include <iosfwd>

namespace my_package::proto {
class TestMessage;
}  // namespace my_package::proto

namespace my_package::proto {
std::ostream& operator<<(std::ostream& out, const TestMessage& message);
}  // namespace my_package::proto

#endif  // TEST_OSTREAM_H_

And can be used as follows:

#include "path/to/test.ostream.h"

my_package::proto::TestMessage message;
message.set_name("test");
message.set_id(123);

// The stream operator is in the same namespace as the message.
std::cout << message;
// This will print the same JSON representation as `base::Value::Dict`

Equality Operator

The generated equal.h header will contain:

// Generated by the proto_extras plugin. DO NOT EDIT!
// source: test.proto

#ifndef TEST_EQUAL_H_
#define TEST_EQUAL_H_

namespace my_package::proto {
class TestMessage;
}  // namespace my_package::proto

namespace my_package::proto {
bool operator==(const TestMessage& lhs, const TestMessage& rhs);
}  // namespace my_package::proto

#endif  // TEST_EQUAL_H_

This can be used to compare two proto messages for equality. The generated operator== handles all field types, including oneofs and maps, and recursively calls operator== for nested messages.

Gmock Matchers

The generated test.h header will contain a matcher called Equals<MessageName> for each message. For the TestMessage proto, the matcher is EqualsTestMessage.

Example usage:

#include "path/to/test.test.h"
#include "testing/gmock/include/gmock/gmock.h"

using ::testing::Not;

TEST(MyTest, Test) {
  my_package::proto::TestMessage message1;
  message1.set_name("test");
  message1.set_id(123);

  my_package::proto::TestMessage message2;
  message2.set_name("test");
  message2.set_id(123);

  my_package::proto::TestMessage message3;
  message3.set_name("different");
  message3.set_id(456);

  // The matcher is in the same namespace as the message.
  EXPECT_THAT(message1, my_package::proto::EqualsTestMessage(message2));
  EXPECT_THAT(message1, Not(my_package::proto::EqualsTestMessage(message3)));
}

The generated .test.h file also contains a PrintTo implementation for each message, which allows gtest to pretty-print the message on test failures. The matcher handles all field types, including repeated fields, maps, and oneofs.

Forcing full protobuf library support

For cases where the message uses the full google::protobuf::Message type, the protobuf_full_support option can be used in the proto_extras GN target to ensure the generated code with the full protobuf library. Due to android build complications, this also requires the use_fuzzing_engine_with_lpm build flag to be set. This option is relevant for base::Value serialization and equality.

AI Agent Guide

This section contains information for AI agents that are tasked with working on the proto_extras library.

Code Generation

The proto_extras library uses two main files for code generation:

• proto_extras_plugin.cc: This file is responsible for generating the to_value, ostream, and equal files. It is a protobuf compiler plugin that is invoked by the proto_library GN template.
• proto_test_extras_plugin.cc: This file is responsible for generating the .test.h and .test.cc files, which contain gmock matchers and PrintTo implementations. It is also a protobuf compiler plugin.

Support Library

The generated code relies on a support library for common functionality:

• proto_extras_lib.h: This file contains common helper functions used by the generated code, such as ToNumericTypeForValue for converting between numeric types and SerializeUnknownFields for serializing unknown fields in MessageLite protos.
• protobuf_full_support.h/.cc: These files provide an alternative implementation of some of the helper functions in proto_extras_lib.h for messages that use the full google::protobuf::Message type. This is necessary for fuzzing targets and other cases where the full protobuf library is used. It also provides MessageDifferencerEquals to compare two full protobuf messages.
• proto_matchers.h: This file contains helper gmock matchers that are used by the generated test code.

Modifying the Library

When tasked with modifying the proto_extras library, it is important to understand which file is responsible for the desired functionality.

• For changes to the base::Value::Dict serialization, stream operator, or equality operators, the relevant file is proto_extras_plugin.cc.
• For changes to the gmock matchers, the relevant files are proto_test_extras_plugin.cc and proto_matchers.h.
• For changes to the helper functions used by the generated code, the relevant file is proto_extras_lib.h or protobuf_full_support.h/.cc.

When adding new functionality, it is recommended to follow the existing pattern of creating a new generation function in the appropriate plugin file and adding a new command-line option to enable it.