| tag | 6d503418bf2a4d045c3da312b7cef3cc2c013a41 | |
|---|---|---|
| tagger | bufdev <bufdev-github@buf.build> | Mon May 11 14:25:42 2020 |
| object | a88f7f6c09ebc93f52d1af2cbe7c84e3e2ee48da |
v0.12.0
| commit | a88f7f6c09ebc93f52d1af2cbe7c84e3e2ee48da | [log] [tgz] |
|---|---|---|
| author | bufdev <bufdev-github@buf.build> | Mon May 11 14:25:42 2020 |
| committer | bufdev <bufdev-github@buf.build> | Mon May 11 14:25:42 2020 |
| tree | 790dbee935d891f58ea898746ee7e4c37e93be23 | |
| parent | 0565ad934a122b396f99250caa24c6b371f0d705 [diff] |
Update to v0.12.0
All documentation is hosted at https://buf.build. Please head over there for more details.
Buf‘s goal is for Protobuf to not only be a good choice on the technical merits, but to be so easy to use that the decision is trivial. Your organization should not have to reinvent the wheel to use Protobuf efficiently and effectively. Stop worrying about your Protobuf management strategy getting out of control. We’ll handle that for you, so you can worry about what matters.
However, we recommend you read the below introduction first!
“Protobuf is so much harder to use than JSON, why should I use Protobuf?”
Buf aims to eventually reverse this sentence. Our goal is for you to say:
“JSON is so much harder to use than Protobuf, why should I use JSON?”
Using an IDL such as Protocol Buffers (“Protobuf”) provides numerous benefits over JSON:
Protobuf is the most stable, widely-adopted IDL in the software industry today. While there are many pros and cons to Protobuf versus other IDLs such as Thrift, FlatBuffers, Avro, and Cap'n Proto, Protobuf provides most companies the most stable platform to build on, along with the largest ecosystem of languages and libraries available.
If you‘ve found us today, we’ll assume you‘re already relatively convinced of these statements. We’ll add a reference document for those new to Protobuf in the future.
If Protobuf is so great, the question is: why do so many companies still choose JSON as their data format in 2020? Usually, the answer comes down to difficulty in adoption:
protoc and associated plugins in a maintainable manner. Organizations end up struggling with distribution of Protobuf files and stubs, even if they use a build system such as Bazel - exposing APIs to external customers remains problematic. It ends up being more trouble than it's worth to expose APIs via Protobuf than via JSON/REST.Done right, adopting Protobuf to represent your structured data and APIs can quite literally produce one of the largest efficiency gains your engineering organization can have. Much of the software we write today can be generated, and many daily software development tasks we perform can be automated away.
In time, Buf aims to solve all this and more. However, there is a long way between that world and the one we have now.
Phase 1 is to solve the API Structure and Backwards Compatibility problems: let's help you maintain consistent Protobuf APIs that maintain compatibility.
We have released the Buf CLI tool, and associated protoc plugins, into public beta.
Buf currently contains:
Buf is designed to be extremely simple to use, while providing functionality for advanced use cases. Features of Buf's include:
Automatic file discovery. By default, Buf will build your .proto files by walking your file tree and building them per your build configuration. This means you no longer need to manually specify your --proto_paths and files every time you run the tool. However, Buf does allow manual file specification through command-line flags if you want no file discovery to occur, for example in Bazel setups.
Selectable configuration of the exact lint and breaking change configuration you want. While we recommend using the defaults, Buf allows you to easily understand and select the exact set of lint and breaking change checkers your organization needs.
Buf provides 40 available lint checkers and 54 available breaking checkers to cover most needs. We believe our breaking change detection truly covers every scenario for your APIs.
Selectable error output. By default, Buf outputs file:line:col:message information for every lint error and every breaking change, with the file path carefully outputted to match the input location, including if absolute paths are used, and for breaking change detection, including if types move across files. JSON output that includes the end line and end column of the lint error is also available, and JUnit output is coming soon.
Editor integration. The default error output is easily parseable by any editor, making the feedback loop for issues very short. Currently, we only provide Vim integration for linting but will extend this in the future to include other editors such as Emacs, VS Code, and Intellij IDEs.
Check anything from anywhere. Buf allows you to not only check a Protobuf schema stored locally as .proto files, but allows you to check many different Inputs:
.proto files, both local and remote..proto files, both local and remote.protoc, from both local and remote (http/https) locations.Speed. Buf‘s internal Protobuf compiler utilizes all available cores to compile your Protobuf schema, while still maintaining deterministic output. Additionally files are copied into memory before processing. As an unscientific example, Buf can compile all 2,311 .proto files in googleapis in about 0.8s on a four-core machine, as opposed to about 4.3s for protoc on the same machine. While both are very fast, this allows for instantaneous feedback, which is especially useful with editor integration. Buf’s speed is directly proportional to the input size, so checking a single file only takes a few milliseconds.
Use protoc as your compiler. Existing lint and breaking change detection tools produce an internal representation of your Protobuf schema in one of two ways:
protoc itself and parsing the result, which not only requires specific management of protoc in relation to the lint/breaking change detection tool, but can be cumbersome and error-prone itself, especially if the tool parses error output from protoc.Buf tackles this issue by using FileDescriptorSets internally for all operations, and allowing these FileDescriptorSets to be produced in one of two ways:
protoc output as buf input, thereby bypassing any compiling or parsing on the part of buf entirely, and instead using protoc, the gold standard of Protobuf compilation.See the Image and compiler documentation for more details.
In short, we don't expect you to natively trust the internal compiler is actually equivalent to protoc - we would want to verify this claim ourselves. There are also cases (such as Bazel setups) where you may already have infrastructure around calling protoc, and may want to just use artifacts from protoc as input to buf.
Use buf as a protoc plugin instead of a standalone tool. You can go a step further and use Buf's lint and breaking change functionality as a protoc plugin with the provided protoc-gen-buf-check-lint and protoc-gen-buf-check-breaking plugins.
Docker image. You can use Buf from the provided Docker image bufbuild/buf as well.
We'll work through the beta over the next couple months, and hope to hit a stable v1.0 by early 2020.
Simultaneously, we'll be working on our next goal, which is to solve the bigger problem: now that you have consistent and compatible APIs, how do you, and your customers, consume them?
Phase 2 is the Buf Schema Registry, Our goal is for the Buf Schema Registry to be a publicly hosted and on-prem service that will receive Images built by buf, and let you produce and consume various generated artifacts such as:
protoc and associated language plugins..proto files alongside Bazel build rules.There's a lot we are planning with the Buf Schema Registry. For a quick overview, see our roadmap.
Phase 3+ will build on top of buf and the Buf Schema Registry, and we'll announce these at at a later date.
To install Buf, proceed to installation. This includes links to an example repository for Travis CI and GitHub Actions integration.
Next, we recommend completing the tour. This tour should only take about 10 minutes, and will give you an overview of most of the existing functionality of Buf.
After completing the tour, check out the remainder of the docs for your specific areas of interest. We‘ve aimed to provide as much documentation as we can for the various components of Buf to give you a full understanding of Buf’s surface area.
Finally, follow the project on GitHub, and contact us if you'd like to get involved.