transport: Pool read buffers used by the HTTP/2 framer (#9032) ## Problem The HTTP/2 framer in gRPC uses a `bufio.Reader` with a 32KB buffer by default. When there are a large number of transports, these buffers consume significant memory, even when the transport is idle. ## Solution #8964 added a `ReadyReader` interface that allows non-memory-pinning reads. This PR replaces the standard `bufio.Reader` with a custom `io.Reader` implementation that uses pooled buffers and releases the buffer once all data is consumed. To defer the re-allocation of the read buffer, the reader calls `ReadOnReady` on the underlying `io.Reader`. For this to work, the underlying `io.Reader` must implement either the `ReadyReader` interface or `syscall.RawConn`. If neither condition is met, the framer gracefully falls back to using the regular `bufio.Reader`. Additional Changes: * The ALTS connection has been refactored to implement the `ReadyReader` interface. * The write buffer pools used by the framer are updated to use the `mem.BufferPool` interface, allowing the pools to be shared across both read and write operations. * Use `syscall.Read` instead of `unix.Read` to avoid triggering the race detector, see comment for details. * Add environment variable protection for the changes to allow fast rollback. ## Benchmarks In a [real-world benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp), where a GCS directpath client downloads a file in a loop, the average "in use" memory falls from 28.3MB to 21.3MB (-24%). Local Benchmarks show no significant difference ``` ❯ go run benchmark/benchresult/main.go streaming-before streaming-after streaming-networkMode_Local-bufConn_false-keepalive_false-benchTime_2m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentCa lls_120-reqSize_1024B-respSize_1024B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteBuffer Size_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-sharedWrite Buffer_true Title Before After Percentage TotalOps 29981273 29966908 -0.05% SendOps 0 0 NaN% RecvOps 0 0 NaN% Bytes/op 4971.06 4971.41 0.00% Allocs/op 19.79 19.79 0.00% ReqT/op 2046721570.13 2045740919.47 -0.05% RespT/op 2046721570.13 2045740919.47 -0.05% 50th-Lat 461.523µs 460.906µs -0.13% 90th-Lat 654.435µs 655.327µs 0.14% 99th-Lat 1.225856ms 1.240984ms 1.23% Avg-Lat 478.845µs 479.553µs 0.15% GoVersion go1.25.0 go1.25.0 GrpcVersion 1.81.0-dev 1.81.0-dev ``` RELEASE NOTES: * transport: Pool HTTP/2 framer read buffers to reduce idle memory consumption. Currently limited to Linux for ALTS and non-encrypted transports (TCP, Unix). To disable, set `GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report any issues.
The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information see the Go gRPC docs, or jump directly into the quick start.
Simply add the following import to your code, and then go [build|run|test] will automatically fetch the necessary dependencies:
import "google.golang.org/grpc"
Note: If you are trying to access
grpc-gofrom China, see the FAQ below.
The golang.org domain may be blocked from some countries. go get usually produces an error like the following when this happens:
$ go get -u google.golang.org/grpc package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc" (https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
To build Go code, there are several options:
Set up a VPN and access google.golang.org through that.
With Go module support: it is possible to use the replace feature of go mod to create aliases for golang.org packages. In your project's directory:
go mod edit -replace=google.golang.org/grpc=github.com/grpc/grpc-go@latest go mod tidy go mod vendor go build -mod=vendor
Again, this will need to be done for all transitive dependencies hosted on golang.org as well. For details, refer to golang/go issue #28652.
Please update to the latest version of gRPC-Go using go get google.golang.org/grpc.
The default logger is controlled by environment variables. Turn everything on like this:
$ export GRPC_GO_LOG_VERBOSITY_LEVEL=99 $ export GRPC_GO_LOG_SEVERITY_LEVEL=info
"code = Unavailable desc = transport is closing"This error means the connection the RPC is using was closed, and there are many possible reasons, including:
It can be tricky to debug this because the error happens on the client side but the root cause of the connection being closed is on the server side. Turn on logging on both client and server, and see if there are any transport errors.