alts: Release read buffer when blocked on socket read (#8964) ## Problem Normally, since the Go `net.Conn` interface provides the abstraction of a blocking read to hide the complexity of non-blocking I/O (epoll, kqueue), users need to pass a read buffer to the `net.Conn.Read` call. Even when the TCP socket doesn't have data, the application needs to hold onto the read buffer. This results in the ALTS conn and the gRPC HTTP/2 stack holding on to 32KB read buffers each, even for non-readable transports. ## Solution On Unix platforms, there is a [RawConn](https://pkg.go.dev/syscall#RawConn#Read) interface that exposes a non-blocking mechanism. The idea is that the Go runtime will call a user-registered callback when the socket is readable. gRPC can use this callback to allocate a buffer from the pool and return it once it has passed the plaintext to the HTTP/2 layer. The same RawConn interface is also available in other OSs, but with slightly different method signatures for the blocking `Read` method. In the future, we can add the same optimization for them and have CI runners to catch regressions. The main abstraction that allows these non-memory-pinning reads is the following interface: ```go // ReadyReader is an optional interface that can be implemented by net.Conn // implementations to enable gRPC to perform non-memory-pinning reads. type ReadyReader interface { // ReadOnReady waits for data to arrive, fetches a buffer, and performs a // read. It returns a pointer to the buffer so you can return it to the pool // later. ReadOnReady(bufSize int, pool mem.BufferPool) (*[]byte, int, error) } ``` In this PR, an implementation is provided that wraps a `RawConn`. This allows the ALTS conn to perform efficient reads. In a future PR, the following changes will enable getting rid of the bufio.Reader in the HTTP/2 layer: 1. `ReadyReader` will be implemented by the ALTS conn. 2. gRPC will implement its own buffered reader that releases the buffer when it's empty. 3. The buffered reader will call `ReadOnReady()` instead of `Read()` on the underlying conn, if supported, to delay the re-allocation of the buffer. ## Benchmarks The following micro-benchmarks show no regression in QPS (LargeMessage test) and a significant reduction in memory usage while performing reads (ReadMemoryUsage). There is an increase in 2 allocs in conn construction due to the use of pointer fields for the `ReadyReader` and read buffer handle, but these happen only when creating a subchannel, not per-RPC. There is an increase in sec/op for the `WriteMemoryUsage` test, but these tests are not meant to measure conn construction time, only the memory effeciency. ``` goos: linux goarch: amd64 pkg: google.golang.org/grpc/credentials/alts/internal/conn cpu: Intel(R) Xeon(R) CPU @ 2.60GHz │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ LargeMessage-48 77.54m ± 2% 76.51m ± 1% ~ (p=0.202 n=15) WriteMemoryUsage-48 6.816µ ± 1% 7.519µ ± 1% +10.31% (p=0.000 n=15) ReadMemoryUsage-48 9.754µ ± 1% 7.021µ ± 1% -28.02% (p=0.000 n=15) geomean 172.7µ 159.3µ -7.81% │ old.txt │ new.txt │ │ B/op │ B/op vs base │ LargeMessage-48 4.578Mi ± 0% 4.579Mi ± 7% ~ (p=0.373 n=15) WriteMemoryUsage-48 41.60Ki ± 0% 41.77Ki ± 0% +0.39% (p=0.000 n=15) ReadMemoryUsage-48 83.06Ki ± 0% 43.25Ki ± 0% -47.94% (p=0.000 n=15) geomean 253.0Ki 203.8Ki -19.44% │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ LargeMessage-48 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=15) ¹ WriteMemoryUsage-48 5.000 ± 0% 7.000 ± 0% +40.00% (p=0.000 n=15) ReadMemoryUsage-48 16.00 ± 0% 18.00 ± 0% +12.50% (p=0.000 n=15) geomean 5.429 6.316 +16.35% ``` In a [real-world benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp), where a GCS directpath client downloads a file in a loop, the average "in use" memory falls from ~35MB to ~20MB. RELEASE NOTES: * alts: pool read buffers to lower memory utilization when sockets are unreadable.
The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information see the Go gRPC docs, or jump directly into the quick start.
Simply add the following import to your code, and then go [build|run|test] will automatically fetch the necessary dependencies:
import "google.golang.org/grpc"
Note: If you are trying to access
grpc-gofrom China, see the FAQ below.
The golang.org domain may be blocked from some countries. go get usually produces an error like the following when this happens:
$ go get -u google.golang.org/grpc package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc" (https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
To build Go code, there are several options:
Set up a VPN and access google.golang.org through that.
With Go module support: it is possible to use the replace feature of go mod to create aliases for golang.org packages. In your project's directory:
go mod edit -replace=google.golang.org/grpc=github.com/grpc/grpc-go@latest go mod tidy go mod vendor go build -mod=vendor
Again, this will need to be done for all transitive dependencies hosted on golang.org as well. For details, refer to golang/go issue #28652.
Please update to the latest version of gRPC-Go using go get google.golang.org/grpc.
The default logger is controlled by environment variables. Turn everything on like this:
$ export GRPC_GO_LOG_VERBOSITY_LEVEL=99 $ export GRPC_GO_LOG_SEVERITY_LEVEL=info
"code = Unavailable desc = transport is closing"This error means the connection the RPC is using was closed, and there are many possible reasons, including:
It can be tricky to debug this because the error happens on the client side but the root cause of the connection being closed is on the server side. Turn on logging on both client and server, and see if there are any transport errors.