Fix flaky test `TestAggregatedClusterSuccess_SwitchBetweenLeafAndAggregate` in aggregate_cluster_test.go. (#9009)
Fixes #8989
This PR addresses the flakiness observed in the
`TestAggregatedClusterSuccess_SwitchBetweenLeafAndAggregate ` test. Test
was using real DNS resolution which introduced non-deterministic delays
and race conditions during testing.
### Changes:
* **Mocked DNS Resolver**: Introduced a `setupDNS` helper function in
`cdsbalancer_test.go` that unregisters the default DNS resolver and
registers a manual resolver for the `dns` scheme. This allows us to
intercept DNS targets and provide mock addresses immediately.
* **Updated Tests**: Updated the following tests in
`aggregate_cluster_test.go` to use the fake DNS resolver:
* `TestAggregatedClusterSuccess_SwitchBetweenLeafAndAggregate`
* `TestAggregateClusterSuccess_ThenUpdateChildClusters`
* `TestAggregateClusterSuccess_ThenChangeRootToEDS`
* **Adjusted Test Timeout**: Increased the `defaultTestTimeout` in
`cdsbalancer_test.go` from `5s` to `10s` to align with the standard test
timeout practices used across the `grpc-go` codebase.
### Testing/Validation:
* Successfully reproduced the flakiness locally by manually injecting a
sleep delay to simulate a slow real DNS resolution update in
`TestAggregatedClusterSuccess_SwitchBetweenLeafAndAggregate`.
* Validated that applying the mock DNS resolver completely resolves the
flakiness.
RELEASE NOTES: N/A
The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information see the Go gRPC docs, or jump directly into the quick start.
Simply add the following import to your code, and then go [build|run|test] will automatically fetch the necessary dependencies:
import "google.golang.org/grpc"
Note: If you are trying to access
grpc-gofrom China, see the FAQ below.
The golang.org domain may be blocked from some countries. go get usually produces an error like the following when this happens:
$ go get -u google.golang.org/grpc package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc" (https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)
To build Go code, there are several options:
Set up a VPN and access google.golang.org through that.
With Go module support: it is possible to use the replace feature of go mod to create aliases for golang.org packages. In your project's directory:
go mod edit -replace=google.golang.org/grpc=github.com/grpc/grpc-go@latest go mod tidy go mod vendor go build -mod=vendor
Again, this will need to be done for all transitive dependencies hosted on golang.org as well. For details, refer to golang/go issue #28652.
Please update to the latest version of gRPC-Go using go get google.golang.org/grpc.
The default logger is controlled by environment variables. Turn everything on like this:
$ export GRPC_GO_LOG_VERBOSITY_LEVEL=99 $ export GRPC_GO_LOG_SEVERITY_LEVEL=info
"code = Unavailable desc = transport is closing"This error means the connection the RPC is using was closed, and there are many possible reasons, including:
It can be tricky to debug this because the error happens on the client side but the root cause of the connection being closed is on the server side. Turn on logging on both client and server, and see if there are any transport errors.