Overview

Go tests in this directory analyze the usage of Etcd API based on collected traces from a Kubernetes cluster. They output information on:

1. Number of calls per gRPC method used by Kubernetes
2. Key pattern
3. Arguments provided to KV, Watch and Lease methods

This information can be used to track the coverage of k8s-etcd contract.

Manual test execution

At first we will manually set up the cluster, run e2e tests, download traces and then execute the test.

1. Customize and set the environment variables used by the code snippets below:

# Used for patches, building kind nodes, and running e2e tests.
export KUBERNETES_REPO="$(go env GOPATH)/src/k8s.io/kubernetes"
# Used when creating kind cluster and running e2e tests.
export KUBECONFIG="${KUBERNETES_REPO}/kind-with-tracing-config"

1. Set up KIND cluster, exercise Kubernetes API, export traces to Jaeger and then download them:

   make k8s-coverage
   

2. Run Go test

   go test -v -timeout 60s go.etcd.io/etcd/tests/v3/robustness/coverage
   

3. Clean up the environment

   kind delete cluster --name kind-with-external-etcd
   docker network rm kind-with-external-etcd
   

Manual trace collection from robustness tests

1. Run Jaeger container:

   docker run --rm --name jaeger \
     -p 16686:16686 \
     -p 4317:4317 \
     jaegertracing/jaeger:2.6.0 --set=extensions.jaeger_storage.backends.some_storage.memory.max_traces=20000000
   

2. Run robustness tests. For example:

   env \
     TRACING_SERVER_ADDR=localhost:4317 \
     GO_TEST_FLAGS='--timeout 10m --count=1 -v --run "^TestRobustness.*/Kubernetes.*"' \
     make test-robustness
   

3. Download traces and put them into tests/robustness/coverage/testdata directory in Etcd git repository:

   curl -v --get --retry 10 --retry-connrefused -o testdata/demo_traces.json \
     -H "Content-Type: application/json" \
     --data-urlencode "query.start_time_min=$(date --date="5 days ago" -Ins)" \
     --data-urlencode "query.start_time_max=$(date -Ins)" \
     --data-urlencode "query.service_name=etcd" \
     "http://localhost:16686/api/v3/traces"
   

4. Run Go test

   go test -v -timeout 60s go.etcd.io/etcd/tests/v3/robustness/coverage
   

   It will show the coverage of Kubernetes-Etcd surface by robustness tests.:w

Automated test execution

Work on improving these tests is tracked in #20182 and periodic runs in #20642.