| commit | eedd5f1fb3d4f8625e4d2ccc6ee8848ec61d458e | [log] [tgz] |
|---|---|---|
| author | Takuto Ikuta <tikuta@chromium.org> | Thu Nov 29 04:25:57 2018 |
| committer | Commit Bot <commit-bot@chromium.org> | Thu Nov 29 04:25:57 2018 |
| tree | acfb257b306395f28872f6bf8c23a72a323fe6cf | |
| parent | 73abd5b31452fb0c11d55e407ae3f56edf69f3d8 [diff] |
Reland "[swarming] Introduce conccurentHTTP2 RoundTripper" This is a reland of 2ee324fb6b887003cdafb128f19494204b01b053 I changed to use http.DefaultTransport for non-https request. Testing the binary performance in https://chromium-swarm.appspot.com/task?id=4149b9c897329610&refresh=10 In the job, https://chromium-swarm.appspot.com/task?id=4149cdcee053bf10&refresh=10&show_raw=1 completed in 2018-11-20T12:16:14.583030 and webkit_layout_test step ended in 2018-11-20T12:18:01, so actual output collect took less than 2 minutes. Original change's description: > [swarming] Introduce conccurentHTTP2 RoundTripper > > This change makes large number of concurrent http2 request scalable. > > Golang's http2.Transport blocks requests if the number of concurrent request reached maxConcurrentStreams (default 1000). > ref: https://github.com/golang/go/issues/13774 > > And this becomes limiting factor when we download isolated outputs for webkit_layout_test that requires more than 20000 http requests to isolateserver. > > With this change, collect from a webkit_layout_test results took 16.4 seconds. > $ url=https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8930326331824041104/+/steps/test_pre_run/0/steps/s__trigger__webkit_layout_tests__with_patch_/0/logs/json.output/0 > $ curl $url | jq '.tasks | to_entries | {"tasks": map(.value | {task_id})}' > ~/go.json > $ time swarming collect -verbose -server chromium-swarm.appspot.com -task-output-stdout=json \ > -output-dir tmp -requests-json ~/go.json -worker 100 -task-summary-json summary.json > > real 0m16.420s > user 0m36.004s > sys 0m11.293s > > > Without this change, it took 44.5 seconds. > $ url=https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8930326331824041104/+/steps/test_pre_run/0/steps/s__trigger__webkit_layout_tests__with_patch_/0/logs/json.output/0 > $ curl $url | jq '.tasks | to_entries | {"tasks": map(.value | {task_id})}' > ~/go.json > $ time swarming collect -verbose -server chromium-swarm.appspot.com -task-output-stdout=json \ > -output-dir tmp -requests-json ~/go.json -worker 100 -task-summary-json summary.json > > real 0m44.514s > user 1m38.394s > sys 0m47.534s > > Bug: 868878 > Change-Id: Icc874a120526146f25c38b7f2f5040c9b09542c0 > Reviewed-on: https://chromium-review.googlesource.com/c/1337186 > Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> > Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Bug: 868878 Change-Id: I8970bfa795b4491f6ab642a0fd91536985cd9a8d Reviewed-on: https://chromium-review.googlesource.com/c/1343912 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
LUCI Go code is meant to be worked on from an Chromium infra.git checkout, which enforces packages versions and Go toolchain version. First get fetch via depot_tools.git then run:
fetch infra cd infra/go eval `./env.py` cd src/go.chromium.org/luci
Contributing uses the same flow as Chromium contributions.