Cache server optimization : Bottle neck is GCE bot type hardware.
Cache server is crashing due to multiple concurrent execution of cacheGShandler. cacheGSHandler is responsible for downloading files from gs to host.During execution cache server isn't storing anything in memory than just the default 32KB buffer during copy.
Limiting the number of concurrent executions to 8 and also reducing the buffer size to 16KB. This would mean more CPU cycles which seems alright as per stats seen while testing.
Adding fix to log output to log.txt instead of stdout.
BUG=b:279805930
TEST=go build ./
TEST=docker run -it -m 20MB --rm --network host --volume ~/chromiumos/src/platform/dev/src/chromiumos/prototytpe/cache/cmd/cacheserver:/creds cache:14 bash -c 'export GOOGLE_APPLICATION_CREDENTIALS=/creds/auth.json && cacheserver --port 8080'
TEST=https://ci.chromium.org/swarming/task/626339f2a9797b10?server=chromeos-swarming.appspot.com
TEST=manual
Change-Id: I415ab75ef5c152123482c3a2d4a2dc2846bbb8cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/dev-util/+/4554356
Auto-Submit: Varun Srivastav <varunsrivastav@google.com>
Commit-Queue: Jaques Clapauch <jaquesc@google.com>
Tested-by: Varun Srivastav <varunsrivastav@google.com>
Commit-Queue: Varun Srivastav <varunsrivastav@google.com>
Reviewed-by: Ming Kong <mingkong@google.com>
Reviewed-by: Jaques Clapauch <jaquesc@google.com>
3 files changed
