vp9 encoder: fix some integer overflows

cap bitrate to 1000Mbps, change bitsaving budget to int64_t

this make test coverage for 2048x2048 - same as for vp8

Bug: webm:1749
Fixed: webm:1749
Change-Id: Ic58d73cb7529b0826d1f501ad09af8e80f706a6e
3 files changed