vp8: fix some implicit signed -> unsigned conv warnings

and vice-versa mostly when dealing with bitmasks

w/clang-11 -fsanitize=undefined

Change-Id: I6d8f676bf87679ba1dad9cb7f55eea172103d9d3
6 files changed