Improve the decoding of the VEX encoded instructions.

Currently our decoding assumes a constant instruction size (based on the
opcode) but this doesn't really work because these instructions use a
Mod R/M operand (see http://www.c-jump.com/CIS77/CPU/x86/X77_0090_addressing_modes.htm).

This add support for the different sizes that these instructions can have,
as well as some restrictions on which variants are supported.

This has been tested on the latest official build of Chrome.

Review-Url: https://codereview.chromium.org/2841863003
5 files changed