Simplify rdmult computation

Recognizing that max dc_quant used in rdmult computation is 21387 and
21387 * 21387 * 88 / 24 is still within the range of int32_t, this
commit simplifies the computation with minor cleanups.

Change-Id: I2ac7e8315d103c0bb39b70c312c87c0fda47b4f9
4 files changed