Use saturated addition in SSSE3 of 32x32 quant

The 32x32 forward transform can potentially reach peak coefficient
value close to 32700, while the rounding factor can go upto 610.
This could cause overflow issue in the SSSE3 implementation of 32x32
quantization process.

This commit resolves this issue by replacing the addition operations
with saturated addition operations in 32x32 block quantization.

Change-Id: Id6b98996458e16c5b6241338ca113c332bef6e70
This commit is contained in:
Jingning Han
2013-09-04 12:03:28 -07:00
parent 1cf2272347
commit 458c2833c0
3 changed files with 12 additions and 2 deletions

View File

@@ -135,6 +135,7 @@ void vp9_quantize_b_32x32_c(int16_t *coeff_ptr, intptr_t n_coeffs,
if (x >= zbin) {
x += ROUND_POWER_OF_TWO(round_ptr[rc != 0], 1);
x = clamp(x, INT16_MIN, INT16_MAX);
y = (((int)(((int)(x * quant_ptr[rc != 0]) >> 16) + x)) *
quant_shift_ptr[rc != 0]) >> 15; // quantize (x)