vp8: fix quantizer clamping

the quantizer is transmitted as 7-bits + sign so needs to be clamped in
the delta + absolute case.

BUG=https://bugzilla.mozilla.org/show_bug.cgi?id=1224361

Change-Id: I9115f5d1d5cf7e0a1d149d79486d9d17de9b9639
This commit is contained in:
James Zern 2015-12-03 16:16:28 -08:00
parent b38a7cd169
commit ff3674a15e

View File

@ -73,10 +73,9 @@ void vp8_mb_init_dequantizer(VP8D_COMP *pbi, MACROBLOCKD *xd)
/* Delta Value */
else
{
QIndex = pc->base_qindex + xd->segment_feature_data[MB_LVL_ALT_Q][mbmi->segment_id];
QIndex = (QIndex >= 0) ? ((QIndex <= MAXQ) ? QIndex : MAXQ) : 0; /* Clamp to valid range */
}
QIndex = (QIndex >= 0) ? ((QIndex <= MAXQ) ? QIndex : MAXQ) : 0; /* Clamp to valid range */
}
else
QIndex = pc->base_qindex;