generic-library/fdk-aac
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Encoder quantizer fix

Browse Source 
AAC Encoder: Saturate quantizer shift value to prevent
  undefined behaviour.

In very rare cases the shift value 'totalShift = (16-4)-(3*(totalShift>>2))'
 can be greater than accu data width. If you apply a shift with more then
 31 bit the result depends on the architecture and is not defined in C.
For certain platforms zeros are shifted in. That would be our desired behaviour.
On other platforms the shift will be applied as modulo. For example >>34
 would be applied as >>2. To prevent this discrepancy the shift value
 is limited/saturated to DFRACT_BITS-1. 'accu >>= fixMin(totalShift,DFRACT_BITS-1)'.

Bug 9428126

Change-Id: I27177654c4dc22cf899bc35dad9cdd040dccb02d
This commit is contained in:
Jean-Michel Trivi 2013-08-08 17:38:33 -07:00
parent 577fcbb570
commit ed247dfa54
2 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libAACenc/src/aacenc_lib.cpp
View File

@ -98,7 +98,7 @@ amm-info@iis.fraunhofer.de
/* Encoder library info */
#define AACENCODER_LIB_VL0 3
#define AACENCODER_LIB_VL1 4
#define AACENCODER_LIB_VL2 1
#define AACENCODER_LIB_VL2 2
#define AACENCODER_LIB_TITLE "AAC Encoder"
#define AACENCODER_LIB_BUILD_DATE __DATE__
#define AACENCODER_LIB_BUILD_TIME __TIME__

4
libAACenc/src/quantize.cpp
View File

@ -127,7 +127,7 @@ static void FDKaacEnc_quantizeLines(INT gain,
accu = fMultDiv2(FDKaacEnc_mTab_3_4[tabIndex],FDKaacEnc_quantTableE[totalShift&3]);
totalShift = (16-4)-(3*(totalShift>>2));
FDK_ASSERT(totalShift >=0); /* MAX_QUANT_VIOLATION */
accu>>=totalShift;
accu >>= fixMin(totalShift,DFRACT_BITS-1);
quaSpectrum[line] = (SHORT)(-((LONG)(k + accu) >> (DFRACT_BITS-1-16)));
}
else if(accu > FL2FXCONST_DBL(0.0f))
@ -140,7 +140,7 @@ static void FDKaacEnc_quantizeLines(INT gain,
accu = fMultDiv2(FDKaacEnc_mTab_3_4[tabIndex],FDKaacEnc_quantTableE[totalShift&3]);
totalShift = (16-4)-(3*(totalShift>>2));
FDK_ASSERT(totalShift >=0); /* MAX_QUANT_VIOLATION */
accu>>=totalShift;
accu >>= fixMin(totalShift,DFRACT_BITS-1);
quaSpectrum[line] = (SHORT)((LONG)(k + accu) >> (DFRACT_BITS-1-16));
}
else