generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Change to use proper type in vp{9,10}_token_state

Browse Source 
"qc" in vp{9,10}_token_state is used to save quantized coefficients, this
commit changes the type from short to tran_low_t to properly reflect
the value range for highbitdepth build.

This fixes an out-of-range bug when optimize_b is used in highbitdepth
build.

Change-Id: Ibf330879e6ac6ae8f099e085caa9d3d9a889fde8
This commit is contained in:
Yaowu Xu 2016-05-04 12:12:02 -07:00
parent 3c2231807f
commit 81eb71f00c
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vp10/encoder/encodemb.c
View File

@ -57,7 +57,7 @@ typedef struct vp10_token_state {
int error; int error;
int next; int next;
int16_t token; int16_t token;
short qc; tran_low_t qc;
} vp10_token_state; } vp10_token_state;
// TODO(jimbankoski): experiment to find optimal RD numbers. // TODO(jimbankoski): experiment to find optimal RD numbers.

2
vp9/encoder/vp9_encodemb.c
View File

@ -59,7 +59,7 @@ typedef struct vp9_token_state {
int error; int error;
int next; int next;
int16_t token; int16_t token;
int16_t qc; tran_low_t qc;
} vp9_token_state; } vp9_token_state;
static const int plane_rd_mult[REF_TYPES][PLANE_TYPES] ={ {10, 6}, {8, 7}, }; static const int plane_rd_mult[REF_TYPES][PLANE_TYPES] ={ {10, 6}, {8, 7}, };