generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Fix int64_t to unsigned int conversion warnings

Browse Source 
Use unsigned int type to store the sse in the pixel domain. The
precision is sufficient to handle sse of block size up to 64x64.
The transform domain version however needs int64_t, since there is
a transfer gain applied in the forward transformation that might
cause unsigned int overflow.

Change-Id: Ifef97c38597e426262290f35341fbb093cf0a079
This commit is contained in:
Jingning Han 2014-08-28 09:21:58 -07:00 committed by Gerrit Code Review
parent 02e6ecdc4c
commit 6ddf1e152a
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vp9/encoder/vp9_rdopt.c
View File

@ -172,7 +172,7 @@ static void model_rd_for_sb(VP9_COMP *cpi, BLOCK_SIZE bsize,
const int ref = xd->mi[0]->mbmi.ref_frame[0];
unsigned int sse;
unsigned int var = 0;
int64_t sum_sse = 0;
unsigned int sum_sse = 0;
const int shift = 8;
int rate;
int64_t dist;