generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Bug fix in probability update savings computation

Browse Source 
Found this bug while tracking down some anomalies in my experiments.
Since vp8_cost_one and vp8_cost_zero return unsigned int, the
bit shift by 8 will be incorrect if the value is negative.
I am cautiously optimistic that this fix will make the prob
updates more correct and somewhat improve results across the board.
But the update probabilities will need to be retuned I think.

Patch 2: Adding more of the same fixes using a macro.

Change-Id: I1a168f040e74e8c67e7225103b1c2af9a611da49
This commit is contained in:
Deb Mukherjee 2012-03-28 15:19:45 -07:00
parent 9f900a1c64
commit 78ecbc98e4
1 changed files with 6 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
vp8/encoder/bitstream.c
View File

@ -44,6 +44,8 @@ extern unsigned int active_section;
int count_mb_seg[4] = { 0, 0, 0, 0 };
#endif
#define vp8_cost_upd ((int)(vp8_cost_one(upd) - vp8_cost_zero(upd)) >> 8)
static void update_mode(
vp8_writer *const w,
int n,
@ -157,8 +159,7 @@ static int prob_update_savings(const unsigned int *ct,
{
const int old_b = vp8_cost_branch(ct, oldp);
const int new_b = vp8_cost_branch(ct, newp);
const int update_b = 8 +
((vp8_cost_one(upd) - vp8_cost_zero(upd)) >> 8);
const int update_b = 8 + vp8_cost_upd;
return old_b - new_b - update_b;
}
@ -1330,8 +1331,7 @@ int vp8_estimate_entropy_savings(VP8_COMP *cpi)
const int old_b = vp8_cost_branch(ct, old);
const int new_b = vp8_cost_branch(ct, newp);
const int update_b = 8 +
((vp8_cost_one(upd) - vp8_cost_zero(upd)) >> 8);
const int update_b = 8 + vp8_cost_upd;
const int s = old_b - new_b - update_b;
@ -1519,8 +1519,7 @@ static void update_coef_probs(VP8_COMP *cpi)
const vp8_prob upd = vp8_coef_update_probs_8x8 [i][j][k][t];
const int old_b = vp8_cost_branch(ct, old);
const int new_b = vp8_cost_branch(ct, newp);
const int update_b = 8 +
((vp8_cost_one(upd) - vp8_cost_zero(upd)) >> 8);
const int update_b = 8 + vp8_cost_upd;
const int s = old_b - new_b - update_b;
const int u = s > 0 ? 1 : 0;
@ -1586,8 +1585,7 @@ static void update_coef_probs(VP8_COMP *cpi)
const vp8_prob upd = vp8_coef_update_probs_8x8 [i][j][k][t];
const int old_b = vp8_cost_branch(ct, old);
const int new_b = vp8_cost_branch(ct, newp);
const int update_b = 8 +
((vp8_cost_one(upd) - vp8_cost_zero(upd)) >> 8);
const int update_b = 8 + vp8_cost_upd;
const int s = old_b - new_b - update_b;
const int u = s > 0 ? 1 : 0;
vp8_write(w, u, upd);