generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Fix another motion vector out of range bug

Browse Source 
This patch fixed a motion vector out of range bug:
vpxenc: ../libvpx/vp9/encoder/vp9_mcomp.c:69:
 mv_cost: Assertion `mv->col >= -((1 << (11 + 1 + 2)) - 1) &&
 mv->col < ((1 << (11 + 1 + 2)) - 1)' failed.

For blocks that returned without having full-pixel search, the original
MV limits were not restored, which caused the failure. Moved the set
MV limit function down to fix the bug.

Change-Id: Id7d798fc7214e95c6e4846c588f0233fcf1a4223
This commit is contained in:
Yunqing Wang 2016-08-11 17:34:20 -07:00
parent f1e12c1bf3
commit a413dbe594
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
vp9/encoder/vp9_rdopt.c
View File

@ -2347,8 +2347,6 @@ static void single_motion_search(VP9_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bsize,
vp9_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL);
}
vp9_set_mv_search_range(&x->mv_limits, &ref_mv);
// Work out the size of the first step in the mv step search.
// 0 here is maximum length first step. 1 is VPXMAX >> 1 etc.
if (cpi->sf.mv.auto_mv_step_size && cm->show_frame) {
@ -2396,6 +2394,10 @@ static void single_motion_search(VP9_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bsize,
}
}
// Note: MV limits are modified here. Always restore the original values
// after full-pixel motion search.
vp9_set_mv_search_range(&x->mv_limits, &ref_mv);
mvp_full = pred_mv[x->mv_best_ref_index[ref]];
mvp_full.col >>= 3;