generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Align internal mfqe framebuffer dimensions

Browse Source 
MFQE postproc crashed with stream dimensions not a multiple of 16.
The buffer was memset unconditionally, so if the buffer allocation
fails we end up trying to write to NULL.

This patch traps an allocation failure with vpx_internal_error(),
and aligns the buffer dimensions to what vp8_yv12_alloc_frame_buffer()
expects.

Change-Id: I3915d597cd66886a24f4ef39752751ebe6425066
This commit is contained in:
John Koleszar 2012-02-07 10:26:48 -08:00
parent 45f4b87e8e
commit 417b852967
1 changed files with 10 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
vp8/common/postproc.c
View File

@ -930,10 +930,16 @@ int vp8_post_proc_frame(VP8_COMMON *oci, YV12_BUFFER_CONFIG *dest, vp8_ppflags_t
{ {
if ((flags & VP8D_DEBLOCK) || (flags & VP8D_DEMACROBLOCK)) if ((flags & VP8D_DEBLOCK) || (flags & VP8D_DEMACROBLOCK))
{ {
if (vp8_yv12_alloc_frame_buffer(&oci->post_proc_buffer_int, oci->Width, oci->Height, VP8BORDERINPIXELS) >= 0) int width = (oci->Width + 15) & ~15;
{ int height = (oci->Height + 15) & ~15;
oci->post_proc_buffer_int_used = 1;
} if (vp8_yv12_alloc_frame_buffer(&oci->post_proc_buffer_int,
width, height, VP8BORDERINPIXELS))
vpx_internal_error(&oci->error, VPX_CODEC_MEM_ERROR,
"Failed to allocate MFQE framebuffer");
oci->post_proc_buffer_int_used = 1;
// insure that postproc is set to all 0's so that post proc // insure that postproc is set to all 0's so that post proc
// doesn't pull random data in from edge // doesn't pull random data in from edge
vpx_memset((&oci->post_proc_buffer_int)->buffer_alloc,126,(&oci->post_proc_buffer)->frame_size); vpx_memset((&oci->post_proc_buffer_int)->buffer_alloc,126,(&oci->post_proc_buffer)->frame_size);