This fixes returning pictures with corrupted data pointers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes out of array writes after resolution changes
No FFmpeg releases are known to be affected by this
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'eee2000b41234ae9465c314e18bfec1700181f32':
mpeg12: Move some ff_mpeg1_* function declarations to a more suitable place
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Also move the declaration to internal.h, and add restrict qualifiers
to the declaration (as in the implementation).
Signed-off-by: Martin Storsjö <martin@martin.st>
* commit '37045e422903695e610cca6ecb753df643ab9380':
mpegvideo: drop vismv code
Conflicts:
libavcodec/mpegvideo.c
vismv code works fine, and thus only the unused fields are droped
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Not all hwaccels implement all codecs, so using one single list for
multiple such codecs means some codecs will be represented in the list,
even though they don't actually handle that codec. Copying specific
lists in each codec fixes that.
Signed-off-by: Martin Storsjö <martin@martin.st>
Not all hwaccels implement all codecs, so using one single list for
multiple such codecs means some codecs will be represented in the list,
even though they don't actually handle that codec. Copying specific
lists in each codec fixes that.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.
The nontrivial parts are:
1) extracting a simplified version of the frame management code from
mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
its own more complex system already and those were set only to appease
the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
for dxva, the draw_horiz_band() call is moved from
ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
because it's now different for h264 and MpegEncContext-based
decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
added some very simplistic frame management instead and dropped the
use of ff_h264_frame_start(). Because of this I also had to move some
initialization code to svq3.
Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Always evaluate to NULL when the source Picture is not located in the
MpegEncContext.picture array. That will only happen for
next/last_picture_ptr when updating the thread context during h264 frame
threaded decoding, where they will point to elements of ref_list. Since
ref_list is not copied during updating the context and is invalid until
it is constructed for the current slice, there is no point in doing
anything complicated with next/last_picture_ptr, as they will get
updated when the ref_list is filled.
REBASE_PICTURE (more specifically, this half of it) takes a Picture
pointer that points into one larger struct, finds the offset of
that Picture within the struct and finds the corresponding field
within another instance of a similar struct.
The pointer difference "pic - (Picture*)old_ctx" is a value given
in sizeof(Picture) units, and when applied back on
(Picture*)new_ctx gets multiplied back with sizeof(Picture). Many
compilers seem to optimize out this division/multiplication, but
not all do.
GCC 4.2 on OS X doesn't seem to remove the division/multiplication,
therefore the new pointer didn't turn out to point to exactly
the right place in the new struct since it only had sizeof(Picture)
granularity (and the Picture is not aligned on a sizeof(Picture)
boundary within the encompassing struct). This bug has been present
before 47318953d as well - with H264, pointers to h->ref_list[0][0]
pointed to 88 bytes before h->ref_list[0][0] after the rebase. After
shrinking Picture, the difference ended up even larger, making
writes via such a Picture pointer overwrite other fields at random
in H264Context, ending up in crashes later.
This fixes H264 multithreaded decoding on OS X with GCC 4.2.
Signed-off-by: Martin Storsjö <martin@martin.st>
* commit 'f81c37e40fe3236d54da12aef9cdba48ba70ec31':
vf_delogo: fix an uninitialized read.
h264: remove obsolete comment.
mpegvideo: remove some unused variables from Picture.
utvideoenc/v410enc: do not set AVFrame.reference.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Since resolution change support this also was exploitable, which is
how it was found.
Fixes read after free and out of array reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
* commit 'f1d8763a02b5fce9a7d9789e049d74a45b15e1e8':
mpegvideo: allocate scratch buffers after linesize is known
Conflicts:
libavcodec/mpegvideo.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.
Fixes fuzzed sample nasa-8s2.ts_s244342.
