Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
In case start_frame() fails, this potentially invalid frame can still be
output to the caller.
Bug-Id: 672
Bug-Id: debian/741240
Bug-Id: ubuntu/1288206
* commit 'd8e89a37267f276afd404bd062e5112a336d1a36':
h264: reset data partitioning at the beginning of each decode call
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d6a33f5d20b6ef2eae2cbb959b001cb125a564b7)
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2005fddcbb4e18e8f7c34326e40609e4a2d83c31)
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '7e513d85e80d730718695d09fcaf0295ae24699e':
h264: check that execute_decode_slices() is not called too many times
See: 64591f8f86f2dfeac13ee6b4e971d069675ca814
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1147d39bca135df8673031d16989ba486a52172a':
h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Prevents using GetBitContexts with data from previous calls.
Fixes access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
Otherwise the ER code might try to use some already freed references.
Fixes possible access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
In this case we may not have a current frame, while first_field being
set implies we do.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
There is no point in delaying the check and it avoids bugs with a
half-initialized context.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 58312b2472d3a44d7458865c459d59ef2e02bf1a)
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
* commit '8883b5f85bfe35509633bc590d19b6a1b495690e':
h264: Fix a typo from the previous commit
h264: Lower bound check for slice offsets
Add missing header to fix compilation after d2a0654
Prepare for 9.12 RELEASE
configure: Add missing dependency of Snow decoder on videodsp
rpza: limit the number of blocks to the total remaining blocks in the frame
Conflicts:
RELEASE
Merged-by: Michael Niedermayer <michaelni@gmx.at>
f777504f640260337974848c7d5d7a3f064bbb45 changed a - in +
CC: libav-stable@libav.org
(cherry picked from commit d922c5a5fbaf0b6c73bd8c81ae059bc6e406961c)
(cherry picked from commit 3ce77e04c2ca4b9e7fa6b94b51e8d7c5f188da86)
And use the value from the specification.
Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f777504f640260337974848c7d5d7a3f064bbb45)
(cherry picked from commit 5bd083d0216d9ee649039c84999fb61386536ac1)
Conflicts:
libavcodec/h264.c
* commit '62ed6da016b789eee00e0fff517df4a254e12e5d':
h264: check that an IDR NAL only contains I slices
mov: Free an earlier allocated array if allocating a new one
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/release/9:
arm: Don't clobber callee saved registers in scalarproduct
alsdec: check block length
h264/mpegvideo: do not provide pixel formats for hwaccels that are not compiled in
mpeg4video_parser: init mpeg4 static tables.
Conflicts:
libavcodec/mpeg4video_parser.c
libavcodec/mpeg4videodec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This ensures that frames do not get mixed on context reinits
Fixes Ticket2836
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3c9dd93faa9f3c250428dd0548c075583aa07cc3)
This simplifies the code and fixes a deadlock
Fixes Ticket2927
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 29ffeef5e73b8f41ff3a3f2242d356759c66f91f)
Conflicts:
libavcodec/h264.c
* commit '747fbe0c212b81952bb27ec7b99fa709081e2d63':
roqvideodec: fix a potential infinite loop in roqvideo_decode_frame().
mp3dec: Fix VBR bit rate parsing
wmaprodec: return an error, not 0, when the input is too small.
vmdaudio: fix invalid reads when packet size is not a multiple of chunk size
h264: check for luma and chroma bit dept being equal
Prepare for 9.4 Release
Conflicts:
RELEASE
libavcodec/vmdav.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
The decoder assumes a single bit depth for all the planes
while the specification allows different bit depths for luma
and chroma.
Avoid the possible problems described in CVE-2013-2277
CC: libav-stable@libav.org
(cherry picked from commit 4987faee78b9869f8f4646b8dd971d459df218a5)
Conflicts:
libavcodec/h264.c
* qatar/release/9:
update Changelog
h264: set ref_count to 0 for intra slices.
h264: on reference overflow, reset the reference count to 0, not 1.
flvdec: Check the return value of a malloc
Conflicts:
Changelog
libavcodec/h264.c
libavformat/flvdec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
CC:libav-stable@libav.org
(cherry picked from commit 437211ae73ef1ed8285b4fed7620502ea4999e11)
Fixes deadlocks waiting for non-existing references with some fuzzed files.
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Since decode_slice_header() returns before the reference lists are
constructed, there are zero valid references.
CC:libav-stable@libav.org
(cherry picked from commit 668e16a0dd1ff56d4beeff5c658d8a2a08dbfac8)
Conflicts:
libavcodec/h264.c
Some applications do not like that.
Fixes VDA
Reduces noise for VDPAU
Tested-by: Guillaume POIRIER <poirierg@gmail.com>
Tested-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dece584a639c9fd61a72e21800815e8397b3b617)
Conflicts:
libavcodec/h264.c
This prevents faulty increasing of has_b_frames
Should fix Ticket 2062
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c230af9bccc3cadb373f9007ba14fffb6c2acc75)
Fixes out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 695af8eed642ff0104834495652d1ee784a4c14d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Without any correctly decoded slices, there can be no frame.
Fixes out of array reads
Found-by: Rafaël Carré
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 60af6c3138dc501a647bc69b374d5d33d5d86ab5)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The existing checks are insufficient to detect a pixel format
changes in case of some damaged streams.
Fixes inconsistency and later out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11c99c78bafa77f679a1a3ba06ad00984b9a4cae)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If the motion vector is at a subpixel position, we need 3 pixels below
the motion vector's wholepel position available, not 2, since the MC
filter is a sixtap filter for the hpel position, and then a bilin filter
for the qpel position.
This patch fixes highly irreproducible (0.1%) fate failures in frame 2
and 4 of h264-conformance-cama2_vtc_b (e.g. first P-frame, first field,
last line of MB x=40,y=2 and second field and last lines of MBs x=39-40,
y=3). These used pre-loopfilter instead of post-loopfilter data because
the await_progress() waited for one line too little in that field, and
the motion vector of these particular MBs happened to align exactly to a
position where that demonstrates the bug.
CC: libav-stable@libav.org
(cherry picked from commit fb845ffdd335a1efd6dfd43e8adeb530397b348e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
It's been returning an error value since
bad446e251405dc250c3cbee199072e083a1e4b9
Also check for the errors it returns.
(cherry picked from commit ea382767ad2191acbe97e90624059723e15f0e4b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Clobbering these tables will temporarily clobber the template used
as a basis for other threads to start decoding from. If the other
decoding thread updates from the template right at that moment,
subsequent threads will get invalid (or, usually, none at all) mmco
tables. This leads to invalid reference lists and subsequent decode
failures.
Therefore, instead, decode the mmco tables only for the first slice in
a field or frame. For other slices, decode the bits and ensure they
are identical to the mmco tables in the first slice, but don't ever
clobber the context state. This prevents other threads from using a
clobbered/invalid template as starting point for decoding, and thus
fixes decoding in these cases.
This fixes occasional (~1%) failures of h264-conformance-mr1_bt_a with
frame-multithreading enabled.
(cherry picked from commit bad446e251405dc250c3cbee199072e083a1e4b9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
* qatar/master:
cmdutils: update copyright year to 2013
h264: check SPS entries directly to detect pixel format changes
forgotten changelogs for 9_beta2
Conflicts:
Changelog
cmdutils.c
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
