Fixes out of array read
Fixes: 1cb91c36c4e55463f14aacb9bdf55b38-asan_heap-oob_106cbce_5617_cov_11212800_h264_mmx_chroma_intra_lf.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
And use the value from the specification.
Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Fixes out of array read
Fixes: caa65cc01655505705129b677189f036-signal_sigsegv_fdcc43_2681_cov_3043376737_PPH422I5_Panasonic_A.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '3a0576702825423abecb32627c530dbc4c0f73bc':
h264: store current_sps_id inside the current sps
Conflicts:
libavcodec/h264.c
libavcodec/h264_ps.c
The current_sps_id is not removed as it used in security related code.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '73e8fab31dc19c4371499e612856accbc00b2820':
h264: print values in case of error
Conflicts:
libavcodec/h264.c
libavcodec/h264_ps.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1f097d168d9cad473dd44010a337c1413a9cd198':
h264: reset data partitioning at the beginning of each decode call
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Prevents using GetBitContexts with data from previous calls.
Fixes access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
This avoids them being cleared before the full initialization finished
Fixes out of array read
Fixes: asan_heap-oob_f0c5e6_7071_cov_1605985132_mov_h264_aac__Demo_FlagOfOurFathers.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Such changes are forbidden in H.264 and lead to race conditions
Fixes out of array read
Fixes: signal_sigsegv_f9796a_1613_cov_3114610371_FM1_BT_B.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This handles macroblock edges for the chroma components in the same way
as for the luma compoment for 4:4:4 streams. The Spec explicitly states
that the deblocking filter is not applied to edges at the boundary of
the picture.
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
* commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1':
h264: do not call field_end if we do not have a current picture
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise the ER code might try to use some already freed references.
Fixes possible access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
In this case we may not have a current frame, while first_field being
set implies we do.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
* commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf':
h264: check that execute_decode_slices() is not called too many times
Conflicts:
libavcodec/h264.c
The check is replaced by an assert() as the mb index should not ever go out
of bounds.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8':
h264: rebuild the default ref list if the reference count changes
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
There is no point in delaying the check and it avoids bugs with a
half-initialized context.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
* commit '58312b2472d3a44d7458865c459d59ef2e02bf1a':
h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
Merged-by: Michael Niedermayer <michaelni@gmx.at>
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
This is a temporary workaround to allow deprecating
avcodec_get_frame_defaults(). The proper solution will be using a
properly allocated AVFrame in Picture.
* commit '5b10ef729f610fcbc9c485e7b643ce53268144cb':
h264: parse frame packing arrangement SEI messages and save relevant stereo3d information
Conflicts:
libavcodec/h264.c
libavcodec/h264_sei.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23':
lavc: use buf[0] instead of data[0] in checks whether a frame is allocated
Conflicts:
libavcodec/h264_refs.c
libavcodec/mpegvideo.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '458446acfa1441d283dacf9e6e545beb083b8bb0':
lavc: Edge emulation with dst/src linesize
Conflicts:
libavcodec/cavs.c
libavcodec/h264.c
libavcodec/hevc.c
libavcodec/mpegvideo_enc.c
libavcodec/mpegvideo_motion.c
libavcodec/rv34.c
libavcodec/svq3.c
libavcodec/vc1dec.c
libavcodec/videodsp.h
libavcodec/videodsp_template.c
libavcodec/vp3.c
libavcodec/vp8.c
libavcodec/wmv2.c
libavcodec/x86/videodsp.asm
libavcodec/x86/videodsp_init.c
Changes to the asm are not merged, they are left for volunteers or
in their absence for later.
The changes this merge introduces are reordering of the function
arguments
See: face578d56
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Allow supporting files for which the image stride is smaller than
the maximum block size + number of subpel mc taps, e.g. a 64x64 VP9
file or a 16x16 VP8 file with -fflags +emu_edge.
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.
Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '28096e0a806e57376541e6222d315619906e3c55':
h264: wait for initial complete frame before outputing frames
Conflicts:
doc/APIchanges
libavcodec/h264.c
libavcodec/mpegvideo.h
libavutil/frame.h
libavutil/version.h
See: a64b028aeb (as well as various later commits)
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This can be optionally disabled whith the "output_corrupt" flags
option. When in "output_corrupt" mode, incomplete frames are
signalled through AVFrame.flags FRAME_FLAG_INCOMPLETE_FRAME.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
* commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208':
h264_parser: Fix POC parsing for the case where MMCO_RESET is present.
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This ensures that all the non zero fields get setup correctly.
Alternatively their initialization could be duplicated.
Fixes Ticket2738
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This allows supporting files for which the image stride is smaller than
the max. block size + number of subpel mc taps, e.g. a 64x64 VP9 file
or a 16x16 VP8 file with -fflags +emu_edge.
* commit 'cab8c5f8e140c96ba3725ab709d823abfd1e31a5':
h264: do not reinitialize the global cabac tables at each slice header
See: 1e2e2c8095
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '71cabb521ac397db3903011d2de7afd3e0fc7ab6':
h264: do not discard NAL_SEI when skipping frames
Conflicts:
libavcodec/h264.c
See: 7d75fb381b
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'c4e43560fe6677e9d60bfb3cffc41c7324e92a0b':
h264data: Move some tables to the only place they are used
Conflicts:
libavcodec/h264data.h
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This reverts commit bf36dc50ea, reversing
changes made to b7fc2693c7.
Conflicts:
libavcodec/h264.c
Keeping support for the old VDPAU API has been requested by our VDPAU maintainer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
