Commit Graph

1511 Commits

Author SHA1 Message Date
Michael Niedermayer
87e46dd5fb Merge commit '0652e024c680420d298cdf3719d0a0c030173fe3'
* commit '0652e024c680420d298cdf3719d0a0c030173fe3':
  h264: reset ref count if decoding the slice header fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 22:51:59 +01:00
Michael Niedermayer
329610303a Merge commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1'
* commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1':
  h264: do not call field_end if we do not have a current picture

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:51:40 +01:00
Michael Niedermayer
f2cb3b36ac Merge commit '7f0e81db3c4ee6f8ce15058bafa72ce928a89f3f'
* commit '7f0e81db3c4ee6f8ce15058bafa72ce928a89f3f':
  h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3

Conflicts:
	libavcodec/h264.c

See: d6a33f5d20
See: 2005fddcbb
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:44:18 +01:00
Michael Niedermayer
357a733f91 Merge commit 'd1b3fabe6945e511bb20fc9ca52b47eb952526ee'
* commit 'd1b3fabe6945e511bb20fc9ca52b47eb952526ee':
  h264: reset first_field if frame_start() fails for missing refs

See: d7599bd8e2
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 21:29:37 +01:00
Anton Khirnov
0652e024c6 h264: reset ref count if decoding the slice header fails
Otherwise the ER code might try to use some already freed references.

Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:51:04 +01:00
Anton Khirnov
00dbff4c3e h264: do not call field_end if we do not have a current picture
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:42:21 +01:00
Anton Khirnov
7f0e81db3c h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:41:59 +01:00
Anton Khirnov
d1b3fabe69 h264: reset first_field if frame_start() fails for missing refs
In this case we may not have a current frame, while first_field being
set implies we do.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:41:24 +01:00
Michael Niedermayer
64591f8f86 Merge commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf'
* commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf':
  h264: check that execute_decode_slices() is not called too many times

Conflicts:
	libavcodec/h264.c

The check is replaced by an assert() as the mb index should not ever go out
of bounds.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:39:38 +01:00
Michael Niedermayer
a60abb1ee0 Merge commit 'bfd26b7ce6efea594f2b99441d900419df3af638'
* commit 'bfd26b7ce6efea594f2b99441d900419df3af638':
  h264: reject mismatching luma/chroma bit depths during sps parsing

Conflicts:
	libavcodec/h264_ps.c

See: bdeb61ccc6
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:28:55 +01:00
Michael Niedermayer
98dcbb47fa avcodec/h264: reset list_count too in case of error in ff_set_ref_count()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:17:45 +01:00
Michael Niedermayer
7ee8a1c562 Merge commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8'
* commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8':
  h264: rebuild the default ref list if the reference count changes

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 16:17:38 +01:00
Anton Khirnov
9eef9eb301 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:25:25 +01:00
Anton Khirnov
bfd26b7ce6 h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:23:45 +01:00
Anton Khirnov
9a026c7298 h264: rebuild the default ref list if the reference count changes
Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-06 08:23:17 +01:00
Michael Niedermayer
8e6af036b9 Merge commit '58312b2472d3a44d7458865c459d59ef2e02bf1a'
* commit '58312b2472d3a44d7458865c459d59ef2e02bf1a':
  h264: reset data_partitioning if decoding the slice header for NAL_DPA fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-04 02:06:21 +01:00
Anton Khirnov
58312b2472 h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-03 16:42:02 +01:00
Dale Curtis
4feca2214a h264: Clear ERContext.cur_pic when unref'ing current picture.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
2014-01-02 23:49:06 +01:00
Michael Niedermayer
74a9c92840 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: namespace the decode function

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-29 11:37:42 +01:00
Michael Niedermayer
6ea05ef278 avcodec/h264: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-28 18:19:46 +01:00
Luca Barbato
4d2bb28931 h264: namespace the decode function
Make much easier debugging.
2013-12-27 07:48:13 +01:00
Michael Niedermayer
e3578fd525 Merge commit '598ce4ab4f1893e0661fc038101487e511937877'
* commit '598ce4ab4f1893e0661fc038101487e511937877':
  h264: call av_frame_unref() instead of avcodec_get_frame_defaults().

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-12 00:25:31 +01:00
Anton Khirnov
598ce4ab4f h264: call av_frame_unref() instead of avcodec_get_frame_defaults().
This is a temporary workaround to allow deprecating
avcodec_get_frame_defaults(). The proper solution will be using a
properly allocated AVFrame in Picture.
2013-12-11 20:39:55 +01:00
Michael Niedermayer
d9339ab553 avcodec/h264: fix code that blindly dereferences NULL DPB
Fixes mixed flushing and decoding NULL packets
Found-by: wm4

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-10 23:50:08 +01:00
Michael Niedermayer
be6e81463c Merge commit '5b10ef729f610fcbc9c485e7b643ce53268144cb'
* commit '5b10ef729f610fcbc9c485e7b643ce53268144cb':
  h264: parse frame packing arrangement SEI messages and save relevant stereo3d information

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_sei.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-09 21:53:23 +01:00
Vittorio Giovara
5b10ef729f h264: parse frame packing arrangement SEI messages and save relevant stereo3d information 2013-12-09 16:02:43 +01:00
Michael Niedermayer
fe540ae6b7 Merge commit 'f0259a587ee3419dd894873ea617b4c98eeaca1c'
* commit 'f0259a587ee3419dd894873ea617b4c98eeaca1c':
  h264: check buffer size before accessing it

Conflicts:
	libavcodec/h264.c

See: ea0ac11e52
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-22 11:56:46 +01:00
Anton Khirnov
f0259a587e h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2013-11-21 20:54:20 +01:00
Michael Niedermayer
56e122787e Merge commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23'
* commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23':
  lavc: use buf[0] instead of data[0] in checks whether a frame is allocated

Conflicts:
	libavcodec/h264_refs.c
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-19 03:56:53 +01:00
Anton Khirnov
a553c6a347 lavc: use buf[0] instead of data[0] in checks whether a frame is allocated
data[0] may be NULL for valid frames with hwaccel pixel formats.
2013-11-18 18:09:48 +01:00
Michael Niedermayer
91e00c4a78 Merge commit '458446acfa1441d283dacf9e6e545beb083b8bb0'
* commit '458446acfa1441d283dacf9e6e545beb083b8bb0':
  lavc: Edge emulation with dst/src linesize

Conflicts:
	libavcodec/cavs.c
	libavcodec/h264.c
	libavcodec/hevc.c
	libavcodec/mpegvideo_enc.c
	libavcodec/mpegvideo_motion.c
	libavcodec/rv34.c
	libavcodec/svq3.c
	libavcodec/vc1dec.c
	libavcodec/videodsp.h
	libavcodec/videodsp_template.c
	libavcodec/vp3.c
	libavcodec/vp8.c
	libavcodec/wmv2.c
	libavcodec/x86/videodsp.asm
	libavcodec/x86/videodsp_init.c

Changes to the asm are not merged, they are left for volunteers or
in their absence for later.
The changes this merge introduces are reordering of the function
arguments

See: face578d56
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-15 15:07:10 +01:00
Ronald S. Bultje
458446acfa lavc: Edge emulation with dst/src linesize
Allow supporting files for which the image stride is smaller than
the maximum block size + number of subpel mc taps, e.g. a 64x64 VP9
file or a 16x16 VP8 file with -fflags +emu_edge.
2013-11-15 10:16:27 +01:00
Michael Niedermayer
9244a68092 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: free the tables and uninitialize the context on flush

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-14 21:11:42 +01:00
Anton Khirnov
9eda9d3322 h264: free the tables and uninitialize the context on flush
Prevents referencing empty frames when the first packet after the flush
does not contain a frame.
2013-11-14 19:00:20 +01:00
Michael Niedermayer
9e5ef1c5c3 h264: Do not treat the initial frame special in handling of frame gaps
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.

Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-14 12:16:01 +01:00
Michael Niedermayer
78e150c5e9 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  hwaccel: Simplify ff_find_hwaccel

Conflicts:
	libavcodec/mpeg12dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-10 17:37:51 +01:00
Luca Barbato
08303d7741 hwaccel: Simplify ff_find_hwaccel
It is always called by passing fields from an AVCodecContext.
2013-11-10 13:59:48 +01:00
Michael Niedermayer
4fb1221e66 h264: reduce whitespace differences to libav
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-03 01:16:31 +01:00
Michael Niedermayer
647adc4290 h264: factor "if(h->sei_recovery_frame_cnt >= 0)" out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-02 01:02:21 +01:00
Michael Niedermayer
f62dfed3de h264: simplify frame_recovered code
This is a separate commit to ease future bisecting in case it breaks something.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-01 12:59:40 +01:00
Michael Niedermayer
78265fcfee Merge commit '28096e0a806e57376541e6222d315619906e3c55'
* commit '28096e0a806e57376541e6222d315619906e3c55':
  h264: wait for initial complete frame before outputing frames

Conflicts:
	doc/APIchanges
	libavcodec/h264.c
	libavcodec/mpegvideo.h
	libavutil/frame.h
	libavutil/version.h

See: a64b028aeb (as well as various later commits)
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-01 12:59:08 +01:00
John Stebbins
28096e0a80 h264: wait for initial complete frame before outputing frames
This can be optionally disabled whith the "output_corrupt" flags
option.  When in "output_corrupt" mode, incomplete frames are
signalled through AVFrame.flags FRAME_FLAG_INCOMPLETE_FRAME.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-10-31 20:14:14 +01:00
Derek Buitenhuis
58d13cea30 h264: Check all allocations
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2013-10-29 14:00:29 +00:00
Derek Buitenhuis
a483aae7d8 h264: Check all allocations
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2013-10-29 12:00:27 +00:00
Michael Niedermayer
3c9dd93faa h264: make flush_change() set mmco_reset
This ensures that frames do not get mixed on context reinits

Fixes Ticket2836

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-26 01:22:38 +02:00
Michael Niedermayer
29ffeef5e7 avcodec/h264: do not trust last_pic_droppable when marking pictures as done
This simplifies the code and fixes a deadlock

Fixes Ticket2927
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-24 23:15:30 +02:00
Michael Niedermayer
f7f74a37b8 Merge commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208'
* commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208':
  h264_parser: Fix POC parsing for the case where MMCO_RESET is present.

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-16 12:51:26 +02:00
Ronald S. Bultje
93f305473f lavc: Convert some remaining strides to ptrdiff_t 2013-10-15 23:58:20 +02:00
Yusuke Nakamura
4baba6c813 h264_parser: Fix POC parsing for the case where MMCO_RESET is present.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-10-15 20:02:55 +02:00
Michael Niedermayer
65c2fe7159 avcodec/h264: call flush_change() on init
This ensures that all the non zero fields get setup correctly.
Alternatively their initialization could be duplicated.

Fixes Ticket2738

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-10 02:25:51 +02:00