1483 Commits

Author SHA1 Message Date
Michael Niedermayer
b188ff5e18 avcodec/h264: Check *log2_weight_denom
Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61296d41e2de3b41304339e4631dd44c2e15f805)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-30 15:38:21 +01:00
Michael Niedermayer
84bc2cea23 avcodec/h264: Clear delayed_pic on deallocation
Fixes use of freed memory

Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8714f6f93d1a32f4e4655209960afcf4c185214)

Conflicts:

	libavcodec/h264.c
2014-12-30 15:38:20 +01:00
Michael Niedermayer
4352a971af avcodec/h264_slice: Clear table pointers to avoid stale pointers
Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce95858ef83f8c25ae347e3ae3b8ba437fd9)

Conflicts:

	libavcodec/h264_slice.c
2014-11-28 18:53:11 +01:00
Michael Niedermayer
f335275c50 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer
f7086be79a avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334badd8af48f13c6fd1e6827f8e3f2c2f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
c5085a0301 avcodec/h264: clear cur_pic structure instead of duplicating it in ff_h264_update_thread_context()
Fixes crash

Found-by: iive
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8710ee11d75eebc17e7d63bc6ffb91766933bd68)

Conflicts:

	libavcodec/h264_slice.c
(cherry picked from commit cb44d69665596d2f19e032fc852162b0a6de0562)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 15:10:07 +02:00
Michael Niedermayer
e8bb67c2a9 avcodec/h264: in the absence of recovery points, be more tolerant on accepting plain I frames
Fixes: Ticket3652
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19c9d1e8e71da04c3ac940206619c0a2d01e5193)
2014-06-10 00:10:40 +02:00
Michael Niedermayer
9098da0329 avcodec/h264: be more tolerant on what pixel format changes trigger reinits
Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8e92ff25469f75f5c1fcbb9ba5721cea341ca34a)
2014-03-17 15:53:21 +01:00
Michael Niedermayer
c7c724056e avcodec/h264: clear chroma planes when flags gray is used
Fixes Ticket3397
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 72e691314027b08955679319394dd0d8477973b7)
2014-02-24 09:45:48 +01:00
Michael Niedermayer
846a9c67ff avcodec/h264: use subsample factors of the used pixel format
Fixes out of array read
Fixes: 1cb91c36c4e55463f14aacb9bdf55b38-asan_heap-oob_106cbce_5617_cov_11212800_h264_mmx_chroma_intra_lf.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c55ff393340998faae887dfac19e7ef128e1e58)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-23 15:04:07 +01:00
Michael Niedermayer
d0d441b350 avcodec/h264: more completely check the loop filter parameters
Fixes out of array read
Fixes: caa65cc01655505705129b677189f036-signal_sigsegv_fdcc43_2681_cov_3043376737_PPH422I5_Panasonic_A.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 91253839e14cce9793ee93f184cef609ca8195d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-23 15:02:14 +01:00
Michael Niedermayer
e7b7e69416 avcodec/h264: update current_sps & sps->new only after the whole slice header decoder and init code finished
This avoids them being cleared before the full initialization finished

Fixes out of array read
Fixes: asan_heap-oob_f0c5e6_7071_cov_1605985132_mov_h264_aac__Demo_FlagOfOurFathers.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8a3b85f3a7952c54a2c36ba1797f7e0cde9f85aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-23 14:49:47 +01:00
Michael Niedermayer
9330bcff9b avcodec/h264: Disallow pps_id changing between slices
Such changes are forbidden in H.264 and lead to race conditions

Fixes out of array read
Fixes: signal_sigsegv_f9796a_1613_cov_3114610371_FM1_BT_B.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e708424b70bef8641e8a090ec4d9e8c4490db87e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-23 14:49:37 +01:00
Michael Niedermayer
9e8464e81b Merge commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf'
* commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf':
  h264: check that execute_decode_slices() is not called too many times

Conflicts:
	libavcodec/h264.c

The check is replaced by an assert() as the mb index should not ever go out
of bounds.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 64591f8f86f2dfeac13ee6b4e971d069675ca814)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 16:42:32 +01:00
Michael Niedermayer
6fb0f37def avcodec/h264: fix code that blindly dereferences NULL DPB
Fixes mixed flushing and decoding NULL packets
Found-by: wm4

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d9339ab55373b12f078a3e3f1e294d8ff78652dd)

Conflicts:

	libavcodec/h264.c
2014-01-13 16:06:13 +01:00
Dale Curtis
dcf0f82d08 h264: Clear ERContext.cur_pic when unref'ing current picture.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
(cherry picked from commit 4feca2214a0b69dcbe4d1c7cd145c3881459e867)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 03:05:30 +01:00
Michael Niedermayer
1497633924 h264: Do not treat the initial frame special in handling of frame gaps
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.

Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e5ef1c5c37208326c59d642e2dc7afd3f10b09b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-17 19:11:07 +01:00
Michael Niedermayer
3c9dd93faa h264: make flush_change() set mmco_reset
This ensures that frames do not get mixed on context reinits

Fixes Ticket2836

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-26 01:22:38 +02:00
Michael Niedermayer
29ffeef5e7 avcodec/h264: do not trust last_pic_droppable when marking pictures as done
This simplifies the code and fixes a deadlock

Fixes Ticket2927
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-24 23:15:30 +02:00
Michael Niedermayer
f7f74a37b8 Merge commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208'
* commit '4baba6c813b7a1f27370e20fb1a87b05fcb39208':
  h264_parser: Fix POC parsing for the case where MMCO_RESET is present.

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-16 12:51:26 +02:00
Yusuke Nakamura
4baba6c813 h264_parser: Fix POC parsing for the case where MMCO_RESET is present.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-10-15 20:02:55 +02:00
Michael Niedermayer
65c2fe7159 avcodec/h264: call flush_change() on init
This ensures that all the non zero fields get setup correctly.
Alternatively their initialization could be duplicated.

Fixes Ticket2738

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-10 02:25:51 +02:00
Clément Bœsch
b46f19100b cosmetics: group remaining .name and .long_name.
See b2bed9325.
2013-10-04 13:33:20 +00:00
Michael Niedermayer
ee77140afa Merge commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2'
* commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2':
  cosmetics: Group .name and .long_name together in codec/format declarations

Conflicts:
	libavcodec/8svx.c
	libavcodec/alac.c
	libavcodec/cljr.c
	libavcodec/dnxhddec.c
	libavcodec/dnxhdenc.c
	libavcodec/dpxenc.c
	libavcodec/dvdec.c
	libavcodec/dvdsubdec.c
	libavcodec/dvdsubenc.c
	libavcodec/ffv1dec.c
	libavcodec/flacdec.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/g726.c
	libavcodec/gif.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/h263dec.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopencore-amr.c
	libavcodec/libopenjpegdec.c
	libavcodec/libopenjpegenc.c
	libavcodec/libspeexenc.c
	libavcodec/libvo-amrwbenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/libxavs.c
	libavcodec/libxvid.c
	libavcodec/ljpegenc.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpeg12dec.c
	libavcodec/mpeg4videodec.c
	libavcodec/msmpeg4dec.c
	libavcodec/pgssubdec.c
	libavcodec/pngdec.c
	libavcodec/pngenc.c
	libavcodec/proresdec_lgpl.c
	libavcodec/proresenc_kostya.c
	libavcodec/ra144enc.c
	libavcodec/rawdec.c
	libavcodec/rv10.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tta.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/wavpack.c
	libavcodec/xbmenc.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-04 12:34:23 +02:00
Diego Biurrun
b2bed9325d cosmetics: Group .name and .long_name together in codec/format declarations 2013-10-03 23:32:01 +02:00
Carl Eugen Hoyos
b5f219c904 Print pix_fmt name on h264 context reinitialisation. 2013-10-03 01:42:23 +02:00
Ronald S. Bultje
face578d56 Rewrite emu_edge functions to have separate src/dst_stride arguments.
This allows supporting files for which the image stride is smaller than
the max. block size + number of subpel mc taps, e.g. a 64x64 VP9 file
or a 16x16 VP8 file with -fflags +emu_edge.
2013-09-28 20:28:08 -04:00
Ronald S. Bultje
c341f734e5 Convert multiplier for MV from int to ptrdiff_t.
This prevents emulated_edge_mc from not undoing mvy*stride-related
integer overflows.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-28 11:28:09 +02:00
Michael Niedermayer
4a89868c5f Merge commit 'cab8c5f8e140c96ba3725ab709d823abfd1e31a5'
* commit 'cab8c5f8e140c96ba3725ab709d823abfd1e31a5':
  h264: do not reinitialize the global cabac tables at each slice header

See: 1e2e2c8095de2d9ea3259305cfeff28f40e4ca12
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 10:25:17 +02:00
Anton Khirnov
cab8c5f8e1 h264: do not reinitialize the global cabac tables at each slice header 2013-09-24 17:13:52 +02:00
Michael Niedermayer
d195e6e410 Merge commit '71cabb521ac397db3903011d2de7afd3e0fc7ab6'
* commit '71cabb521ac397db3903011d2de7afd3e0fc7ab6':
  h264: do not discard NAL_SEI when skipping frames

Conflicts:
	libavcodec/h264.c

See: 7d75fb381ba774a8d2fa7de0c12140cd465c0255
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-24 15:38:34 +02:00
Michael Niedermayer
af2bb83dff Merge commit 'edaba18021e4793116f0f266e65077578a5e742e'
* commit 'edaba18021e4793116f0f266e65077578a5e742e':
  h264: log extradata skip only for non-ignored NALs

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-24 15:30:21 +02:00
Rainer Hochecker
71cabb521a h264: do not discard NAL_SEI when skipping frames
Setting AVDISCARD_NONREF (e.g. after flushing) resulted in 100% dropped frames.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-09-24 13:24:29 +02:00
Vittorio Giovara
edaba18021 h264: log extradata skip only for non-ignored NALs
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-09-24 13:24:28 +02:00
Vittorio Giovara
1cad7171dd h264: remove an unused static constant
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-09-20 14:40:10 +03:00
Rainer Hochecker
7d75fb381b h264: do not discard NAL_SEI when skipping frames
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-09 11:42:26 +02:00
Michael Niedermayer
93cf7b0195 avcodec/h264: set er.ref_count earlier
Fixes Ticket2910

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-01 04:02:03 +02:00
Michael Niedermayer
626739ebbb avcodec/h264: Free rbsp_buffer before copying context over it
Fixes memleak
Fixes Ticket1900

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:30:31 +02:00
Michael Niedermayer
be30e44dd9 avcodec/h264: Zero rbsp_buffer earler to ensure no duplicated pointers can leak
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:29:52 +02:00
Michael Niedermayer
ecbf838c7d h264: prevent rbsp_buffer values from becoming duplicated
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 03:01:19 +02:00
Michael Niedermayer
16466d92b9 Merge commit 'c4e43560fe6677e9d60bfb3cffc41c7324e92a0b'
* commit 'c4e43560fe6677e9d60bfb3cffc41c7324e92a0b':
  h264data: Move some tables to the only place they are used

Conflicts:
	libavcodec/h264data.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:21:48 +02:00
Michael Niedermayer
58e12732db Merge commit '2a61592573d725956a4377641344afe263382648'
* commit '2a61592573d725956a4377641344afe263382648':
  avcodec: Remove some commented-out debug cruft

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mvpred.h
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 11:18:43 +02:00
Diego Biurrun
c4e43560fe h264data: Move some tables to the only place they are used 2013-08-20 20:49:37 +02:00
Diego Biurrun
2a61592573 avcodec: Remove some commented-out debug cruft 2013-08-20 19:59:50 +02:00
Michael Niedermayer
a0c6c8e53e Revert "Merge commit of 'vdpau: remove old-style decoders'"
This reverts commit bf36dc50ea448999c8f8c7a35f6139a7040f6275, reversing
changes made to b7fc2693c70fe72936e4ce124c802ac23857c476.

Conflicts:

	libavcodec/h264.c

Keeping support for the old VDPAU API has been requested by our VDPAU maintainer

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-17 22:17:49 +02:00
Michael Niedermayer
921c1d4c95 Merge commit 'c1076d8479a6c0ee2e0c4b0e2151df5b0228438e'
* commit 'c1076d8479a6c0ee2e0c4b0e2151df5b0228438e':
  h264: check one context_init() allocation

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:35:44 +02:00
Michael Niedermayer
d2d8e259fd Merge commit '5eb488bfa835f2902a31ba99d57c16ae36c4f598'
* commit '5eb488bfa835f2902a31ba99d57c16ae36c4f598':
  h264: use explicit variable names for *_field_flag

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:28:49 +02:00
Michael Niedermayer
e0b45ca730 Merge commit 'b3dc260e7fa6f3f852dd5cb7d86763c4b5736714'
* commit 'b3dc260e7fa6f3f852dd5cb7d86763c4b5736714':
  h264: return meaningful values

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-11 10:08:48 +02:00
Vittorio Giovara
c1076d8479 h264: check one context_init() allocation
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-10 13:36:38 +02:00
Vittorio Giovara
5eb488bfa8 h264: use explicit variable names for *_field_flag
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-08-10 13:36:25 +02:00