Commit Graph

130 Commits

Author SHA1 Message Date
Michael Niedermayer
aa427537b5 avcodec/h264_refs: Check that long references match before use
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 03:27:28 +01:00
wm4
9747219958 avcodec/h264, videotoolbox: do not return invalid frames on failure
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.

These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
2015-11-20 11:30:18 +01:00
Michael Niedermayer
9ec17e45b2 avcodec/h264_refs: Allow slightly larger pps_ref_count[0] in PAFF RAP detection heuristic
Fixes Ticket4738

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-29 03:03:52 +02:00
Michael Niedermayer
0a6aa30f7c avcodec/h264_refs: extend RAP heuristic to multiple PPS
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-29 03:03:30 +02:00
Michael Niedermayer
4aa0de644a avcodec/h264_refs: discard mismatching references
Fixes inconsistency and out of array access
Fixes: asan_heap-oob_17301a3_2100_cov_3226131691_ff_add_pixels_clamped_mmx.m2ts

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-30 03:24:26 +02:00
Michael Niedermayer
77412f0ecd Merge commit '6d4d3fee63c46d921c4870feab79269af94e84e1'
* commit '6d4d3fee63c46d921c4870feab79269af94e84e1':
  h264: make sure the current picture is not made a long ref multiple times

See: 34ea5f418e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 20:56:30 +02:00
Anton Khirnov
6d4d3fee63 h264: make sure the current picture is not made a long ref multiple times
Fixes possible invalid reads, once one of those refs is freed, but the
others remain.
CC: libav-stable@libav.org
2015-06-27 16:46:01 +02:00
Michael Niedermayer
36f862e04c Merge commit 'a0f2946068c62e18cb05ac25c0df3d86077251a6'
* commit 'a0f2946068c62e18cb05ac25c0df3d86077251a6':
  h264: use properly allocated AVFrames

Conflicts:
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_refs.c
	libavcodec/h264_slice.c
	libavcodec/svq3.c
	libavcodec/vda_h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 16:35:24 +02:00
Anton Khirnov
a0f2946068 h264: use properly allocated AVFrames 2015-04-29 05:52:58 +02:00
James Almer
ba625dd8a1 avcodec: use av_mod_uintp2() where useful
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
2015-04-21 22:41:20 -03:00
Michael Niedermayer
8f7b022c8c Merge commit '6a85dfc830f51f1f5c2d36d4182d265c1ea3ba25'
* commit '6a85dfc830f51f1f5c2d36d4182d265c1ea3ba25':
  lavc: Replace av_dlog and tprintf with internal macros

Conflicts:
	libavcodec/aacdec.c
	libavcodec/audio_frame_queue.c
	libavcodec/bitstream.c
	libavcodec/dcadec.c
	libavcodec/dnxhddec.c
	libavcodec/dvbsubdec.c
	libavcodec/dvdec.c
	libavcodec/dvdsubdec.c
	libavcodec/get_bits.h
	libavcodec/gifdec.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_refs.c
	libavcodec/imc.c
	libavcodec/interplayvideo.c
	libavcodec/jpeglsdec.c
	libavcodec/libopencore-amr.c
	libavcodec/mjpegdec.c
	libavcodec/mpeg12dec.c
	libavcodec/mpegvideo_enc.c
	libavcodec/mpegvideo_parser.c
	libavcodec/pngdec.c
	libavcodec/ratecontrol.c
	libavcodec/rv10.c
	libavcodec/svq1dec.c
	libavcodec/vqavideo.c
	libavcodec/wmadec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-20 04:10:10 +02:00
Vittorio Giovara
6a85dfc830 lavc: Replace av_dlog and tprintf with internal macros 2015-04-19 12:41:59 +01:00
Michael Niedermayer
aa6f58dd96 avcodec/h264_refs: More completely clear slice contexts in ff_h264_remove_all_refs()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-04 19:30:10 +02:00
Michael Niedermayer
429de04320 avcodec/h264_refs: Do not set reference to things which dont exist
Fixes deadlock
Fixes Ticket4428
Fixes Ticket4429

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-04 18:25:44 +02:00
Michael Niedermayer
a38758a97e avcodec/h264_refs: fix null pointer dereference with -DTRACE
Found-by: carl
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-31 01:57:00 +02:00
Carl Eugen Hoyos
b3673f3414 lavc/h264_refs: Fix compilation with -DTRACE. 2015-03-29 12:34:43 +02:00
Michael Niedermayer
d8151a7e94 Merge commit 'a12d3188cbec15e22070e139fa5cc541da07e2c3'
* commit 'a12d3188cbec15e22070e139fa5cc541da07e2c3':
  h264: use a smaller struct for the ref lists

Conflicts:
	libavcodec/h264_direct.c
	libavcodec/h264_mb.c
	libavcodec/h264_picture.c
	libavcodec/h264_refs.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-21 23:05:34 +01:00
Michael Niedermayer
55215383dc Merge commit 'f42485dbce614b3f63182845da43db690b427b7c'
* commit 'f42485dbce614b3f63182845da43db690b427b7c':
  h264: use a separate GetBitContext for slice data

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_parser.c
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-21 17:13:24 +01:00
Michael Niedermayer
cc2cfdc497 Merge commit '95eb35f30513e335990ad0d5dca6ddc318477291'
* commit '95eb35f30513e335990ad0d5dca6ddc318477291':
  h264: move the ref lists variables into the per-slice context

Conflicts:
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_direct.c
	libavcodec/h264_mb.c
	libavcodec/h264_picture.c
	libavcodec/h264_refs.c
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-21 15:18:10 +01:00
Michael Niedermayer
97c8ecaada Merge commit '56febc993b928ccc039a32158ca60b234c311875'
* commit '56febc993b928ccc039a32158ca60b234c311875':
  h264: move the slice type variables into the per-slice context

Conflicts:
	libavcodec/h264.c
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-21 14:30:19 +01:00
Michael Niedermayer
756dc7bd18 Merge commit '92c6c2a605f9b077b8fbc25b7ed6625541232b87'
* commit '92c6c2a605f9b077b8fbc25b7ed6625541232b87':
  h264: split weighted pred-related vars into per-slice context

Conflicts:
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_mb.c
	libavcodec/svq3.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-21 13:13:12 +01:00
Anton Khirnov
a12d3188cb h264: use a smaller struct for the ref lists
There is no need to store a whole H264Picture, with a full AVFrame
embedded in it. This should allow getting rid of the embedded AVFrame
later.
2015-03-21 11:27:16 +01:00
Anton Khirnov
f42485dbce h264: use a separate GetBitContext for slice data 2015-03-21 11:27:14 +01:00
Anton Khirnov
95eb35f305 h264: move the ref lists variables into the per-slice context 2015-03-21 11:27:13 +01:00
Anton Khirnov
56febc993b h264: move the slice type variables into the per-slice context 2015-03-21 11:27:13 +01:00
Anton Khirnov
92c6c2a605 h264: split weighted pred-related vars into per-slice context 2015-03-21 11:27:09 +01:00
Michael Niedermayer
ecfd48dc06 avcodec/h264: Move COPY_PICTURE() to h264.h so it can be used from other parts of the h264 decoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-08 02:27:27 +01:00
Michael Niedermayer
e09ad5bd0d avcodec/h264_refs: set last_pic_for_ec only if it has not been set previously
This ensures we do not loose the frame in case or multiple clears
Fixes out of array read
Fixes: asan_heap-oob_2fa47ea_2100_cov_1278768963_ff_add_pixels_clamped_mmx.m2ts

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-05 14:57:42 +01:00
Michael Niedermayer
855463c007 avcodec/h264: Keep a reference to the last picture for EC
This and the next commit improve error concealment for
green-block-artifacts-from-canon-100-hs.MOV

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-11 17:30:02 +01:00
Michael Niedermayer
60dbed6067 Merge commit '4b1f5e5090abed6c618c8ba380cd7d28d140f867'
* commit '4b1f5e5090abed6c618c8ba380cd7d28d140f867':
  cosmetics: Write NULL pointer inequality checks more compactly

Conflicts:
	libavcodec/dvdsubdec.c
	libavcodec/h263dec.c
	libavcodec/libxvid.c
	libavcodec/rv10.c
	libavcodec/utils.c
	libavformat/format.c
	libavformat/matroskadec.c
	libavformat/segment.c
	libavutil/opt.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-15 21:26:12 +02:00
Michael Niedermayer
fb33bff990 Merge commit 'f929ab0569ff31ed5a59b0b0adb7ce09df3fca39'
* commit 'f929ab0569ff31ed5a59b0b0adb7ce09df3fca39':
  cosmetics: Write NULL pointer equality checks more compactly

Conflicts:
	cmdutils.c
	ffmpeg_opt.c
	ffplay.c
	libavcodec/dvbsub.c
	libavcodec/dvdsubdec.c
	libavcodec/dvdsubenc.c
	libavcodec/dxa.c
	libavcodec/libxvid_rc.c
	libavcodec/mpegvideo.c
	libavcodec/mpegvideo_enc.c
	libavcodec/rv10.c
	libavcodec/tiffenc.c
	libavcodec/utils.c
	libavcodec/vc1dec.c
	libavcodec/zmbv.c
	libavdevice/v4l2.c
	libavformat/matroskadec.c
	libavformat/movenc.c
	libavformat/sdp.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-15 21:00:50 +02:00
Gabriel Dume
4b1f5e5090 cosmetics: Write NULL pointer inequality checks more compactly
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-08-15 05:34:13 -07:00
Gabriel Dume
f929ab0569 cosmetics: Write NULL pointer equality checks more compactly
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-08-15 03:18:18 -07:00
Michael Niedermayer
19c9d1e8e7 avcodec/h264: in the absence of recovery points, be more tolerant on accepting plain I frames
Fixes: Ticket3652
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-04 04:25:14 +02:00
Michael Niedermayer
45ba9d8545 Merge commit 'cba4e6062a02d5bf684e13a770be88aa1fec717e'
* commit 'cba4e6062a02d5bf684e13a770be88aa1fec717e':
  More correct printf format specifiers

Conflicts:
	libavcodec/h264_ps.c
	libavcodec/h264_refs.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-02 02:54:09 +02:00
Diego Biurrun
cba4e6062a More correct printf format specifiers
This avoids compiler warnings about argument and specifier type mismatch.
2014-04-02 00:27:22 +02:00
Michael Niedermayer
9517900bef Merge commit 'e0c16e4e3259cf50b5bac4c23bb6e517f397c74b'
* commit 'e0c16e4e3259cf50b5bac4c23bb6e517f397c74b':
  mpegvideo: move mpegvideo formats-related defines to mpegutils.h

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_mvpred.h
	libavcodec/svq1enc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 12:36:48 +01:00
Michael Niedermayer
c237e88d51 Merge commit '9b749c8274f6b6f35dde2cf29b99fa4f719abf87'
* commit '9b749c8274f6b6f35dde2cf29b99fa4f719abf87':
  h264: move relevant fields from Picture to H264Picture

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-17 05:51:27 +01:00
Vittorio Giovara
e0c16e4e32 mpegvideo: move mpegvideo formats-related defines to mpegutils.h 2014-03-16 23:04:41 +01:00
Vittorio Giovara
9b749c8274 h264: move relevant fields from Picture to H264Picture 2014-03-16 23:01:00 +01:00
Michael Niedermayer
5aab33dea3 Merge commit 'f4bd9fe326ad1315a74206939ae56df93b940a09'
* commit 'f4bd9fe326ad1315a74206939ae56df93b940a09':
  h264: fix clang warning about uninitialized variable

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-01 05:03:59 +01:00
Michael Niedermayer
153b870d1e Merge commit '17a63ff0cd187b9e50e4a47862750295976853b1'
* commit '17a63ff0cd187b9e50e4a47862750295976853b1':
  h264: update flag name in ff_h264_decode_ref_pic_list_reordering()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-01 05:02:58 +01:00
Vittorio Giovara
f4bd9fe326 h264: fix clang warning about uninitialized variable 2014-02-28 18:15:15 +01:00
Vittorio Giovara
17a63ff0cd h264: update flag name in ff_h264_decode_ref_pic_list_reordering()
This is the name used in the specifications.
2014-02-28 18:14:34 +01:00
Michael Niedermayer
649686d89b avcodec/h264_refs: remove lost frames instead of disfavoring them
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-27 11:16:23 +01:00
Michael Niedermayer
b5005def8a avcodec/h264: avoid using lost frames as references
Fixes Ticket3386

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-27 03:26:03 +01:00
Michael Niedermayer
5cbd7ce016 Merge commit 'f8c507f44b4c994895fc7ad954f009f61de69b1c'
* commit 'f8c507f44b4c994895fc7ad954f009f61de69b1c':
  h264: Refactor ff_h264_decode_ref_pic_list_reordering

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-14 13:24:30 +01:00
Luca Barbato
f8c507f44b h264: Refactor ff_h264_decode_ref_pic_list_reordering
In preparation for MVC support.
2014-02-13 23:34:50 +01:00
Michael Niedermayer
59fb3c4be8 Merge commit '98cc9efc5f6b4dc847ed3b9b60b1c3bcd14e6791'
* commit '98cc9efc5f6b4dc847ed3b9b60b1c3bcd14e6791':
  h264: prevent two matching fields from being both a short/long ref combination

See: 92002db3eb
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 22:41:49 +01:00
Anton Khirnov
98cc9efc5f h264: prevent two matching fields from being both a short/long ref combination
Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-01-18 20:42:39 +01:00