Commit Graph

26101 Commits

Author SHA1 Message Date
Michael Niedermayer
4352a971af avcodec/h264_slice: Clear table pointers to avoid stale pointers
Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce9585)

Conflicts:

	libavcodec/h264_slice.c
2014-11-28 18:53:11 +01:00
Michael Niedermayer
bc0a6add0a avcodec/svq1dec: zero terminate embedded message before printing
Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
3467dfed6b avcodec/cook: check that the subpacket sizes fit in block_align
Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
ffd5ccee5d avcodec/g2meet: check tile dimensions to avoid integer overflow
Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354)

Conflicts:

	libavcodec/g2meet.c
2014-11-28 18:53:11 +01:00
Michael Niedermayer
a4b6d5574f avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
a2619a0a36 avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f)

Conflicts:

	libavcodec/dnxhddec.c
2014-11-28 18:53:11 +01:00
Michael Niedermayer
8c5c45b9e1 avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
dc2f4b7267 avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680cacee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
824f29e4fb avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Michael Niedermayer
917946e1c9 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e606)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:11 +01:00
Christophe Gisquet
3fd2ff1b4f utvideoenc: properly set slice height/last line
Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer
f335275c50 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer
21808e2181 avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer
bf2605c356 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
c10c71452f avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
05d7e92e4f avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
96357894ff avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
4841b2759f avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-11-28 18:44:38 +01:00
Michael Niedermayer
5d852f338d avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
ceb9d67a0a avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
ce9d497755 avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
b5298c464f avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
9ec550c364 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
2014-11-28 18:44:38 +01:00
Michael Niedermayer
23a22b0da9 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer
c658f6c34d avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Pascal Massimino
77367f2728 avcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Gianluigi Tiesi
34ef754854 avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
James Almer
84487650e2 x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Pascal Massimino
5ecc4a644a libavcodec/webp: treat out-of-bound palette index as translucent black
See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
cd190f0c3f avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
31f5d5f690 avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
7644b29222 avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
f7086be79a avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
8c8950f982 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
cb8f645faf avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
6ab793c2b9 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet
80b6632b36 wavpackenc: proper buffer allocation
The allocation didn't account for headers, that can be easily 79 bytes.
As a result, buffers allocated for a few samples (e.g. 5 in the original
bug) could be undersized.

Fixed ticket #2881.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2ba58bec20)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet
656bf0ca79 wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf53)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet
407982b8f9 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet
92096acc0a proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet
e1ed566c1c proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bcc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
c9da441eb4 avcodec/snow: fix null pointer dereference in cleanup after allocation failure
Fixes: snowf.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a162146ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
f405267493 avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Anton Khirnov
f7ed48938a cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer
7d69132775 avcodec/wavpackenc: Fix log2sample() result value
Found-by: CSA
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e706fe7640)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:36 +01:00
Michael Niedermayer
384be84bbe avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:36 +01:00
Michael Niedermayer
f99675627d avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:36 +01:00
Michael Niedermayer
d5eca1651f avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:36 +01:00
Alessandro Ghedini
aa09543659 vc1: Do not return an error when skipping b frames
This caused mpv (and possibly others) to fallback to software decoding after
seeking a VC1 stream.

Bug-Id: 667

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit cdf6eb5a97)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-23 10:51:53 +02:00
Michael Niedermayer
b7638af942 avcodec/hevc: treat current_sps like sps_list
This simplifies the management of current_sps
Fixes Ticket3458

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880dbe43ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 30b6e9377d3b0b1341147d8484fce24301e7b8a0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 22:19:27 +02:00