Commit Graph

54995 Commits

Author SHA1 Message Date
Michael Niedermayer
694c3dab36 avcodec/svq1dec: zero terminate embedded message before printing
Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
8efb06c873 avcodec/cook: check that the subpacket sizes fit in block_align
Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
e2865d9316 avcodec/g2meet: check tile dimensions to avoid integer overflow
Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354)

Conflicts:

	libavcodec/g2meet.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
bde9e859b3 avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
85cac770bd avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f)

Conflicts:

	libavcodec/dnxhddec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
808b0ccc03 avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc2)

Conflicts:

	libavcodec/dxa.c
2014-11-02 02:20:10 +01:00
Michael Niedermayer
2cf83677b3 avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680cacee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:10 +01:00
Michael Niedermayer
c0be9d7264 avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
70402f6ee7 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e606)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
9b1673531c postproc: fix qp count
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0b7e5d0d75)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
d7906baa85 postproc/postprocess: fix quant store for fq mode
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 941aaa39e8)

Conflicts:

	tests/ref/fate/filter-pp3
(cherry picked from commit 705748caf3f6a4a3e74ad3d2fc547a5a0213a521)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
322470e606 swresample/swresample: fix sample drop loop end condition
Fixes Ticket3985

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9fefa499f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
0e71738262 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
128b0510e1 avformat/mpegts: Check desc_len / get8() return code
Fixes out of array read
Fixes: signal_sigsegv_844d59_10_signal_sigsegv_a17bb7_366_mpegts_mpeg2video_mp2_dvbsub_topfield.rec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3d7f00ee3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
ff59edb6dc avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
b4e0acfa04 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:23 +02:00
Michael Niedermayer
d36bb362a6 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:12 +02:00
Michael Niedermayer
be7105dff6 avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:43 +02:00
Michael Niedermayer
97fdbd12f9 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:40 +02:00
Michael Niedermayer
db48767d7c avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-10-03 18:35:17 +02:00
Michael Niedermayer
ae81d9a7da avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:33:44 +02:00
Michael Niedermayer
abbfc4d87e avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:47:32 +02:00
Michael Niedermayer
16775c7aaa avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Conflicts:

	libavcodec/mmvideo.c
2014-10-03 15:10:48 +02:00
Michael Niedermayer
ac82e318bb avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:29 +02:00
Michael Niedermayer
49d69844f5 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:26:25 +02:00
Michael Niedermayer
3f9a148022 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:25:22 +02:00
Reimar Döffinger
8622618839 configure: add noexecstack to linker options if supported.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit b7082d953f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-28 12:11:19 +02:00
Michael Niedermayer
52fd0cda2c avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:41 +02:00
lvqcl
2eda0e705a avutil/x86/cpu: fix cpuid sub-leaf selection
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e58fc44649)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 13:58:56 +02:00
Gianluigi Tiesi
f238de1990 avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:15 +02:00
James Almer
992ce9777c x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:26 +02:00
Katerina Barone-Adesi
47fe68eec8 apetag: Fix APE tag size check
The size variable is (correctly) unsigned, but is passed to several functions
which take signed parameters, such as avio_read, sometimes after having
numbers added to it. So ensure that size remains within the bounds that
these functions can handle.

CC: libav-stable@libav.org
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c5560e72d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-17 21:38:26 +02:00
Michael Niedermayer
beb83f0a40 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 14:37:56 +02:00
Michael Niedermayer
2b415192fd avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
6e70816f71 avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
2fdb02693b avformat/swfdec: Do not change the pixel format
This is currently not supported
Fixes part of Ticket 3539

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2430304df)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
d7d29f0d43 avformat/swfdec: Use side data to communicate w/h changes to the decoder
Fixes reading from freed data
Fixes part of Ticket3539

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c55d0ff32)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a9734e7d3017ffc9539eaac2a8acce3ad427f746)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
d76c9b5665 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
d4253b3a5b avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
9fc7de8d80 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Christophe Gisquet
2958b8b86e wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf53)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-19 18:16:23 +02:00
Christophe Gisquet
caf08defa6 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 15:45:05 +02:00
Christophe Gisquet
57a6cd8ab1 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:37 +02:00
Christophe Gisquet
b469fce85d proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bcc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:35 +02:00
Michael Niedermayer
656f930160 avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:24:05 +02:00
Anton Khirnov
c56d6f3552 cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:24:05 +02:00
Michael Niedermayer
04973b02c3 avformat/tee: flip assigment direction
Found-by: CSA
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2e6fdcb7f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:24:05 +02:00
Michael Niedermayer
bcb10b99f4 ffmpeg_opt: Use av_guess_codec() instead of AVOutputFormat->*codec
Fixes part of ticket2236

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 956f4087c6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 03:42:22 +02:00
Michael Niedermayer
65f581472c avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00
Michael Niedermayer
c7fac44ef8 avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00