24919 Commits

Author SHA1 Message Date
Michael Niedermayer
70402f6ee7 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e6069636a918c8c04db17e864e3d8151c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
0e71738262 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
ff59edb6dc avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d2715837d125a9cc714fb430ccf6c9d6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:20:09 +01:00
Michael Niedermayer
b4e0acfa04 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:23 +02:00
Michael Niedermayer
d36bb362a6 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1ca26d0fff3a3baf61a40207032143f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:12 +02:00
Michael Niedermayer
be7105dff6 avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864be8fb9653643519dea1c6492f1dde57)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:43 +02:00
Michael Niedermayer
97fdbd12f9 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c506957f40aafd794e06de1a7e341e9d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:40 +02:00
Michael Niedermayer
db48767d7c avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934a29df05c7af65aef2a480b15f572c4)

Conflicts:

	libavcodec/pngdec.c
2014-10-03 18:35:17 +02:00
Michael Niedermayer
ae81d9a7da avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020c2dbf0201fe7df3dad9e7e0b2e1bb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:33:44 +02:00
Michael Niedermayer
abbfc4d87e avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa419aa5d320540d5a1b3f8fd9b82ab7e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:47:32 +02:00
Michael Niedermayer
16775c7aaa avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e)

Conflicts:

	libavcodec/mmvideo.c
2014-10-03 15:10:48 +02:00
Michael Niedermayer
ac82e318bb avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376a736d243aef4a1d121abebce912e6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:29 +02:00
Michael Niedermayer
49d69844f5 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:26:25 +02:00
Michael Niedermayer
3f9a148022 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:25:22 +02:00
Michael Niedermayer
52fd0cda2c avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3b68ebccfe308fa839fc30fa634e4de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:41 +02:00
Gianluigi Tiesi
f238de1990 avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c18c8cf3fe2a4b5cc1ebf2f3300bdfe5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:15 +02:00
James Almer
992ce9777c x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067430de1655a7b28536b5afed48bded5)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:26 +02:00
Michael Niedermayer
beb83f0a40 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd28130bf0dd409b2367ca0f8c9b2417)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 14:37:56 +02:00
Michael Niedermayer
2b415192fd avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba43f64312ae4a05d97afecf1b7b1330c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
6e70816f71 avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37511fbbc98b164c07620b9ebc59ec68)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
d76c9b5665 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474ab201badaae269f3a2cef4824b8c1f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
d4253b3a5b avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c70c8fccdcd56fd19276e668cfb7de8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
9fc7de8d80 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f30525f88692f53134eba41f1f2d90db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Christophe Gisquet
2958b8b86e wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf534a4ead634b4a593c66ebf756910b9b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-19 18:16:23 +02:00
Christophe Gisquet
caf08defa6 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ffc744fedcf77edf2fdb33c964b83370)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 15:45:05 +02:00
Christophe Gisquet
57a6cd8ab1 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff4635c077b2bc8b8d3637d933b6629d803)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:37 +02:00
Christophe Gisquet
b469fce85d proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bccdcfdb41b9f5bbae01d55961bfd0693)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:35 +02:00
Michael Niedermayer
656f930160 avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a16e1b2874bb037a86f317449c58770)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:24:05 +02:00
Anton Khirnov
c56d6f3552 cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e28238d4a4ef1b7e8b42c1cca256da36)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:24:05 +02:00
Michael Niedermayer
65f581472c avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd2643c883522ffa565be4b226ce798c78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00
Michael Niedermayer
c7fac44ef8 avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a593b1c0f8e46fca00ead1d30ee1cd418)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00
Michael Niedermayer
8d48223a32 avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1241de9354a8efdbf8252d0f1a6f933)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:25 +02:00
Michael Niedermayer
01e2dd53a0 cavsdec: fix qp fixed slice handling
Fixes Ticket3400

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0accf24b15ac5a01a67768f41c896ef4e4b8b4a2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:25 +02:00
Michael Niedermayer
b94621ac6b avcodec/alsdec: Clear MPEG4AudioConfig so that no use of uninitialized memory is possible
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e6bd5481cf42a9765c492c77754d4633092cece)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
9fe7e45030 avcodec/aic: fix quantization table permutation
Fixes Ticket3700

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a2004b6d11ff962361420c3150fe760cf1f7115)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
3c169251ee avcodec/mss4: Fix () in MKVAL() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf7ff0146c76b93c32edf5230a28b9590acf5105)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
81a2740919 avcodec/mss34dsp: fix () in SOP* macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e720c5c815e510188a0bda654662383f2c48050)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
a144d305dd avcodec/mlpdec: fix () in MSB_MASK() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa160af08b6f42f17e93124aef86e3f6eec70d51)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
4965da92ef avcodec/ivi_dsp: add some missing () to macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f276bf303cbb7a8fed3c388135007bc29f45f8d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
a83304814c avcodec/h264: fix () in macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit af62b42736c00332d39965168b5cc966a06f07d6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
db6f3be941 avcodec/golomb-test: fix () in EXTEND() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 97e6b5ee3a16fee7d130f19f4dcee030f14d91cf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
2f1f407044 avcodec/aac: fix () in IS_CODEBOOK_UNSIGNED macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa915d4193e13187773c500b80c7df6baeb22c3b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
bdd2f7c7a7 avcodec/libvorbisenc: dont add the duration to AV_NOPTS_VALUE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e66c7232d96e4ae8f05b52da2b84dfaa4e4da3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
62b9c99aec avcodec/diracdec: move mc buffer allocation to per frame
Fixes out of array accesses for non default buffers with large strides

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a30f08505a4e85718896ff233c97be41a9754ca)
(cherry picked from commit 9c9fc79d9237d28e33161cb2e75082d8ad232b2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
d9d66d2d16 avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4329605289e25bb071ec1c1182bf25fc83b09aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
4d9aad95b5 iavcodec/vc1dec: Fix missing {}
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb53beb81a5b9192c79de401f1e1e13fadddc429)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
66ca450fcb avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90e17745791354fbb87899fc4803085a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
8162b69091 avcodec/diracdec: fix undefined behavior with shifts
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8598f6ce61ccda3f2ff0c730b009fb650e42986)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
6f6f5f9c8b avcodec/mjpegen: Fix declared argument size
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 256f530603ef3838a712a4fcd737b46b7bce455e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
2ca68b98ba avcodec/x86/idct_sse2_xvid: fix non C99 inline function
Found-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 46d5625f44185271862337d61cd246fd569c42a4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00