33569 Commits

Author SHA1 Message Date
James Zern
cb4985ea9a libvpxenc: remove some unused ctrl id mappings
VP8E_UPD_ENTROPY, VP8E_UPD_REFERENCE, VP8E_USE_REFERENCE were removed
from libvpx and the remaining values were never used here

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Zern <jzern@google.com>
(cherry picked from commit 6540fe04a3f9a11ba7084a49b3ee5fa2fc5b32ab)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:28 +01:00
Michael Niedermayer
d27d59fef7 avcodec/vp3: ensure header is parsed successfully before tables
Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:28 +01:00
Michael Niedermayer
ea4d9cb3e5 avcodec/jpeg2000dec: Check bpno in decode_cblk()
Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
e5a4044771 avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
10fc3d690c avcodec/hevc: Fix integer overflow of entry_point_offset
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
0fa92fee43 avcodec/dirac_parser: Check that there is a previous PU before accessing it
Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
cf79fd0317 avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
b253035ab2 avcodec/dirac_parser: Fix potential overflows in pointer checks
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
ded0a04153 avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows
Fixes: 549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_7ffff6ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66e05f6ff5e5c105bdd7bf3a49234ddac1b592c5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
16c5da92d9 avcodec/wmaprodec: Fix overflow of cutoff
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c56f8303e676556ea09bfac73d881c6c9057259)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
73966b0161 avcodec/vp3: Fix "runtime error: left shift of negative value"
Fixes: 5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_7ffff6ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18268f761bffb37552f59f87542fef3d5c80618c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
d295ddffe1 avcodec/apedec: Check length in long_filter_high_3800()
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
aab65146af avcodec/vp3: always set pix_fmt in theora_decode_header()
Fixes assertion failure
Fixes: d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_7ffff6ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a814f1d364ba912adf61adef158168c5f7604e93)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
8cd3def81d avcodec/mpeg4videodec: Check available data before reading custom matrix
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
2fbf723585 avcodec/h264_refs: Check that long references match before use
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa427537b529cd584cd73222980286d36a00fe28)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
7372b42b18 avcodec/utils: Clear dimensions in ff_get_buffer() on failure
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abee0a1c60612e8638640a8a3738fffb65e16dbf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
fe89682d7c avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f03bebc79f76df3a3e5bb9e1bc32baabfb7797c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
ce6dd54711 avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
d32c9723a6 avcodec/vp3: Clear context on reinitialization failure
Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6105b7219a90438deae71b0dc5a034c71ee30fc0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
1cbc2cb508 avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Michael Niedermayer
e348769544 avcodec/hevc_cabac: Fix multiple integer overflows
Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d5028f61e44b7607b6a547f218f7d85217490a5b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:26 +01:00
Michael Niedermayer
9ce96a688b avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit feb3f39614b88c113211a98dda1bc2fe5c3c6957)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:26 +01:00
Michael Niedermayer
9d5bdca3d8 avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75422280fbcdfbe9dc56bde5525b4d8b280f1bc5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:26 +01:00
Michael Niedermayer
31e54f41a0 avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:26 +01:00
Michael Niedermayer
ff3e717003 avcodec/ffv1dec: Clear quant_table_count if its invalid
Fixes deallocation of corrupted pointer
Fixes: 343dfbe142a38b521ed069dc4ea7c03b/signal_sigsegv_421427_4074_ffb11959610278cd40dbc153464aa254.avi
No releases affected

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e04126072e984f8db5db9da9303c89ae01f7d6bb)

Fixes ticket #5052.
2015-12-02 12:03:17 +01:00
Michael Niedermayer
1c6243228c avcodec/ffv1dec: Print an error if the quant table count is invalid
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a8b254e436dce2f5c8c6459108dab4b02cc6b79b)
2015-12-02 11:57:34 +01:00
Michael Niedermayer
4c718691ea avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8000d484b83aafa752d84fbdbfb352ffe0dc64f8)

Conflicts:

	libavcodec/cabac.h
2015-11-27 14:07:03 +01:00
Michael Niedermayer
24c504bd0a avcodec/cabac_functions: Fix "left shift of negative value -31767"
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f6b05f5228979dab0e149deca7a30d22e98af5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 14:03:01 +01:00
Michael Niedermayer
6b91701de3 avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4ea4d2f438c9a7eba37980c9a87be4b34943e4d5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:06:39 +01:00
Michael Niedermayer
71a3113333 avcodec/pngdec: Replace assert by request for sample for unsupported TRNS cases
Fixes assertion failure
Fixes: 7f646252a30ee28b583aac1f82e7985e/signal_sigabrt_7ffff6ae7cc9_7353_62fc077bf2f454d39e188c69807193a6.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a62178be80dd6a643973f62002fc0ed42495c358)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:06:39 +01:00
Michael Niedermayer
2dc1f3a02b avcodec/jpeg2000dec: Fix potential integer overflow with tile dimensions
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65d3359fb366ea265a8468d76a111cb7352f0b55)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:06:39 +01:00
Michael Niedermayer
9a9dda615b avcodec/jpeg2000: Use av_image_check_size() in ff_jpeg2000_init_component()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 016fd413f9168816924f21c0c1ffb578f7226221)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:42 +01:00
Michael Niedermayer
b3bc05290a avcodec/wmaprodec: Check for overread in decode_packet()
Fixes assertion failure
Fixes: 0256e92df2df7e933b43a2c70e4c8040/signal_sigabrt_7ffff6ae7cc9_1358_999ac18684788221490757582ce9af84.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ad698e24e6b9dde57c4e01c145bcddfe9d6e4a3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:42 +01:00
Michael Niedermayer
093e58228e avcodec/smacker: Check that the data size is a multiple of a sample vector
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a9af07a49295e014b059c1ab624c40345af5892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:42 +01:00
Michael Niedermayer
948d93d3f4 avcodec/takdec: Skip last p2 sample (which is unused)
Fixes out of array read
Fixes: cb3f38b08b4541523974667c7d1eee9e/asan_heap-oob_2659e18_9838_021fd5cd635bf76cede6398cd9ecbcdd.tak

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08b520636e96ba6888b669b9b3f4c414631ea1d2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
824e72d9b7 avcodec/dxtory: Fix input size check in dxtory_decode_v1_410()
Fixes potential out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 76b6f4b7d91901929177cc61d9810dcca0bb40c1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
c7a970a3a0 avcodec/dxtory: Fix input size check in dxtory_decode_v1_420()
Fixes out of array read
Fixes: c50c4aa6cefda71b19a31ea12302980c/asan_heap-oob_12be5fd_7011_33ebd015a74976215934add72b9c8352.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9caa9414ccf2dcf8aee2695377dee830a5024c82)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
6af38b2718 avcodec/error_resilience: avoid accessing previous or next frames tables beyond height
The height of tables can be rounded up for MBAFF but this does not imply that is also true
for the previous frames

Fixes out of array reads
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a105f52855d08e4ab1ed7306da8e32fc90d6d647)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
0ce7baa245 avcodec/dpx: Move need_align to act per line
Fixes out of array read
Fixes: 61cf123c081ee2bb774d307c75bdb99e/asan_heap-oob_1224f76_5546_bee833ffae73f752b489b9eeaac52db7.dpx

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8aaae8e0f1519bc99bd717ea3067c9cfdb68def)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
7f0b58947d avcodec/flashsv: Check size before updating it
Fixes out of array read
Fixes: 3c857d4d90365731524716e6d051e43a/signal_sigsegv_7f4f59bcc29e_1386_20abd2c8e655cb9c75b24368e65fe3b1.flv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 17705f5d4f57c15f9b9bb9cfcbbb4621fed2fc70)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
f64ffbbdc0 avcodec/ivi: Check image dimensions
Fixes integer overflow
Fixes: 1e32c6c591d940337c20b197ec1c4d3d/asan_heap-oob_4a52e5_8946_0bb0d9e863def56005e49f1d89bdc94d.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df91aa034b82b77a3c4e01791f4a2b2ff6c82066)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
c0748b9954 avcodec/utils: Better check for channels in av_get_audio_frame_duration()
Fixes integer overflow
Fixes: 0c2625f236ced104d402b4a03c0d65c7/asan_generic_274e1ce_5990_9314e7a67c26aecf011b178ade9f217c.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4e16ad2868a1819de6680fc355a8eb20164adaea)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Michael Niedermayer
dbfec68d32 avcodec/jpeg2000dec: Check for duplicate SIZ marker
Fixes: 0231a17345734228011c6f35a64e4594/asan_heap-oob_1d92a72_3218_1213809a9e3affec77e4c191fdfdc0a9.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44a7f17d0b20e6f8d836b2957e3e357b639f19a2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 16:05:41 +01:00
Andreas Cadhalpun
34f2d74555 aacsbr: don't call sbr_dequant twice without intermediate read_sbr_data
Doing that doesn't make sense, because the only purpose of sbr_dequant
is to process the data from read_sbr_data.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1c3e43a6273822e1369818b80f34c8464e1009d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Andreas Cadhalpun
22017f7745 hqx: correct type and size check of info_offset
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.

Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1ed7fcd42af956979abf4e32cd3c9ee17622bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Michael Niedermayer
bdbfc12e7f avcodec/jpeg2000: Change coord to 32bit to support larger than 32k width or height
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0eb7de19736891a9386ab66549f780e904a3b6a7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Michael Niedermayer
c3a44a2a55 avcodec/jpeg2000dec: Check SIZ dimensions to be within the supported range
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6ef819c40bcc2175edba7ce9e20c3036c01b36b9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Michael Niedermayer
d1d48d4319 avcodec/jpeg2000: Check comp coords to be within the supported size
Fixes assertion failure
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1a8cbcb35ef2759a66b4f0875785e4b3f277057)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Andreas Cadhalpun
6b0bc64f54 mpegvideo: clear overread in clear_context
Otherwise the h263p decoder can try to copy overread bytes, even though
buffer is NULL.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 6a69a175e7b5c5393528ed0f5753e41573fa0df2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00
Michael Niedermayer
983e63b490 avcodec/avrndec: Use the AVFrame format instead of the context
Fixes out of array read
Fixes: 20dd01398dee0f6d83d7e5410a2ae8eb/signal_sigsegv_39eeb1f_4001_62efbdf1c60748dabf1ec310b59525fd.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ccba8aaff2ef5649495ae48bc5c90bd8ff32e6f3)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-11-26 01:35:46 +01:00