19317 Commits

Author SHA1 Message Date
Michael Niedermayer
bc07ec7b2f avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c91869f4e5cdacd8d6be52f8b89d4ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
3a893335bd avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2ecfc47d796382973ba0e48b8f6f92a2)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e65098fd2ccd8456f667151dfc9cda42)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
c23597ec7e avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d09b64bf112c2bf7a34b1ef3b93dbd3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
b66a0dce11 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e9354772de446e8196dabf9aad1567b22f74d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
c598a46821 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa7fc7d38db582e75b8b093fb3c1608d)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9acfaa934e8feb8fba90765c9d0b989)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
912b7fe922 dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61310beba705957e6366f0392a0b005b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
84c84204b3 avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b55096361723b364afa43b79a3f5619cdc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
82746c468b avcodec/avpacket/av_packet_split_side_data: ensure that side data padding is initialized
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 240fd8c96f59ebe9dcfc4152a1086cd3f63400c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e48318802b3caa493a40c0584afc30cc866d9d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
2289bffa31 avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9abfb2ae75bb32bc44e2c34194c8dc40a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
b650d5376c avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2014-03-08 15:32:43 +01:00
Michael Niedermayer
ec39abcd37 avcodec/truemotion2: Fix av_freep arguments
Fixes null pointer dereference
Fixes Ticket2944

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c54aa2fb0f869ec025933944cbd1634fffe95d09)

Conflicts:

	libavcodec/truemotion2.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
66fcda7e70 avcodec/mjpegdec: Add some sanity checks to ljpeg_decode_rgb_scan()
These prevent the rgb ljpeg code from being run on parameters that it doesnt
support. No testcase available but it seems possible to trigger these.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61c68000eda643dfce96dc46b488d39fd5c4e309)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
fdf57ad2ff avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Martin Storsjö
69f724a538 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)
2013-12-21 10:00:18 +01:00
Michael Niedermayer
929100ae98 avcodec/cabac: force get_cabac to be not inlined
works around bug in gccs inline asm register assignment
Fixes Ticket3177

gcc from 4.4 to 4.6 is affected at least, no non affected gccs known
clang seems not affected

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0538b29ae8002c44f27bae8a1a6fc6e646998be5)
2013-12-09 10:40:08 +01:00
Michael Niedermayer
d49761b396 avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a89761964bdd0a023eff6d37a1131fb7e1d7a0)

Conflicts:
	libavcodec/h264_refs.c
2013-10-26 01:26:00 +02:00
Michael Niedermayer
87f719a03f avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f17b64723884c9fa72cfcbd369a1ba2)

Conflicts:
	libavcodec/h264_refs.c
(cherry picked from commit 6636dd551fda4fac77f2caa25d24d81abcadcd71)
2013-10-24 11:40:01 +02:00
Carl Eugen Hoyos
a5ef62ede1 Avoid a deadlock when decoding wma.
Fixes ticket #2925.
(cherry picked from commit ec8a4841f7e81040f9a2757f23e70dff5e6b33a4)
2013-09-02 10:03:49 +02:00
Michael Niedermayer
2d945ac68f avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6a527d8bc807d09b93f971c0fe0bb07)
2013-08-30 23:42:00 +02:00
Michael Niedermayer
1934bb7536 h264: skip error concealment when SPS and slices are mismatching
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 695af8eed642ff0104834495652d1ee784a4c14d)

Conflicts:

	libavcodec/h264.c
2013-08-25 21:05:59 +02:00
Michael Niedermayer
39ed544262 Merge commit '072be3e8969f24113d599444be4d6a0ed04a6602'
* commit '072be3e8969f24113d599444be4d6a0ed04a6602':
  h264: set parameters from SPS whenever it changes

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9d887c225466576ae80ef7f2b109e866ff137b2)

Conflicts:

	libavcodec/h264.c
2013-08-25 14:49:11 +02:00
Michael Niedermayer
f9e6fb50ce jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 13:11:53 +02:00
Michael Niedermayer
453e2f1528 avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:49 +02:00
Michael Niedermayer
b25c3063b2 avcodec/flashsv: check diff_start/height
Fixes out of array accesses
Fixes Ticket2844

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880c73cd76109697447fbfbaa8e5ee5683309446)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:49 +02:00
Michael Niedermayer
29b14db916 update all trac links to use the trac subdomain
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:48 +02:00
Michael Niedermayer
d94d383f13 xbmdec: fix off by one error in scanf()
Fixes out of array access

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 88c1b0e7a852e48d9f0e3d79c44edaa86e59acfe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:48 +02:00
Michael Niedermayer
2a8c3a7895 avcodec/kmvc: fix MV checks
Fixes Ticket2813
Fixes regression since 70b5583

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3cd8aaa2b2e78faf039691e1c31ff4f8d94e3bc6)
2013-07-31 03:03:05 +02:00
Michael Niedermayer
c9f34c8260 mpeg12dec: avoid reinitialization on PS changes when possible.
Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
	libavcodec/mpeg12dec.c
2013-07-09 00:48:42 +02:00
Hendrik Leppkes
274ec187dc mathops/x86: work around inline asm miscompilation with GCC 4.8.1
The volatile is not required here, and prevents a miscompilation with GCC
4.8.1 when building on x86 with --cpu=i686

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 659df32a9d8984081ccd54adc3aee7daeb33388d)
2013-06-24 08:48:31 +02:00
Michael Niedermayer
7a472e0da9 alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)
2013-06-13 00:11:02 +02:00
Claudio Freire
f6bca606f1 AAC encoder: Fix rate control on twoloop.
Fixes a case where multichannel bitrate isn't accurately
targetted by psy model alone, never achieving the target bitrate.
Now fixed.

Fixes ticket #2625.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
2013-06-02 16:30:15 +02:00
Michael Niedermayer
5fd83f29f7 h264_cavlc: fix reading skip run
Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
	libavcodec/h264_cavlc.c
2013-05-30 23:22:00 +02:00
Michael Niedermayer
1b0028a3c5 smacker: remove av_clip_int16()
Fixes Ticket2425

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2211c76287e073a9e176fde7dbb9a63ceb2af8d1)
2013-05-21 00:03:44 +02:00
Carl Eugen Hoyos
7dd042e657 Fix type of shared flac table ff_flac_blocksize_table[].
Fixes ticket #2533.
(cherry picked from commit a07ac1f7888fd08e42da2bed0421e74f1cfac177)
2013-05-05 20:39:45 +02:00
Michael Niedermayer
fff5f65540 Merge remote-tracking branch 'jamrial/release/0.11' into release/0.11
* jamrial/release/0.11:
  lavc/bink: Chech for malloc failure

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-20 22:10:40 +01:00
Alexander Kojevnikov
7cb24ecd4c mp3dec: Fix VBR bit rate parsing
When parsing the Xing/Info tag, don't set the bit rate if it's an Info tag.

When parsing the stream, don't override the bit rate if it's already set,
otherwise calculate the mean bit rate from parsed frames. This way, the bit
rate will be set correctly both for CBR and VBR streams.

Signed-off-by: Alexander Kojevnikov <alexander@kojevnikov.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 29d8cd265a536063420afe78375b2176a9e1abc5)

Conflicts:

	tests/ref/lavf-fate/mp3

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-20 22:02:28 +01:00
Michael Niedermayer
864a7e73b9 huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
2013-03-20 22:02:28 +01:00
Michael Niedermayer
562aa82d2a huffyuvdec: Check init_vlc() return codes.
Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

	libavcodec/huffyuv.c
2013-03-20 22:02:28 +01:00
James Almer
c095137d1b lavc/bink: Chech for malloc failure
Based on commit 8ab2173ed141aa2c3336be7f9880340dfb8dcf5e
2013-03-20 17:51:15 -03:00
Carl Eugen Hoyos
b63dbe2220 Do not (re-)set libx264 parameter b_tff if interlaced encoding was not requested.
Reconfiguring can break x264 lossless encoding.

Fixes ticket #2165.
(cherry picked from commit 75c7e4583f4fd727d236a12763a265502fe00988)
2013-03-18 02:19:13 +01:00
Michael Niedermayer
5df2dc0f94 aacsbr: Check for envelope scalefactors overflowing
This prevents various values from becoming stuck at NAN and
output to become silent
If someone knows a cleaner solution, thats welcome!

Fixes Ticket2335

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8978c743fb1d1f5a0d6dbdd83ff05817f8a41230)
2013-03-08 20:09:20 +01:00
Michael Niedermayer
ebe645f02b h264: Reset last_pocs in case of reference or frame number inconsistencies
This prevents faulty increasing of has_b_frames
Should fix Ticket 2062

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c230af9bccc3cadb373f9007ba14fffb6c2acc75)
2013-02-14 09:24:11 +01:00
Michael Niedermayer
470ee0c660 h264_refs: Print default in case we are missing a reference.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a39d36146a40cb52b3560fd02c73eccf72603b8f)
2013-02-14 09:24:08 +01:00
Piotr Bandurski
d338632e9f tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2b0fe4d1b4c6256bd37265fc45aed3d)
2013-01-07 00:29:28 +01:00
Clément Bœsch
8bc0127b3d lavc/ass_split: check for NULL pointer in ff_ass_split_override_codes().
This is consistent with the other ff_ass_split_* functions.

It also fixes a crash when trying to split a dialog with text=NULL
(which seems to happen when the text of the dialog is empty); basically,
this commit fixes crashes when trying to encode an empty text subtitle
dialog (see subrip and mov_text encoders).

Fixes Ticket2048.
(cherry picked from commit c83002a4f8042ccfa0688a9a18e8fa0369c1fda8)
2013-01-01 18:19:50 +01:00
Michael Niedermayer
5f0e5b4048 mpeg1video: fix regression with slices != threads
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a01679586cd9ac8470b81c0299fc7e13fd980d64)
2012-12-13 00:21:18 +01:00
Michael Niedermayer
e9ded2e1a9 mpeg1video: support multi threaded slice encoding.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03df9720168335482f00898fc16b56ff4878d0e7)

Conflicts:
	libavcodec/mpeg12enc.c
2012-12-13 00:21:09 +01:00
Janne Grunau
69cc119d64 h264: slice-mt: get last_pic_dropable from master context
Fixes fate-h264-conformance-cvnlfi2_sony_h and smllwebdl.mkv from
https://github.com/OpenELEC/OpenELEC.tv/issues/1557 .
(cherry picked from commit 24c62ea7a5df44804be88150aa0c45e6796b5da9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-06 16:44:49 +01:00
Michael Niedermayer
54ac3d10d1 mp3dec: Fix possibly exploitable crash
I was sadly unable to find a non fuzzed mp3 that uses the
feature that contained the bug (and i searched hard ...), thus
while this fixes the security issue. It may or may not fix
mixed blocks in 8khz mp3s, i cant say due to lack of samples to test.

Security issue exists since: b37d945dd4213cb8e92146571b0374cd45d52286

Reported-by: Dale Curtis <dalecurtis@google.com>
(Probably) Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 94041febc53a6da10e34c2bfff9ff1d580fdce60)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-06 16:28:27 +01:00
Paul B Mahol
99dbda3008 aasc: fix out of array write
Closes #1619.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 8a57ca5c6a1c0ad28afa7ea6f824981e6761cce1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-09-15 00:21:58 +02:00