14796 Commits

Author SHA1 Message Date
Laurent Aimar
60a1384013 Avoid NULL dereference on corrupted bitstream with real decoder.
rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 140dbcff357351570426efd0e0bc00e380f5436e)
2011-09-21 20:56:53 +02:00
Laurent Aimar
b59919afe2 Reject slices that does not have the same type than the first one in RV10/RV20 decoder.
This prevents crashes with some corrupted bitstreams.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d788af6cf696d547a442c47e1ce6f93bc9fc97b6)
2011-09-21 20:56:53 +02:00
Michael Niedermayer
764ffdd0ec check all svq3_get_ue_golomb() returns.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 979bea13003ef489d95d2538ac2fb1c26c6f103b)
2011-09-21 20:56:53 +02:00
Michael Niedermayer
ed9e561490 rv34: check for size mismatch
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35f38b3ab9d755aede5bce8abbe1cb9c07027f8a)
2011-09-21 20:56:53 +02:00
Laurent Aimar
4d8330d095 Fix js_vlc_bits value validation when joint stereo is used in cook decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 352c878de6fca57a3f2a6369e27ca442fa5f9518)
2011-09-21 19:50:08 +02:00
Laurent Aimar
30d7dce94f Fix potential overreads in rv34 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a0a64cb26bc6aaa7f29106bded8a27f3db918ee)
2011-09-21 19:50:03 +02:00
Michael Niedermayer
fa3f7391be h264: allow disabling bitstream overread protection by using the fast flag.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205c13685f36a17f406b7cd07e14b6d2a61f396a)
2011-09-21 19:49:30 +02:00
Alex Converse
b7000d0517 xan: Add some buffer checks
(cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
2011-09-21 19:47:12 +02:00
Alex Converse
169e634457 xan: Remove extra trailing newline
(cherry picked from commit 350f57bd7b77b36c150540ad2aae9de42cc4dfca)
2011-09-21 19:47:06 +02:00
Laurent Aimar
053bc4ce8b Fixed size given to init_get_bits() in xan decoder.
(cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
2011-09-21 19:47:00 +02:00
Michael Niedermayer
e952ff6981 smacker: fix a few off by 1 errors
stereo & 16bit is untested due to lack of samples

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d67e74929c2c0d09383c2f4a9c904bfdd839c67c)
2011-09-21 19:46:34 +02:00
Michael Niedermayer
9cee26dfde smacker: add forgotten *
found by fenrir

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f98edc73c599badaa0c075fbffb519a150d03d80)
2011-09-21 19:46:23 +02:00
Kostya Shishkov
ad6177e52c rv34: check that subsequent slices have the same type as first one.
This prevents some crashes when corrupted bitstream reports e.g. P-type
slice in I-frame. Official RealVideo decoder demands all slices to be
of the same type too.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 23a1f0c59241465ba30103388029a7afc0ead909)
2011-09-21 19:36:53 +02:00
Laurent Aimar
c9316b7c6d Fixed invalid read access on extra data in cinepak decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc255275f6293a060518271a151e1ce75499e874)
2011-09-21 18:02:40 +02:00
Laurent Aimar
2bf9a09a2c Fixed segfaults on corruped smacker streams in the decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d07ac1853da29ea696243160e02154ebf758d1ee)
2011-09-21 18:02:29 +02:00
Michael Niedermayer
54544100a3 h264: prevent an out of array read in decode_nal_units()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea0ac11e52b9cf5264f3d4eb7543e760c2a5fbb4)
2011-09-21 18:02:18 +02:00
Michael Niedermayer
97437dada6 h264dec: Prevent CABAC and CAVLC bitsteram overreading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23f5cff92cdcfa55a735c458fcb5f95c0e0f3b1f)
2011-09-21 18:02:13 +02:00
Art Clarke
c8736de331 libspeex encoder wraper
taken from svn head of xuggle
(cherry picked from commit a52cdcd296c40882c3b4f88958990c56f0ce3019)
2011-09-21 18:01:25 +02:00
Joakim Plate
92f1b5df32 dvbsubdec: don't hardcode subtitle colors count in dvbsubdec to 16
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a3294ef0069c898a495c783dfdb2f0bf1dc6583)
2011-09-21 18:01:20 +02:00
Laurent Aimar
82e4fd193f Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8bfea4ab4e2cb32bc7bf6f697ee30a238c65d296)
2011-09-21 18:01:13 +02:00
Alex Converse
3a0649ddeb cljr: init_get_bits size in bits instead of bytes
(cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
2011-09-21 18:01:09 +02:00
Alex Converse
9f05400ea8 indeo2: fail if input buffer too small
(cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
2011-09-21 18:01:02 +02:00
Alex Converse
09cfd6f597 indeo2: init_get_bits size in bits instead of bytes
(cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
2011-09-21 18:00:54 +02:00
Michael Niedermayer
b2af83a9ed cabac test: Change input to test, so a wider range of states is tested.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1eb805ed70a0aac35523c7a308145b6b8d05441e)
2011-09-21 18:00:40 +02:00
Michael Niedermayer
f38b2a6be8 cabac test: match encode and decode side
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 830d7d5c4fbf81265ca5260d59fe8467ac97b2e6)
2011-09-21 18:00:36 +02:00
Michael Niedermayer
db93a5a0c8 cabac: fix cabac encoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 89653ea728bd59da2b88967ad648cce685950879)
2011-09-21 18:00:18 +02:00
Laurent Aimar
b5fe6bee01 Fixed deference of NULL pointer in motionpixels decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 824f98f442996eaee9204b132752cf5114fc94cf)
2011-09-21 18:00:10 +02:00
Laurent Aimar
7181adab80 Fixed size given to init_get_bits().
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5e0580b93a5bda34f62a5df50c1b15e610d4ad1)
2011-09-21 17:59:48 +02:00
Laurent Aimar
bac822025e Fixed size given to init_get_bits() in ffv1 decoder.
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8362a0ffed0b15a9977fcde947bcb49dfc06638b)
2011-09-21 17:59:43 +02:00
Alex Converse
8a8aafd2b9 wavpack: Check error codes rather than working around error conditions.
(cherry picked from commit dba2b63a98bdcac7bda1a8a2c48950518c075e17)
2011-09-21 17:59:36 +02:00
Michael Niedermayer
a13ef61051 rc: finetune convergence failure fix
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 73e0ec2ff453799e361c67395b93df11986609b3)
2011-09-21 17:59:30 +02:00
Michael Niedermayer
4fbc35cd53 rc: fix convergence failure
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed14517c234299387d401959c6037f25e0937f02)
2011-09-21 17:59:25 +02:00
Laurent Aimar
558cf502ac Fixed invalid writes in wavpack decoder on corrupted bitstreams.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0aedab03405849962b469277afe047aa2c61a87f)
2011-09-21 17:57:33 +02:00
Chris Rankin
b0da6a744a qcelpdec: fix the return value of qcelp_decode_frame().
(cherry picked from commit 04c13dca8812e8302686887b6e8201d4ad25b7d8)
2011-09-21 17:57:01 +02:00
Michael Niedermayer
d99613bad6 jpeglsdec: fix infinite loop
Fixes Ticket331

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bd358e128f47c14c7bc9dfceb8dedb3e5697f017)
2011-09-21 17:57:01 +02:00
Kostya Shishkov
c026f336b9 wavpack: fix wrong return value in wavpack_decode_block()
This function should return number of samples decoded, not number of bytes
decoded.
Spotted by Uoti Urpala.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bcd4aa8bec19a17bd633e3db13357784c05b6ca5)
2011-09-21 17:56:15 +02:00
Reimar Döffinger
5c2d684986 Check extradata size on resolution change.
Ignore resolution change if resolution not defined in extradata.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 09c5f990bc7629dfbee8c760fd485936c60a7b40)
2011-09-21 17:56:15 +02:00
Gavin Kinsey
30442fa217 jpegdec: set color_range
(cherry picked from commit 2f870e262e953815060de0ebea9b6653133b203e)
2011-09-21 17:56:15 +02:00
Michael Niedermayer
e7d10f5a90 mpeg4: fix typo in mpeg4_encode_gop_header()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f5bda9fcbb54c9c27503ab1bdb86838eb6f602f1)
2011-09-21 17:56:15 +02:00
Michael Niedermayer
ca5dfd1550 h264: clean all non null elements of delayed_pic[]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 66ce282df54edb33515d9ef3015b737428b6e0c3)
2011-09-21 17:56:14 +02:00
Michael Niedermayer
1979a9b4f2 h264: change MAX_DELAYED_PIC_COUNT check to av_assert0
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b955ab2f492e3b8c4cbf614f68f8d5bf79e1d1e1)
2011-09-21 17:56:14 +02:00
Laurent Aimar
d805b8f454 rv34: Check for invalid slice offsets
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4cc7732386eb36661ed22d1200339b38a5fa60bc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:40:36 +02:00
Laurent Aimar
a01387bb35 rv34: Fix potential overreads
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b4ed3d78cb6c41c9d3ee5918c326ab925edd6a89)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:40:36 +02:00
Laurent Aimar
11b72c073c rv34: Avoid NULL dereference on corrupted bitstream
rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d0f6ab0298f2309c6104626787ed73416298b019)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:40:36 +02:00
Laurent Aimar
bb6702f206 rv10: Reject slices that does not have the same type as the first one
This prevents crashes with some corrupted bitstreams.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4a29b471869353c3077fb4b25b6518eb1047afb7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:40:34 +02:00
Laurent Aimar
6ddb12b688 Fixed size given to init_get_bits().
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit b59efc94347ccf0cbc2ff14a5a9e99819c5bdc4d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:28:31 +02:00
Michael Niedermayer
c34968c6d4 smacker: fix a few off by 1 errors
stereo & 16bit is untested due to lack of samples

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5166376f24545207607f61ed8ff4e1b0572ff320)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:28:20 +02:00
Laurent Aimar
a5107aab98 Check for invalid VLC value in smacker decoder.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 6489455495fc5bfbebcfe3f57e5d4fdd6a781091)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:28:14 +02:00
Laurent Aimar
bc2dd37e4f Check and propagate errors when VLC trees cannot be built in smacker decoder.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9676ffba8346791f494451e68d2a3b37a2918a9b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-21 14:28:10 +02:00
Alex Converse
dc6ee18363 xan: Add some buffer checks
(cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-13 17:25:35 +02:00