14736 Commits

Author SHA1 Message Date
Laurent Aimar
1b26a734b2 Fix potential pointer arithmetic overflows in rle_unpack() of vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35cb6854bb76b4a5b6f2aea2dce81e18d7ab61cd)
2011-10-01 20:25:21 +02:00
Laurent Aimar
02bdeff1ef Fix out of bound reads in rle_unpack() of vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4749e074987d45cb98935a683a7ee8e1ee376d86)
2011-10-01 20:25:16 +02:00
Laurent Aimar
55efeba2b5 Check for out of bound reads in vmd_decode() of vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e07377e7366d038d6f28792c46745463a0542650)
2011-10-01 20:25:10 +02:00
Laurent Aimar
08657a2a8a Fix potential pointer arithmetic overflows in lz_unpack of vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 78cb39d2b2ad731dd3b984b0c0711b9f1d6de004)
2011-10-01 20:24:57 +02:00
Laurent Aimar
f40b04e917 Prevent out of bound read in lz_unpack in vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5127f465bd3e2cf9cbf66dea3cf7b481b522d266)
2011-10-01 20:24:52 +02:00
Laurent Aimar
d92bfc98f9 Prevent NULL dereferences when the previous frame is missing in vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a6383bebcb03a785797007031ad1c9786a508a5)
2011-10-01 20:24:46 +02:00
Laurent Aimar
1ed90c84f6 Check for invalid update parameters in vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7aed1280ea14b60fceae04d71dfd03e1daf2d04)
2011-10-01 20:24:39 +02:00
Laurent Aimar
21c9d92646 Fix potential overread in vmd audio decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 00cbe9e4053fd562b6f21e76aca6636ff926b637)
2011-10-01 20:24:31 +02:00
Laurent Aimar
be22dc60f5 vp56:Fix error recovery code on size changes in vp5/6 decoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1aad9cd9d26adcca1d398a4cb3d67f89a6fa8967)
2011-10-01 20:23:03 +02:00
Laurent Aimar
35f8ad420a vp6:Reset the internal state when aborting key frames header parsing in vp6 decoder.
It prevents leaving the state only half initialized.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 91f104496bb7632ed5ff03798e06dd8af014f0d9)
2011-10-01 20:22:52 +02:00
Michael Niedermayer
f71c761a9e h264: pass buffer & size to ff_h264_decode_extradata()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 20:11:00 +02:00
Laurent Aimar
101e38e08a h264: Check for out of bounds reads in ff_h264_decode_extradata().
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57764c699671b32f0c70795feafc0c6d7491f198)
2011-10-01 19:54:49 +02:00
Sean McGovern
1cf6348cf7 fft: avoid a signed overflow
As a signed integer, 1<<31 overflows, so force it to unsigned.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit c2d3f561072132044114588a5f56b8e1974a2af7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 19:50:22 +02:00
Jean First
8c0a0f10df tiffenc: initialize forgotten avctx.
(cherry picked from commit f7e797aa5c987c39b55666a2d41877ef2aec40bc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 19:49:11 +02:00
Jean First
92566cf6ee tiffenc: Add forgotten avclass to context.
(cherry picked from commit 43c481e56929789883f5b078e0e86d50fcc0025e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 19:49:05 +02:00
Michael Niedermayer
03e7314dd8 aacsbr: add a assert0 to check for a inconsistency that
occurd during debug. I dont know if this can happen normally but if so
it would be quite bad.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit abe0dbea2e228621e97184e39159d189b6085fe3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 19:48:34 +02:00
Michael Niedermayer
0d68a6f72d mpeg4videoenc: remove forgotten return -1
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9bb7395a10ae44eb0f0f3f070f23124dfaee06e)
2011-10-01 19:30:31 +02:00
Michael Niedermayer
a0acc9eff6 mpeg4videoenc: guess a good aspect when we cant store the exact one.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 394781a89706479aa56749a9d69c4e74b398dd71)
2011-10-01 19:30:06 +02:00
Michael Niedermayer
e62ca1ab74 mpegvideo: increase emu edge buffer size
This fixes a crash with 422 H.264

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7322483d72d4abefae9f5c08c611f521de7236a5)
2011-10-01 01:32:23 +02:00
Michael Niedermayer
ceede3a802 h264: fix FIXME and use list_count in ff_h264_fill_mbaff_ref_list()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 237d31e0b98b95eef687e612177ca3ea24b709fc)
2011-09-28 23:36:54 +02:00
Michael Niedermayer
be9183de2e h264: More correct ref_count check in decode_slice_header()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc9ce40069bde3d28f8d0b3e5bd733ae255fecb5)
2011-09-28 23:36:39 +02:00
Sascha Sommer
9960710b87 Fix segfault in save_bits:
use put_bits_count to get the buffer fill state instead of
num_saved_bits as num_saved_bits is sometimes reset when
frames are lost
(Ticket 495)
(cherry picked from commit 780d45473c32fa356c8ce385c3ea4692567c3228)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4f6187c7356111540024901932294e9807061dd0)
2011-09-27 03:06:04 +02:00
Mans Rullgard
ecda54a640 ppc: fix 32-bit PIC build
On 32-bit ppc, the GOT pointer must be loaded manually.
This adds a "get_got" assembler macro to compute the
GOT address.  The "movrel" macro is updated to take an
additional parameter containing the GOT address since
no register is reserved for this purpose on ppc32.
These changes have no effect on ppc64 builds.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 6e4a35ced96cdf31a9d3bd82fd147554750af839)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-09-26 19:31:32 +02:00
Laurent Aimar
2bbb142a14 rv34: Check for invalid slice offsets
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4cc7732386eb36661ed22d1200339b38a5fa60bc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:33 +02:00
Laurent Aimar
b4a1bf0bbf rv34: Fix potential overreads
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b4ed3d78cb6c41c9d3ee5918c326ab925edd6a89)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:33 +02:00
Laurent Aimar
f0bcba238a rv34: Avoid NULL dereference on corrupted bitstream
rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d0f6ab0298f2309c6104626787ed73416298b019)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:33 +02:00
Laurent Aimar
28d948ac44 rv10: Reject slices that does not have the same type as the first one
This prevents crashes with some corrupted bitstreams.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4a29b471869353c3077fb4b25b6518eb1047afb7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:33 +02:00
Laurent Aimar
54a178f28f Fixed size given to init_get_bits().
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit b59efc94347ccf0cbc2ff14a5a9e99819c5bdc4d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Michael Niedermayer
78cd2e18a4 smacker: fix a few off by 1 errors
stereo & 16bit is untested due to lack of samples

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5166376f24545207607f61ed8ff4e1b0572ff320)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Laurent Aimar
0d93b03e68 Check for invalid VLC value in smacker decoder.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 6489455495fc5bfbebcfe3f57e5d4fdd6a781091)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Laurent Aimar
5b1f79b092 Check and propagate errors when VLC trees cannot be built in smacker decoder.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9676ffba8346791f494451e68d2a3b37a2918a9b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Alex Converse
61ddc8271d xan: Add some buffer checks
(cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Laurent Aimar
e6694dce1c Fixed size given to init_get_bits() in xan decoder.
(cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Laurent Aimar
9b30b7b9bf Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8bfea4ab4e2cb32bc7bf6f697ee30a238c65d296)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Alex Converse
384ed15c2a cljr: init_get_bits size in bits instead of bytes
(cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:32 +02:00
Alex Converse
6550e2b5c5 indeo2: fail if input buffer too small
(cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Alex Converse
af32fa929a indeo2: init_get_bits size in bits instead of bytes
(cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Laurent Aimar
07b3c4cde5 ffv1: Fixed size given to init_get_bits() in decoder.
init_get_bits() takes a number of bits and not a number of bytes as
its size argument.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 46b004959bb7870a361a57272cd5fa7eea34250b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Alex Converse
5d4c065476 wavpack: Check error codes rather than working around error conditions.
(cherry picked from commit dba2b63a98bdcac7bda1a8a2c48950518c075e17)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Laurent Aimar
4b84e995ad Fixed invalid access in wavpack decoder on corrupted bitstream.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 55354b7de21e7bb4bbeb1c12ff55ea17f807c70c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Laurent Aimar
685940da4c Fixed invalid writes in wavpack decoder on corrupted bitstreams.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0aedab03405849962b469277afe047aa2c61a87f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Laurent Aimar
aee461277a Fixed invalid access in wavpack decoder on corrupted extra bits sub-blocks.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit beefafda639dd53fc59c21d8a7cf8334da9a1062)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Anton Khirnov
a4f2973b2d lavc: fix type for thread_type option
It should be flags, not int.
(cherry picked from commit fb47997edb9d8ff16fc380d005a08c0545624aa6)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:31 +02:00
Alberto Delmás
db5e27f94e VC1: Fix first/last row checks with slices
In some places 0/mb_height were used in place of start_mb_y/end_mb_y.

Fixes SA00049, SA00058, SA10091, SA10097, SA10131, SA20021, SA30030

Improves PSNR in SA00054, SA00059, SA00060, SA10096, SA10098, SA20022,
SA30031, SA30032, SA40012, SA40013

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 1cf82cab0840d669198ea76ab0363aa661950647)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Ronald S. Bultje
97ce2a29b6 vc1: properly zero coded_block[] edges on new slice entry.
Previously, we would leave the left edge uninitialized, which led to
CBP prediction errors on slice edges, e.g. in SA10098.vc1.
(cherry picked from commit d4b9974465baf893e90527a366e7a7411ded1ef8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Jeff Downs
ce8f40a7b9 h264: fix PCM intra-coded blocks in monochrome case
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 6581e161c5f46733a5619208483de29416eb9a51)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Jeff Downs
45b3f7c71e h264: correct implicit weight table computation for long ref pics
Correct computation of implicit weight tables when referencing pictures
that are marked for long reference.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 87cf70eb237e7586cc7399627dafa1b980ec0b7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Jeff Downs
8ad6555f82 h264: correct the check for invalid long term frame index in MMCO decode
The current check on MMCO parameters prohibits a "max long term frame index
plus 1" of 16 (frame idx of 15) for the "set max long term frame index" MMCO.
Fix this off-by-one error to allow the full range of legal values.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 29a09eae9a827f4dbc9c4517180d8fe2ecef321a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Alex Converse
b4099a6dc5 aac: Only output configure if audio was found.
Audio found is not triggered on a CCE because a CCE alone has no output.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit d8425ed4af6d8fce62ff363cc590f85e57bac06b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Kostya Shishkov
dec458b900 rv10/20: tell decoder to use edge emulation
This removes out-of-edge motion compensation artifacts (easily spotted green
blocks in avplay, gray blocks in transcoding), for example here:
http://samples.libav.org/samples/real/tv_watching_t1.rm

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 331971116d7d36743601bd2dc5384c5211d3bb48)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00