14752 Commits

Author SHA1 Message Date
Justin Ruggles
2137d99086 vorbisdec: check output buffer size before writing output
(cherry picked from commit 60aa1a358d9c1c8f891e72246d5dcd897857eca8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 03:18:52 +01:00
Justin Ruggles
e9de2d98a9 twinvq: check output buffer size before decoding
(cherry picked from commit e53eecd0e7211973a1a9757f559bdd93a1848901)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 03:18:52 +01:00
Alex Converse
93f1159af5 vp6: Fix illegal read.
(cherry picked from commit 2a6eb06254df79e96b3d791b6b89b2534ced3119)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:54:13 +01:00
Justin Ruggles
b08001e00a shorten: check output buffer size before decoding
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:30:29 +01:00
Justin Ruggles
e1ea35fb52 shorten: check for realloc failure
(cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:24:03 +01:00
Matthew Einhorn
f531193690 Fixes avpicture_layout to not write past buffer end.
avpicture_get_size() returns the size of buffer required for avpicture_layout.
For pseudo-paletted formats (gray8...) this size does not include the palette.
However, avpicture_layout doesn't know this and still writes the palette. Consequently,
avpicture_layout writes passed the length of the buffer. This fixes it
by fixing avpicture_layout so that it doesn't write the palette for these formats.

Signed-off-by: Matthew Einhorn <moiein2000@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e662b263d9c500270a8f1dc7e1b81b51d5bdfd4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:23:47 +01:00
Ronald S. Bultje
15a7fe106c pthread: copy coded frame dimensions in update_context_from_thread
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit feadcd1bdcbb4601f4ff01878027264fde985ee1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:23:28 +01:00
Ronald S. Bultje
d32f509de1 vp8: prevent read from uninitialized memory in decode_mvs
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 0f0b5d643401d4d83322eeee0e57eb5a226ef9ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:59 +01:00
Ronald S. Bultje
5f5f36b52e vp8: force reallocation in update_thread_context after frame size change
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 56535793810584f5b3ae59e62cea66fe22d0307d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:52 +01:00
Ronald S. Bultje
d1166f03be vp8: fix return value if update_dimensions fails
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit f05c2fb6eb1f9ddaec3c07d1874ba62ec0891269)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:45 +01:00
Justin Ruggles
62cf52c860 truespeech: check to make sure channels == 1
(cherry picked from commit 3e7a176759e8a8e66d65c779b47b5bba793dfd4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:09:22 +01:00
Justin Ruggles
7e95a12d51 mlpdec: validate that the reported channel count matches the actual output
channel count
(cherry picked from commit caa845851d790f894a2ccbe12580934f75545f92)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:09:17 +01:00
Mans Rullgard
1c3d46a924 h264: fix HRD parameters parsing
The bit_rate_value_minus1 and cpb_size_value_minus1 elements
allow a wider range than get_ue_golomb() supports.  This
adds a get_ue_golomb_long() function supporting up to 31
leading zeros, which is the maximum for these syntax
elements, and uses it in decode_hrd_parameters().

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit fdba370f8a1bdfc22ecbdf3c7148c2f8680a4ac4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:08:09 +01:00
Justin Ruggles
800ab099e3 smacker: validate channels and sample format.
(cherry picked from commit ff1f89de2da3472d133e2c95bf7c9ad2d88df33d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:49 +01:00
Justin Ruggles
e6b2255329 smacker: check buffer size before reading output size
(cherry picked from commit cf044f8bff0d28dbc34492f18b0d18b3ba8bad9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:44 +01:00
Justin Ruggles
7f7b2e89e2 smacker: validate number of channels
(cherry picked from commit e190e453bd1e4d4b409ed3556b3a50d1087c15d7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:39 +01:00
Mans Rullgard
73f85eae68 sipr: fix get_bits(0) calls
Zero-length get_bits() is undefined, must check before calling.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit c79d2a20bad59298188171f1316a830d563a41ee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:46 +01:00
Mans Rullgard
190807a56c 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda407628e298f33d610e9e04a8b2945d24665)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:17 +01:00
Mans Rullgard
33029d7353 wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d0611d9a6c10eaa90b3c058ecdffe76676)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:13 +01:00
Mans Rullgard
c41950099d mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be4b479bfa8d60ed3eb13e0f89e439c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:08 +01:00
Mans Rullgard
115d88c4b2 h264pred: use unsigned types for pixel values, fix signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 60f10e0ad37418cc697765d85b0bc22db70f726a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:48 +01:00
Laurent Aimar
a65045915f qtrle: check for out of bound writes.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7fb92be7e50ea4ba5712804326c6814ae02dd190)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:23 +01:00
Laurent Aimar
adb12c4deb xxan: check for out of bound accesses
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a68a6a4fb19caecc91d5f7fe3ef4f83f6d3c4586)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:15 +01:00
Laurent Aimar
ca58b215ab txd: check for out of bound reads.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e182de9a98272fbe4f368000911191aaeb0d6fb3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:11 +01:00
Laurent Aimar
67c46b9b30 qtrle: check for invalid line offset
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4ed7c3fe9f99b89f86b65710d8855dc572f1a25)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:07 +01:00
Laurent Aimar
7ab0b6b7ed vqavideo: check for out of bound reads.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d45702f7f257c1cfcd3ce3287bf258854528a4a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:04 +01:00
Laurent Aimar
2fdbc1d553 vqavideo: check for invalid/unsupported version
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b226af39107f09cd650875388250f4968eea54db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:54 +01:00
Laurent Aimar
5415c488f9 eamad: release the reference frame on video size changes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6c1fb3e7631178c2a45c6c41b9b8d9ee3a5298fc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:49 +01:00
Laurent Aimar
79bafbb0dd eamad: check for out of bound reads when doing MC
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit da35797359cec148f3fe59894c62727b0422d75a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:43 +01:00
Laurent Aimar
7b3c851526 eamad: avoid NULL derefence when missing the reference frame.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e20554a6d33e92b81dc3cfed6082e03bff2a7f8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:38 +01:00
Laurent Aimar
1b6e6439fa eatgv: fix pointer arithmetic overflows.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6bfe0d4c3d7de11e859ea6720b011cf5fdf5ef03)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:34 +01:00
Laurent Aimar
4474051370 eatgv: fix out of bound reads on corrupted motions vectors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 09302a897d1990b1338f049fcd29638d736b8823)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:29 +01:00
Laurent Aimar
1646d2d2ae eamad: clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 74b9c598396f76407c6b3841c10bc67ddddb2a98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:23 +01:00
Mans Rullgard
f7be632cbd vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606e00057a706ed30165bc7329db3823f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:12 +01:00
Mans Rullgard
4ba0e03759 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c026a9eb547dab07781320a7e2564312)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:09 +01:00
Mans Rullgard
37ce6ba425 dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42be7a02c23976216b47fd63b80d6c7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:04 +01:00
Mans Rullgard
c2c83dcb32 aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304acd82cb219e3f66ca9cd6efb2194fa4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:58 +01:00
Mans Rullgard
4c5cdb493c put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496aad6f8b09deabbef4fe5fd829e142d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:53 +01:00
Laurent Aimar
06b15b3715 h264: fix the size of PPS::chroma_qp_table
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e588a5c2d418fdcf08ece076a2642de44f444d55)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:41 +01:00
Michael Niedermayer
614ef0dc0d h264: fix fill_colmap() to not store entries mbaff style when the reference is not mbaff at all
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a3ba542af39ba4425cbb44d2a0ae09c212b40001)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:34 +01:00
Ronald S. Bultje
5d2b6006f0 mpegvideo: fix position of bottom edge.
It was wrong in colorspaces where horizontal and vertical chroma
subsampling are not the same, e.g. 422.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:58:35 +01:00
Laurent Aimar
b491c15c85 h254: explicitly initialize bit depth/chroma idc
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:53:56 +01:00
Justin Ruggles
2809f4ab93 qcelp: check output buffer size before decoding
(cherry picked from commit e43dd3d2a8e106169e707484090a2d973ece2184)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:43:10 +01:00
Justin Ruggles
c2d017e88f sipr: fix the output data size check and only calculate it once.
(cherry picked from commit 1b5a189f06879338088809b3049ea7620f4e7e78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:59 +01:00
Michael Niedermayer
4f45967cf5 ff_dv_frame_profile2: Check input buffer size.
Based on code by DivX, Inc. / drffmpeg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 51b0694bc051cda2bfed048a35e694d1047c6ef0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:46 +01:00
Justin Ruggles
78eab18740 qdm2: check output buffer size before decoding
(cherry picked from commit 7d49f79f1cd47783a963a757a6563b9cac29db62)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:37 +01:00
Justin Ruggles
fc8c0ee09f mpc8: check output buffer size before decoding
(cherry picked from commit 5674d4b0a35a34b75e3533a8580e0b5a0a8895a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:13 +01:00
Justin Ruggles
490617b6ff mpc7: return error if packet is too small.
(cherry picked from commit 8290d1f38b438f1b070de67645c8b4a42014c7ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:03 +01:00
Justin Ruggles
b833859daa mpc7: check output buffer size before decoding
(cherry picked from commit c8b5c4d27409dfdcec80868686b173ba446c998b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:39:47 +01:00
Gwenole Beauchesne
7d52ed686b vaapi: fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
(cherry picked from commit 825dd135d829d85778e6c37f685a6088fef01b2e)
2011-10-12 11:27:11 +02:00