32051 Commits

Author SHA1 Message Date
Michael Niedermayer
efd30c4d95 dv: Fix null pointer dereference due to ach=0
dv: Fix null pointer dereference due to ach=0

Fixes part2 of CVE-2011-3929

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 5a396bb3a66a61a68b80f2369d0249729bf85e04)
2012-03-06 15:28:00 -08:00
Michael Niedermayer
d7fddc97d4 dv: check stype
dv: check stype

Fixes part1 of CVE-2011-3929
Possibly fixes part of CVE-2011-3936

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 635bcfccd439480003b74a665b5aa7c872c1ad6b)
2012-03-06 15:28:00 -08:00
Dale Curtis
feed0c6b6a mpegaudiodec: Prevent premature clipping of mp3 input buffer.
Instead of clipping extrasize based on EXTRABYTES, clip based on the
amount of buffer actually left. Without this fix, there are warbles
and other distortions in the test case below.

http://kevincennis.com/mix/assets/sounds/1901_voxfx.mp3
(cherry picked from commit b7165426917f91ebcad84bdff366824f03b32bfe)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-06 15:28:00 -08:00
Alex Converse
d0e53ecff7 mp3dec: Fix a heap-buffer-overflow
In some cases, what is left to read from ptr is smaller than EXTRABYTES.

Based on a patch by Thierry Foucu <tfoucu@gmail.com>.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit f372ce119bd2458fa0b4ddfb2af3a36621df99f7)
2012-03-06 15:28:00 -08:00
Alex Converse
1ca84aa162 mpeg12: Pad framerate tab to 16 entries.
There are many places where we read an unchecked 4-bit index into it.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit dfa37fe8a3d9243dd339d94befa065e2c90b29e6)
2012-03-06 15:28:00 -08:00
Michael Niedermayer
d5f2382d03 kgv1dec: Increase offsets array size so it is large enough.
Fixes CVE-2011-3945

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 807a045ab7f51993a2c1b3116016cbbd4f3d20d6)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit a02e8df973f5478ec82f4c507f5b5b191a5ecb6b)
2012-03-06 15:28:00 -08:00
Alex Converse
416849f2e0 kmvc: Check palsize.
Fixes: CVE-2011-3952

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Based on fix by Michael Niedermayer
(cherry picked from commit 386741f887714d3e46c9e8fe577e326a7964037b)
2012-03-06 15:28:00 -08:00
Alex Converse
dd37038ac7 nsvdec: Propagate errors
Related to CVE-2011-3940.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit c898431ca5ef2a997fe9388b650f658fb60783e5)

Conflicts:

	libavformat/nsvdec.c
2012-03-06 15:28:00 -08:00
Alex Converse
e410dd1792 nsvdec: Be more careful with av_malloc().
Check results for av_malloc() and fix an overflow in one call.

Related to CVE-2011-3940.

Based in part on work from Michael Niedermayer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 8fd8a48263ff1437f9d02d7e78dc63efb9b5ed3a)
2012-03-06 15:28:00 -08:00
Michael Niedermayer
ffdc41f039 nsvdec: Fix use of uninitialized streams.
Fixes CVE-2011-3940 (Out of bounds read resulting in out of bounds write)

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c011706bc752d34bc6ada31d7df2ca0c9af7c6b)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 6a89b41d9780325ba6d89a37f2aeb925aa68e6a3)
2012-03-06 15:28:00 -08:00
Martin Storsjö
ca7e97bdcf g722: Fix the QMF scaling
This fixes clipping if the encoder input used the full 16 bit
input range (samples with a magnitude below 16383 worked fine).
The filtered subband samples should be 15 bit maximum, while
the code earlier produced them scaled to 16 bit.

This makes the decoder output have double the magnitude
compared to before.

The spec reference samples doesn't test the QMF at all, which
was why this part slipped past initially.

(cherry picked from commit b087ce2bee81db8cc5caffb8f0a4f6c7c92a30fe)

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-06 15:45:30 +02:00
Justin Ruggles
4ae138cb12 ac3dsp: do not use pshufb in ac3_extract_exponents_ssse3()
We need to do unsigned saturation in order to cover the corner case when the
absolute coefficient value is 16777215 (the maximum value).

Fixes Bug #216
(cherry picked from commit d483bb58c318b0a6152709cf28263d72200b98f9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-06 13:55:35 +01:00
Fabian Greffrath
003f7e3dd0 Fix format string vulnerability detected by -Wformat-security.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c9dbac36ad4bac07f6c1d06d465e361ab55bcb95)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:01:37 +01:00
Ronald S. Bultje
85eb76a23f h264: fix mmxext chroma deblock to use correct TC values.
(cherry picked from commit b0c4f04338234ee011d7b704621347ef232294fe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:43 +01:00
Ronald S. Bultje
5186984ee9 h264: change underread for 10bit QPEL to overread.
This prevents us from reading before the start of the buffer, and thus
prevents crashes resulting from this behaviour. Fixes bug 237.
(cherry picked from commit 291c9b62855d555ac5385e23219461b6080da7db)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:31 +01:00
Ronald S. Bultje
b5331b979b cscd: use negative error values to indicate decode_init() failures.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8a9faf33f2b4f40afbc3393b2be49867cea0c92d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Vitor Sessak
11f3173e1b amrnbdec: check frame size before decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 882abda5a26ffb8e3d1c5852dfa7cdad0a291d2d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Ronald S. Bultje
cd17195d1c h264: prevent overreads in intra PCM decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d1604b3de96575195b219028e2c4f08b2259aa7d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Justin Ruggles
1128b10247 wmaenc: fix m/s stereo encoding for the first frame
We need to set ms_stereo in encode_init() in order to avoid incorrectly
encoding the first frame as non-m/s while flagging it as m/s. Fixes an
uncomfortable pop in the left channel at the start of playback.

CC:libav-stable@libav.org
(cherry picked from commit 51ddf35c9017018e58c15275ff5b129647a0c94d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
6a073aa7a7 wmaenc: limit allowed sample rate to 48kHz
ff_wma_init() allows up to 50kHz, but this generates an exponent band
size table that requires 65 bands. The code assumes 25 bands in many
places, and using sample rates higher than 48kHz will lead to buffer
overwrites.

CC:libav-stable@libav.org
(cherry picked from commit 1ec075cfecac01f9a289965db06f76365b0b1737)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
073891e875 wmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE
This is near the theoretical limit for wma frame size and is the most that
our decoder can handle. Allowing higher bit rates will just end up padding
each frame with empty bytes.

Fixes invalid writes for avconv when using very high bit rates.

CC:libav-stable@libav.org
(cherry picked from commit c2b8dea1828f35c808adcf12615893d5c740bc0a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
2e341bc99a wmaenc: require a large enough output buffer to prevent overwrites
The maximum theoretical frame size is around 17000 bytes. Although in
practice it will generally be much smaller, we require a larger buffer
just to be safe.

CC: libav-stable@libav.org
(cherry picked from commit dfc4fdedf8cfc56a505579b1f2c1c5efbce4b97e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Alex Converse
b7c8fff803 mpegts: Do not call read_sl_header() when no bytes remain in the buffer.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4df369692ea8aee7094ac0f233cef8d1bee139a3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Alex Converse
3f7e90cf0c mpegts: Pad the packet buffer in handle_packet().
This allows it to be used with get_bits without the thread of overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1aa708988ac131cf7d5c8bd59aca256a7c974df9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
78d4f8cc56 amrwb: remove duplicate arguments from extrapolate_isf().
Prevents warnings because the dst and src overlap (are the same) in the
memcpy() inside the function.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9d87374ec0f382c8394ad511243db6980afa42af)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
de2656ec25 amrwb: error out early if mode is invalid.
Prevents using the invalid mode as an index in a static array, which
would generate invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 154b8bb80029e71d562e8936164266300dd35a0e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
9686a2c2cf matroska: check buffer size for RM-style byte reordering.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9c239f6026a170866a4a0c96908980ac2cfaa8b3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
b863979c0f wma: fix invalid buffer size assumptions causing random overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 349b7977e408f18cff01ab31dfa66c8249b6584a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
fecd7468fc wmadec: Verify bitstream size makes sense before calling init_get_bits.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 48f1e5212c90b511c90fa0449655abb06a9edda2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
19da1a39e8 rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f6528537fdd88820f3a4683d5e595d7b3a62689)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
7e88df99e1 lcl: return negative error codes on decode_init() errors.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd17a40a7e0eba21b5d27c67aff795e2910766e4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Justin Ruggles
7f3f85544c avutil: add AVERROR_UNKNOWN
Useful to return instead of -1 when the cause of the error is unknown,
typically from an external library.
(cherry picked from commit c9bca801324f03746757aef8549ebd26599adec2)

Conflicts:

	doc/APIchanges
	libavutil/avutil.h

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
750f5baf30 h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 0ce4fe482c27abfa7eac503a52fdc50b70ccd871)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
a63f3f714c huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 63c9de6469005974288f4e4d89fc79a590e38c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
1dd1ee00d5 vmnc: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 07a180972fb369bb59bf6d4f8edb4598c51e80d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
4493af756b rpza: error out on buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 78e9852a2e3b198ecd69ffa0deab3fa22a8e5378)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
e904e9b720 qtrle: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e54ae60e46f737b8e9a96548971091f7ab6b8f7c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
5f896773e0 swscale: fix another integer overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 791de61bbb0d2bceb1037597b310e2a4a94494fd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
b2dcac7141 vp56: error out on invalid stream dimensions.
Prevents crashes when playing corrupt vp5/6 streams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8bc396fc0e8769a056375c1c211f389ce0e3ecc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:23:11 +01:00
Ronald S. Bultje
40ccc81146 asf: don't seek back on EOF.
Seeking back on EOF will reset the EOF flag, causing us to re-enter
the loop to find the next marker in the ASF file, thus potentially
causing an infinite loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bb6d5411e1e1a8e0608b1af1c4addee654dcbac5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:22:35 +01:00
Ronald S. Bultje
1c63d61372 asf: error out on ridiculously large minpktsize values.
They cause various issues further down in demuxing.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6e57a02b9f639af53acfa9fc742c1341400818f8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:21:57 +01:00
Anton Khirnov
2ad77c60ef lavf: add functions for accessing the fourcc<->CodecID mapping tables.
Fixes bug 212.
(cherry picked from commit dd6d3b0e025cb2a16022665dbb8ab1be18dc05e8)

Conflicts:

	doc/APIchanges

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-29 10:44:37 +01:00
Paul B Mahol
a1556d37b8 avutil: make intfloat api public
The functions are already av_ prefixed and intfloat header is already provided.
Install libavutil/intfloat.h

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8b933129b932f523a746e921a0a20b8dd8816971)

Conflicts:

	doc/APIchanges

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-29 10:44:37 +01:00
Alex Converse
083a8a0037 mjpegbdec: Fix overflow in SOS.
Based in part by a fix from Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2011-3947

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit b57d262412204e54a7ef8fa1b23ff4dcede622e5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 22:20:45 +01:00
Ronald S. Bultje
71a939fee4 oma: don't read beyond end of leaf_table.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 934cd18a43151ba4b819d9270d539cdb26f6e079)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 22:10:55 +01:00
Ronald S. Bultje
9dbd437da2 Indeo3: fix crashes on corrupt bitstreams.
Splits at borders of cells are invalid, since it leaves one of the
cells with a width/height of zero. Also, propagate errors on buffer
allocation failures, so we don't continue decoding (which crashes).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fc9bc08dca9ac32526251e19fcf738d23b8c68d1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 20:57:44 +01:00
Ronald S. Bultje
2510e1476e vorbis: fix overflows in floor1[] vector and inverse db table index.
(cherry picked from commit 24947d4988012f1f0fd467c83418615adc11c3e8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 18:11:15 +01:00
Reinhard Tartler
0f839cff6b Fix parser not to clobber has_b_frames when extradata is set.
Because in contrast to the decoder, the parser does not setup low_delay.
The code in parse_nal_units would always end up setting has_b_frames
to "1", except when stream is explicitly marked as low delay.
Since the parser itself would create 'extradata', simply reopening
the parser would cause this.

This happens for instance in estimate_timings_from_pts(), which causes the
parser to be reopened on the same stream.

This fixes Libav #22 and FFmpeg (trac) #360

CC: libav-stable@libav.org

Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(commit 31ac0ac29b6bba744493f7d1040757a3f51b9ad7)

Comments and description adapted by Reinhard Tartler.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 790a367d9ecd04360f78616765ee723f3fe65645)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 15:56:55 +01:00
Ronald S. Bultje
abe3572878 rm: prevent infinite loops for index parsing.
Specifically, prevent jumping back in the file for the next index, since
this can lead to infinite loops where we jump between indexes referring
to each other, and don't read indexes that don't fit in the file.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aac07a7a4c2c7a4a29cf6dbc88c1b9fdd191b99d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 10:04:04 +01:00
Ronald S. Bultje
0d30e2c6f2 fraps: release reference buffer on pix_fmt change.
Prevents crash when trying to copy from a non-existing plane in e.g.
a RGB32 reference image to a YUV420P target image

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 830f70442a87a31f7c75565e9380e3caf8333b8a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00