15278 Commits

Author SHA1 Message Date
Fabian Greffrath
003f7e3dd0 Fix format string vulnerability detected by -Wformat-security.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c9dbac36ad4bac07f6c1d06d465e361ab55bcb95)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:01:37 +01:00
Ronald S. Bultje
85eb76a23f h264: fix mmxext chroma deblock to use correct TC values.
(cherry picked from commit b0c4f04338234ee011d7b704621347ef232294fe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:43 +01:00
Ronald S. Bultje
5186984ee9 h264: change underread for 10bit QPEL to overread.
This prevents us from reading before the start of the buffer, and thus
prevents crashes resulting from this behaviour. Fixes bug 237.
(cherry picked from commit 291c9b62855d555ac5385e23219461b6080da7db)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:31 +01:00
Ronald S. Bultje
b5331b979b cscd: use negative error values to indicate decode_init() failures.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8a9faf33f2b4f40afbc3393b2be49867cea0c92d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Vitor Sessak
11f3173e1b amrnbdec: check frame size before decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 882abda5a26ffb8e3d1c5852dfa7cdad0a291d2d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Ronald S. Bultje
cd17195d1c h264: prevent overreads in intra PCM decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d1604b3de96575195b219028e2c4f08b2259aa7d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Justin Ruggles
1128b10247 wmaenc: fix m/s stereo encoding for the first frame
We need to set ms_stereo in encode_init() in order to avoid incorrectly
encoding the first frame as non-m/s while flagging it as m/s. Fixes an
uncomfortable pop in the left channel at the start of playback.

CC:libav-stable@libav.org
(cherry picked from commit 51ddf35c9017018e58c15275ff5b129647a0c94d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
6a073aa7a7 wmaenc: limit allowed sample rate to 48kHz
ff_wma_init() allows up to 50kHz, but this generates an exponent band
size table that requires 65 bands. The code assumes 25 bands in many
places, and using sample rates higher than 48kHz will lead to buffer
overwrites.

CC:libav-stable@libav.org
(cherry picked from commit 1ec075cfecac01f9a289965db06f76365b0b1737)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
073891e875 wmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE
This is near the theoretical limit for wma frame size and is the most that
our decoder can handle. Allowing higher bit rates will just end up padding
each frame with empty bytes.

Fixes invalid writes for avconv when using very high bit rates.

CC:libav-stable@libav.org
(cherry picked from commit c2b8dea1828f35c808adcf12615893d5c740bc0a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
2e341bc99a wmaenc: require a large enough output buffer to prevent overwrites
The maximum theoretical frame size is around 17000 bytes. Although in
practice it will generally be much smaller, we require a larger buffer
just to be safe.

CC: libav-stable@libav.org
(cherry picked from commit dfc4fdedf8cfc56a505579b1f2c1c5efbce4b97e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Ronald S. Bultje
78d4f8cc56 amrwb: remove duplicate arguments from extrapolate_isf().
Prevents warnings because the dst and src overlap (are the same) in the
memcpy() inside the function.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9d87374ec0f382c8394ad511243db6980afa42af)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
de2656ec25 amrwb: error out early if mode is invalid.
Prevents using the invalid mode as an index in a static array, which
would generate invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 154b8bb80029e71d562e8936164266300dd35a0e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
b863979c0f wma: fix invalid buffer size assumptions causing random overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 349b7977e408f18cff01ab31dfa66c8249b6584a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
fecd7468fc wmadec: Verify bitstream size makes sense before calling init_get_bits.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 48f1e5212c90b511c90fa0449655abb06a9edda2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
19da1a39e8 rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f6528537fdd88820f3a4683d5e595d7b3a62689)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
7e88df99e1 lcl: return negative error codes on decode_init() errors.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd17a40a7e0eba21b5d27c67aff795e2910766e4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
750f5baf30 h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 0ce4fe482c27abfa7eac503a52fdc50b70ccd871)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
a63f3f714c huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 63c9de6469005974288f4e4d89fc79a590e38c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
1dd1ee00d5 vmnc: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 07a180972fb369bb59bf6d4f8edb4598c51e80d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
4493af756b rpza: error out on buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 78e9852a2e3b198ecd69ffa0deab3fa22a8e5378)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
e904e9b720 qtrle: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e54ae60e46f737b8e9a96548971091f7ab6b8f7c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje
b2dcac7141 vp56: error out on invalid stream dimensions.
Prevents crashes when playing corrupt vp5/6 streams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8bc396fc0e8769a056375c1c211f389ce0e3ecc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:23:11 +01:00
Alex Converse
083a8a0037 mjpegbdec: Fix overflow in SOS.
Based in part by a fix from Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2011-3947

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit b57d262412204e54a7ef8fa1b23ff4dcede622e5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 22:20:45 +01:00
Ronald S. Bultje
9dbd437da2 Indeo3: fix crashes on corrupt bitstreams.
Splits at borders of cells are invalid, since it leaves one of the
cells with a width/height of zero. Also, propagate errors on buffer
allocation failures, so we don't continue decoding (which crashes).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fc9bc08dca9ac32526251e19fcf738d23b8c68d1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 20:57:44 +01:00
Ronald S. Bultje
2510e1476e vorbis: fix overflows in floor1[] vector and inverse db table index.
(cherry picked from commit 24947d4988012f1f0fd467c83418615adc11c3e8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 18:11:15 +01:00
Reinhard Tartler
0f839cff6b Fix parser not to clobber has_b_frames when extradata is set.
Because in contrast to the decoder, the parser does not setup low_delay.
The code in parse_nal_units would always end up setting has_b_frames
to "1", except when stream is explicitly marked as low delay.
Since the parser itself would create 'extradata', simply reopening
the parser would cause this.

This happens for instance in estimate_timings_from_pts(), which causes the
parser to be reopened on the same stream.

This fixes Libav #22 and FFmpeg (trac) #360

CC: libav-stable@libav.org

Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(commit 31ac0ac29b6bba744493f7d1040757a3f51b9ad7)

Comments and description adapted by Reinhard Tartler.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 790a367d9ecd04360f78616765ee723f3fe65645)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 15:56:55 +01:00
Ronald S. Bultje
0d30e2c6f2 fraps: release reference buffer on pix_fmt change.
Prevents crash when trying to copy from a non-existing plane in e.g.
a RGB32 reference image to a YUV420P target image

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 830f70442a87a31f7c75565e9380e3caf8333b8a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
a0473085f3 kgv1: release reference picture on size change.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6c4c27adb61b2881a94ce5c7d97ee1c8adadb5fe)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
e537dc230b kgv1: use avctx->get/release_buffer().
Also fixes crashes on corrupt bitstreams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 33cd32b389864f2437c94e6fd7dc109ff5f0ed06)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
19f4943d12 lcl: error out if uncompressed input buffer is smaller than framesize.
This prevents crashes when trying to read beyond the end of the buffer
while decoding frame data.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be129271eac04f91393bf42a490ec631e1a9abea)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
bf6d1a1ca7 mjpeg: abort decoding if packet is too large.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ab492ca2ab105aeb24d955f3f03756bdb3139ee1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Alex Converse
424b6edd19 tiff: Prevent overreads in the type_sizes array.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 447363870f2f91e125e07ac2d0820359a5d86b06)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
8e3dc37bc0 truemotion2: error out if the huffman tree has no nodes.
This prevents crashers and errors further down when reading nodes in the
empty tree.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2b83e8b7005d531bc78b0fd4f699e9faa54ce9bb)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
8011a29fa8 vc1parse: call vc1_init_common().
The parser uses VLC tables initialized in vc1_common_init(), therefore
we should call this function on parser init also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c742ab4e81bb9dcabfdab006d6b8b09a5808c4ce)

Conflicts:

	libavcodec/vc1.h

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
fe710f2074 wma: don't return 0 on invalid packets.
Return 0 means "please return the same data again", i.e. it causes an
infinite loop. Instead, return an error.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9d3050d3e95e307ebc34a943484c7add838d1220)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
bba43a1ea0 mjpegb: don't return 0 at the end of frame decoding.
Return 0 indicates "please return the same data again", i.e. it causes
an infinite loop. Instead, return that we consumed the buffer if we
finished decoding succesfully, or return an error if an error occurred.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 74699ac8c8b562e9f8d26e21482b89585365774a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje
95a9d44dc3 mp3on4: require a minimum framesize.
If bufsize < headersize, init_get_bits() will be called with a negative
number, causing it to fail and any subsequent call to get_bits() will
crash because it reads from a NULL pointer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3e13005cac6e076053276b515f5fcf59a3f4b65d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Ronald S. Bultje
27558bd87e huffyuv: error out on bit overrun.
On EOF, get_bits() will continuously return 0, causing an infinite
loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 84c202cc37024bd78261e4222e46631ea73c48dd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Ronald S. Bultje
5ab9294a8d als: prevent infinite loop in zero_remaining().
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit af468015d972c0dec5c8c37b2685ffa5cbe4ae87)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Ronald S. Bultje
cfd7d166e2 cook: prevent div-by-zero if channels is zero.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 941fc1ea1ed7f7d99a8b9e2607b41f2f2820394a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Ronald S. Bultje
5bcd47cf63 vc1: prevent using last_frame as a reference for I/P first frame.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ae591aeea58d64399b8281be31dacec0de85ae04)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Ronald S. Bultje
7046ae5593 tta: error out if samplerate is zero.
Prevents a division by zero later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 7416d610362807848236ceff1bc6740dbc82842d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Janne Grunau
d19e3e19d6 vc1: prevent null pointer dereference on broken files
CC: libav-stable@libav.org
(cherry picked from commit 510ef04a461b3b54a762c6141ad880cbed85981f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Alex Converse
04597e2595 smacker: Sanity check huffman tables found in the headers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit 9adf25c1cf78dbf1d71bf386c49dc74cb8a60df0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Alex Converse
683213230e aac: fix infinite loop on end-of-frame with sequence of 1-bits.
Based-on-work-by: Ronald S. Bultje <rsbultje@gmail.com>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1cd9a6154bc1ac1193c703cea980ed21c3e53792)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Alex Converse
ad0ee682b3 wma: Clip WMA1 and WMA2 frame length to 11 bits.
The MDCT buffers in the decoder are only sized for up to 11 bits. The
reverse engineered documentation for WMA1/2 headers say that that for
all samplerates above 32kHz 11 bits are used. 12 and 13 bit support
were added for WMAPro. I was unable to make any Microsoft tools generate
a test file at a samplerate above 48kHz.

Discovered by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d78bb1a4b2a3a415b68e4e6dd448779eccec64e3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:15 +01:00
Janne Grunau
ba418ad400 rv20: prevent calling ff_h263_decode_mba() with unset height/width
Prevents a crash of VLC during playback of a invalid matroska file,
found by John Villamil <johnv@matasano.com>.

CC: libav-stable@libav.org
(cherry picked from commit c3e10ae4127c998b809066926a410f40ebd47593)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:14 +01:00
Ronald S. Bultje
6dcbbdc011 flac: fix infinite loops on all-zero input or end-of-stream.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 52e4018be47697a60f4f18f83551766df31f5adf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 10:03:14 +01:00
Ronald S. Bultje
e43bd4fa58 golomb: use HAVE_BITS_REMAINING() macro to prevent infloop on EOF.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 46b3fbc30b7aaf7fdd52391734cfd6d93af8720a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 10:03:14 +01:00
Ronald S. Bultje
25b4ed053f get_bits: add HAVE_BITS_REMAINING macro.
(cherry picked from commit b44b41633f110e9d938165e0f79c9d32191fc135)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 10:03:14 +01:00