15187 Commits

Author SHA1 Message Date
Ronald S. Bultje
105601c151 wmavoice: fix stack overread.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 262196445cf03fda0f7e41c4b968f4f7bf060e6b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
3a4949aa50 indeo4: fix out-of-bounds function call.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 68fd077f68bdde864bb7328d72a040849c616261)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
bf3998d71e mimic: don't use self as reference, and report completion at end of decode().
Fixes hangs on corrupt samples that reference self-frames.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 80387f0e2568746dce4a68e2217297029a053dae)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
87208b8fc4 mpeg4: report frame decoding completion at ff_MPV_frame_end().
Prevents hangs on corrupt input.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6ccb96bc955b2087ec71033d99b3dcd5203eaf2)

Conflicts:

	libavcodec/mpegvideo.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Kostya Shishkov
1ee0cd1ad7 dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2012-03-14 23:32:15 +01:00
Ronald S. Bultje
b594732475 dca: don't use av_clip_uintp2().
The argument is not a literal, thus causing the ARM v6 or later
builds to break.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2012-03-14 23:30:19 +01:00
Michael Niedermayer
ce15406e78 snow: check reference frame indices.
Fixes NULL ptr dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 1f8ff2b13cbfef790385818664ed12e763e7c75b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:35:09 +01:00
Michael Niedermayer
c9e95636a8 snow: reject unsupported chroma shifts.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit c9837954e7b968d44f82e7cdb7618e9f523b196c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:55 +01:00
Ronald S. Bultje
6e5c07f4c8 xa_adpcm: limit filter to prevent xa_adpcm_table[] array bounds overruns.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 86020073dbb9a3a9d1fbb76345b2ca29ba1f13d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:36 +01:00
Ronald S. Bultje
c999a8ed65 h264: increase reference poc list from 16 to 32.
Interlaced images can have 32 references (16 per field), so limiting the
array size to 16 leads to invalid writes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 48cbe4b092113eae0b3e5d6a08b59027f913a884)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:13 +01:00
Ronald S. Bultje
4d343a6f47 h264: stricter reference limit enforcement.
Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e0febda22d0e0fab094a9c886b0e0f0f662df1ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:33:15 +01:00
Michael Niedermayer
a81a6d9c80 h264: improve parsing of broken AVC SPS
Parsing the entire NAL as SPS fixes decoding of some AVC bitstreams
with broken escaping. Since the size of the NAL unit is known and
checked against the buffer end we can parse it entirely without buffer
overreads.

Fixes playback of
http://streams.videolan.org/streams/mp4/Mr_MrsSmith-h264_aac.mp4

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3aa661ec561d7a20812b84b353b0d7855ac346c8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:27:22 +01:00
Alex Converse
48f0eeb2e5 Replace computations of remaining bits with calls to get_bits_left().
(cherry picked from commit 3574a85ce57366ba7429edef93d5cad8640fb68c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:27:16 +01:00
Ronald S. Bultje
d26e47bf6c png: convert to bytestream2 API.
Protects against overreads in the input buffer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4c25269cedd042abcb823c42d33609564861c374)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:14:28 +01:00
Ronald S. Bultje
568a474a08 roqvideo: convert to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cdf15771621bce7959b3e53b21426c5ba747e17b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:40 +01:00
Ronald S. Bultje
9a66cdbc16 smc: port to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8febcb9fc178926687ee19d32d2b3150da899867)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:28 +01:00
Ronald S. Bultje
ddb1149e25 tgq: convert to bytestream2 API.
This protects against input buffer overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1255eed533b4069db7f205601953ca54c0dc42c9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:19 +01:00
Ronald S. Bultje
f6778f58d4 algmm: convert to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit a55d5bdc6e28a2cfefc440d792de5cc4f02377e2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:19 +01:00
Paul B Mahol
e4e4d92641 jvdec: unbreak video decoding
The safe bitstream reader broke it since the buffer size was specified
in bytes instead of bits.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
CC: libav-stable@libav.org
(cherry picked from commit a1c036e961a32f7208e7315dabfa0ee99d779edb)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:02:23 +01:00
Michael Niedermayer
de0ff4ce69 h264: Fix invalid interlaced/progressive MB combinations for direct mode prediction.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 758ec111538ccd487686e8677aa754ee4d82beaa)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:52 +01:00
Anton Khirnov
6548cb2578 libx264: add 'stats' private option for setting 2pass stats filename.
x264 always opens the file itself with fopen, so we cannot use the
standard lavc stats mechanism.

CC: libav-stable@libav.org
(cherry picked from commit d533e395e14d403948ca2424efbcee92429ef8e1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:12 +01:00
Anton Khirnov
f6257cf4b7 libx264: fix help text for slice-max-size option.
CC: libav-stable@libav.org
(cherry picked from commit 9d5c131ecec75fcfb1b4b56f74f2b2756bf0027a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:02 +01:00
Janne Grunau
d94256d36c Revert "h264: clear trailing bits in partially parsed NAL units"
This reverts commit 729ebb2f185244b0ff06d48edbbbbb02ceb4ed4e.

There was an off-by-one error in the bit mask calculation clearing
actually the last valid bit and causing
http://bugzilla.libav.org/show_bug.cgi?id=227

The broken sample (Mr_MrsSmith-h264_aac.mp4) the commit was fixing
does not work after correcting the off-by-one error.

CC: libav-stable@libav.org
(cherry picked from commit 8a6037c3900875ccab8d553d2cc659bdef2c9d0e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:56:55 +01:00
Ronald S. Bultje
7bb97a61df mpc: pad mpc_CC/SCF[] tables to allow for negative indices.
MPC8 allows indices of mpc_CC up to -1, and mpc_SCF up to -6, thus pad
the tables by that much on the left end.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d7eabd50425a61b31e90c763a0c3e4316a725404)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:48:29 +01:00
Ronald S. Bultje
c65eadee5d xxan: protect against chroma LUT overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f77bfa837636a99a4034d31916a76f7d1688cf5a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
a43f4bd601 xxan: convert to bytestream2 API.
Protects against overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 55188278169c3a1838334d7aa47a1f7a40741690)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
8f881885c2 xxan: don't read before start of buffer in av_memcpy_backptr().
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1279e286b00e99f343adb51e251f036a3df6f32)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
26521d87ba dsicinvideo: validate buffer offset before copying pixels.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c95fefa0420be9cc0f09a95041acf11114aaacd0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
e1a4143793 cook: error out on quant_index values outside [-63, 63] range.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 97e48b2f541396ef6e8816a555bac1bb993d7a6a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
b9482a6efd cook: extend channel uncoupling tables so the full bit range is covered.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 37cc8600d0313838cab5b886b9d373e5819aa24f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
88c3cc019c cook: expand dither_tab[], and make sure indexes into it don't overflow.
Fixes overflows in accessing dither_tab[].

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 442c3a8cb1785d74f8e2d7ab35b1862b7088436b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:40:29 +01:00
Ronald S. Bultje
9980e4df3b huffyuv: add padding to classic (v1) huffman tables.
We slightly overread the input buffer, so we require
padding at the end of the buffer, as is documented in the
get_bits API. Without padding, we'll read uninitialized
data or beyond the end of the .rodata, which may crash.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4ffe5e2aa5241f8da9afd2c8fbc854dcc916c5f9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:36:39 +01:00
Ronald S. Bultje
d4f2786cda avs: fix infinite loop on end-of-stream.
The codec would keep returning the last decoded frame if the stream
contains B-frames, since it wouldn't clear that frame from the list of
frames to be returned to the user.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 83f15a1228895434a982c840b09edccd1c64e800)

Conflicts:

	libavcodec/cavsdec.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:36:02 +01:00
Alex Converse
2744fdbd9e tiffdec: Prevent illegal memory access caused by recycled pointers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd0be63049ed46660993d0550a4f0847a0b942ea)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:55 +01:00
Ronald S. Bultje
1fcc2c6091 wma: fix off-by-one in array bounds check.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b4bccf3e4e58f6fe58043791ca09db01a4343fac)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:39 +01:00
Ronald S. Bultje
74871ac70a dv: check buffer size before reading profile.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e97efecec82ca8458a9bbd75a91ebf556abde362)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:21 +01:00
Ronald S. Bultje
9cb7f6e54a raw: move buffer size check up.
This way, it protects against overreads for 4bpp/2bpp content also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cc5dd632cecc5114717d0b90f8c2be162b1c6ee8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:23:58 +01:00
Ronald S. Bultje
ed6aaf579d dca: prevent accessing static arrays with invalid indexes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e6ffd997cbc06426e75d3fa291b991866c84a79b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:22:32 +01:00
Ronald S. Bultje
e1b4614ab4 lpcm: fix sample size calculation for 20bit LCPM.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1320dc3bed281bb2f3c5531c52b6a6246e2394a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:12:00 +01:00
Ronald S. Bultje
12247a13e0 Don't use ff_cropTbl[] for IDCT.
Results of IDCT can by far outreach the range of ff_cropTbl[], leading
to overreads and potentially crashes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c23acbaed40101c677dfcfbbfe0d2c230a8e8f44)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Ronald S. Bultje
9def2f200e error_resilience: initialize s->block_index[].
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6193ff68549ecbaf1a4d63a0e06964ec580ac620)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Ronald S. Bultje
7b676935ee svq3: protect against negative quantizers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 11b940a1a8e7e5d5b212935a3ce78aeda577f5f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Justin Ruggles
4a15240a27 mov: set channel layout for AC-3 streams based on the 'dac3' atom info
fixes Bug 225
(cherry picked from commit 3798205a77ce275613098ecb48645e6029811f14)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:54 +01:00
Janne Grunau
a47b96bdd3 rv34: handle size changes during frame multithreading
Factors all context dynamic memory handling to its own functions.
Fixes bug 220.
(cherry picked from commit 2bd730010da24d035639586bb13862abe36cc1b8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:54 +01:00
Alex Converse
48ac765efe rv10/20: Fix slice overflow with checked bitstream reader.
(cherry picked from commit 9243ec4a508c81a621e941bb7e012e2d45d93659)
2012-03-06 15:31:23 -08:00
Michael Niedermayer
522645e38f h263dec: Disallow width/height changing with frame threads.
Fixes CVE-2011-3937

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 71db86d53b5c6872cea31bf714a1a38ec78feaba)

Conflicts:

	libavcodec/h263dec.c

Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-06 15:28:01 -08:00
Alex Converse
e891ee4bf6 adpcm: Clip step_index values read from the bitstream at the beginning of each frame.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit bbeb29133b55b7256d18f5aaab8b5c8e919a173a)
2012-03-06 15:28:01 -08:00
Alex Converse
ef673211e7 tiff: Make the TIFF_LONG and TIFF_SHORT types unsigned.
TIFF v6.0 (unimplemented) adds signed equivalents.
(cherry picked from commit e32548d1331ce05a054f1028fcdda8823a4f215a)
2012-03-06 15:28:01 -08:00
Alex Converse
eaeaeb265f dpcm: ignore extra unpaired bytes in stereo streams.
Fixes: CVE-2011-3951

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ce7aee9b733134649a6ce2fa743e51733f33e67e)
2012-03-06 15:28:01 -08:00
Alex Converse
db315c796d svq3: Prevent illegal reads while parsing extradata.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 9e1db721c4329f4ac166a0bcc002c8d75f831aba)
2012-03-06 15:28:01 -08:00