15441 Commits

Author SHA1 Message Date
Martin Storsjö
802deb2d13 svq3: Check for any negative return value from ff_h264_check_intra_pred_mode
Also pass on any returned error code.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1115689d54ea95a084421f5a182b8dc56cbff978)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/svq3.c
2014-01-07 09:43:56 +01:00
Martin Storsjö
290783b848 vp3: Check the framerate for validity
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 6fc8226e29055858f28973bb3d27b63b3b65e616)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit b4c479a82adbb1301e3e549cd80cdd65208ddd05)
2014-01-07 09:43:56 +01:00
Martin Storsjö
7f80928c0e cavsdec: Make sure a sequence header has been decoded before decoding pictures
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e90a6846c2c006fbebd00e1f2789f4a86fafacef)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/cavsdec.c
2014-01-07 09:43:56 +01:00
Michael Niedermayer
b81d804f2a zmbvdec: Check the buffer size for uncompressed data
Also don't pointlessly set the buffer size to 1 after copying
one packet.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0d61f260010707f3028b818e8b24598e1a83d696)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-07 09:43:56 +01:00
Martin Storsjö
7d8a4bb8d2 shorten: Break out of loop looking for fmt chunk if none is found
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b26742cc308552f242ee2bf93b07a3ff509f4edc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-07 09:43:56 +01:00
Martin Storsjö
712945d21e shorten: Use a checked bytestream reader for the wave header
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 49568851bf1700e3d9ea9cda29208d0df3c2c38b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-07 09:43:56 +01:00
Martin Storsjö
d5c104c1ae smacker: Make sure we don't fill in huffman codes out of range
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0679cec6e8802643bbe6d5f68ca1110a7d3171da)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-07 09:43:56 +01:00
Martin Storsjö
b6fc0127ce arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-12-20 22:09:24 +02:00
Michael Niedermayer
6f6cd7dbe5 roqvideodec: check dimensions validity
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae610451170cd5a28b33950006ff0bd23036845)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a52eb9f7fcd8d9167fb4a5ba015b612)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 488f87be873506abb01d67708a67c10a4dd29283)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 52b18c1fde65efac7f6e6104b76d39bf8d0a34ee)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-12-14 14:31:29 -05:00
Michael Niedermayer
b5736759ee qdm2: check array index before use, fix out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281f7ef1c29284e3a4cadfe0f227ffde1ed)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 39bec05ed42e505d17877b0c23f16322f9b5883b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0b2b8ab979624b0cce673d5e99255482d7c553ad)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-12-14 14:31:03 -05:00
Reinhard Tartler
d4a24e43ed alsdec: check block length
Fix writing over the end

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Addresses: CVE-2013-0845
(cherry picked from commit 2a0fb7286d67c47e44aa76c237ede117b22af616)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 3f7d89034bfe50893927cc92ddcb95a2e9b4178d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-12-14 14:30:24 -05:00
Diego Biurrun
62c8bf00bb x86: fft: Remove 3DNow! optimizations, they break FATE 2013-10-30 19:19:44 +01:00
Diego Biurrun
a1b82c6b1c x86: ac3dsp: Drop mmx variant of ac3_max_msb_abs_int16
The function accidentally uses mmxext instructions, so it causes sigill
on mmx-only CPUs and provides no benefit on CPUs with mmxext available.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-29 17:03:25 +01:00
Luca Barbato
2ed8a550da aac: Check init_get_bits return value
Some code paths can call it with invalid length.

CC: libav-stable@libav.org
(cherry picked from commit 71953ebcf94fe4ef316cdad1f276089205dd1d65)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-28 14:16:37 +01:00
Luca Barbato
ef67d8107e aac: return meaningful errors
(cherry picked from commit 07c52e2c7c60b087fd023cd9771778973def0b33)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/aacdec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
8119336df4 dsicinav: K&R formatting cosmetics
(cherry picked from commit fcae3ff124ee97c9265e3b93f3d41238b2aee9bd)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dsicinav.c
2013-10-28 14:16:37 +01:00
Anton Khirnov
be8b796f55 vcr1: add sanity checks
Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8aba7968dd604aae91ee42cbce0be3dad7dceb30)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vcr1.c
2013-10-28 14:16:37 +01:00
Anton Khirnov
8297853917 pictordec: pass correct context to avpriv_request_sample
Fixes invalid reads.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry-picked from commit fe9bb61f9a16be19ad91875632c39e44b7a99a8a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/pictordec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
b8ba48c725 dsicinav: Clip the source size to the expected maximum
A packet larger than cin->bitmap_size does not make sense.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd8189932147a524fe43532b46baa35e8be92a1b)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dsicinav.c
2013-10-28 14:16:37 +01:00
Luca Barbato
1682c9fb59 alsdec: Clean up error paths
Fix at least a memory leak.

CC: libav-stable@libav.org
(cherry picked from commit ca488ad480360dfafcb5766f7bfbb567a0638979)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/alsdec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
4a11d773f9 nuv: check rtjpeg_decode_frame_yuv420 return value
CC: libav-stable@libav.org
(cherry picked from commit 85ac12587bfef970d0e0e4abc292df346daf8478)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
abb41f19cc nuv: Reset the frame on resize
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
c1ebdef01b nuv: Use av_fast_realloc
The decompressed buffer can be used after codec_reinit, so it must be
preserved.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2df0776c2293efb0ac12c003843ce19332342e01)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Anton Khirnov
d2eddcfc83 nuv: return meaningful error codes.
(cherry picked from commit 3344f5cb747bb1f54cc34878b66dc0536f194720)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
36fc320747 nuv: Pad the lzo outbuf
And properly update the buf_size with the correct size.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 075dbc185521f193c98b896cd63be3ec2613df5d)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
cda26ab21e nuv: Do not ignore lzo decompression failures
Update the fate reference since the last broken frame is not decoded
anymore.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aae159a7cc4df7d0521901022b778c9da251c24e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
5312fb8287 8bps: Bound-check the input buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd7b4da0f4627bb6c4a7c2575da83fe6b261a21c)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/8bps.c
2013-10-16 23:05:50 +02:00
Luca Barbato
2da49df19e lavc: set the default rc_initial_buffer_occupancy
rc_buffer_size is not set before.

Solve the initial the rate control underflow issue reported in
bug 222.

CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bff3607547fdbb6e32b3830a351e6a33280c1e0d)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-16 22:54:30 +02:00
Luca Barbato
c25bbb6fdb 4xm: Reject not a multiple of 16 dimension
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f034f255c49050e894ab9b88087c09ebe249f3f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:12 +02:00
Luca Barbato
12dc01bb1f 4xm: do not overread the prestream buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be373cb50d3c411366fec7eef2eb3681abe48f96)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:06 +02:00
Luca Barbato
cd9b0bb07a 4xm: validate the buffer size before parsing it
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit de2e5777e225e75813daf2373c95e223651fd89a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:03 +02:00
Luca Barbato
53c76b6803 indeo: Do not reference mismatched tiles
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f9e5261cab067be7278f73d515bc9b601eb56202)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:57 +02:00
Luca Barbato
7999ff8966 indeo: Sanitize ff_ivi_init_planes fail paths
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 28dda8a691f1c723a4a9365ab85f9625f1330096)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:30 +02:00
Luca Barbato
a0b8f85f29 indeo: Bound-check before applying motion compensation
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 25a6666f6c07c6ac8449a63d7fbce0dfd29c54cd)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:41:49 +02:00
Luca Barbato
c02b9e6e63 indeo: Bound-check before applying transform
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dc79685195a45c9b8b17d7b93d118e0aefa45462)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:41:12 +02:00
Luca Barbato
efe710f8a0 indeo: reject negative array indexes
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

(cherry picked from commit 6a10142faa1cca8ba2bfe51b970754f62d60f320)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:39:02 +02:00
Luca Barbato
aedde1a48d indeo: Cosmetic formatting
Trim some overly long lines.

(cherry picked from commit 6dfacd7ab126aea1392949d1aa10fdc3d3eeb911)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:38:28 +02:00
Luca Barbato
c5da487a38 indeo: Refactor ff_ivi_init_tiles and ivi_decode_blocks
Spin large and mostly self contained blocks into stand alone
functions.

(cherry picked from commit 62256010e9bc8879e2bf7f3b94af8ff85e239082)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:19:17 +02:00
Luca Barbato
f21dce6044 indeo: Refactor ff_ivi_dec_huff_desc
Spare an indentation level.

(cherry picked from commit f6f36ca8ca1b2526d3abff7d7c627322d3bce912)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:03:27 +02:00
Martin Storsjö
5473d23ece mpegvideo: Avoid 32-bit wrapping of linesize multiplications
This makes sure that linesize * start_y doesn't overflow, so that
emulated_edge_mc can get back the original value if needed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a711a2cb473dc95708f371a82c85c97fe789b5c2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:53:02 -04:00
Michael Niedermayer
7a9af1da39 mjpegb: Detect changing number of planes in interlaced video
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit af11fa5409cc72fc45ca7f3527400beca10967b9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:52:45 -04:00
Anton Khirnov
5773065a71 pictordec: break out of both decoding loops when y drops below 0
Otherwise picmemset can get called with negative y, resulting in an
invalid write.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5f7aecde02a95451e514c809f2794c1deba80695)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:46:49 -04:00
Luca Barbato
c225c620c6 ac3: Return proper error codes
(cherry picked from commit b1f9cdc37ff5d5b391d2cd9af737ab4e5a0fc1c0)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:33 -04:00
Luca Barbato
37e69e2dee ac3: Clean up the error paths
(cherry picked from commit 818d1f1a3e89d35213af0bd5dc4a772713951882)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:22 -04:00
Luca Barbato
3dff283de1 ac3: Do not clash with normal AVERROR
The parsing function return AVERROR and AAC_AC3_PARSE_ERROR values,
make sure they are not misunderstood.

(cherry picked from commit 6258d362b82934a2c27557e0984aed372d98091a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:05 -04:00
Luca Barbato
86c169c5b6 dxa: Make sure the reference frame exists
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5ef7c84a9374681c64722a96d91741f3b990af2b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dxa.c
2013-09-23 19:43:07 -04:00
Luca Barbato
91355bec88 h261: check the mtype index
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c59967fa7cc5bc2fa06b36c17d2c207240c06b3e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/h261dec.c
2013-09-23 19:42:57 -04:00
Luca Barbato
75b1b13aff vc1: check mb_height validity.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 43bacd5b7d3d265a77cd29d8abb131057796aecc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:09 -04:00
Luca Barbato
54e0386369 vc1: check the source buffer in vc1_mc functions
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 090cd0631140ac1a3a795d2adfac5dbf5e381aa2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vc1dec.c
2013-09-23 19:41:09 -04:00
Luca Barbato
9c779b5dd0 bink: Bound check the quantization matrix.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9991298f2c4d9022ad56057f15d037e18d454157)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:08 -04:00