15529 Commits

Author SHA1 Message Date
Diego Biurrun
a1b82c6b1c x86: ac3dsp: Drop mmx variant of ac3_max_msb_abs_int16
The function accidentally uses mmxext instructions, so it causes sigill
on mmx-only CPUs and provides no benefit on CPUs with mmxext available.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-29 17:03:25 +01:00
Luca Barbato
2ed8a550da aac: Check init_get_bits return value
Some code paths can call it with invalid length.

CC: libav-stable@libav.org
(cherry picked from commit 71953ebcf94fe4ef316cdad1f276089205dd1d65)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-28 14:16:37 +01:00
Luca Barbato
ef67d8107e aac: return meaningful errors
(cherry picked from commit 07c52e2c7c60b087fd023cd9771778973def0b33)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/aacdec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
8119336df4 dsicinav: K&R formatting cosmetics
(cherry picked from commit fcae3ff124ee97c9265e3b93f3d41238b2aee9bd)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dsicinav.c
2013-10-28 14:16:37 +01:00
Anton Khirnov
be8b796f55 vcr1: add sanity checks
Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8aba7968dd604aae91ee42cbce0be3dad7dceb30)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vcr1.c
2013-10-28 14:16:37 +01:00
Anton Khirnov
8297853917 pictordec: pass correct context to avpriv_request_sample
Fixes invalid reads.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry-picked from commit fe9bb61f9a16be19ad91875632c39e44b7a99a8a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/pictordec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
b8ba48c725 dsicinav: Clip the source size to the expected maximum
A packet larger than cin->bitmap_size does not make sense.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd8189932147a524fe43532b46baa35e8be92a1b)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dsicinav.c
2013-10-28 14:16:37 +01:00
Luca Barbato
1682c9fb59 alsdec: Clean up error paths
Fix at least a memory leak.

CC: libav-stable@libav.org
(cherry picked from commit ca488ad480360dfafcb5766f7bfbb567a0638979)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/alsdec.c
2013-10-28 14:16:37 +01:00
Luca Barbato
4a11d773f9 nuv: check rtjpeg_decode_frame_yuv420 return value
CC: libav-stable@libav.org
(cherry picked from commit 85ac12587bfef970d0e0e4abc292df346daf8478)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
abb41f19cc nuv: Reset the frame on resize
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
c1ebdef01b nuv: Use av_fast_realloc
The decompressed buffer can be used after codec_reinit, so it must be
preserved.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2df0776c2293efb0ac12c003843ce19332342e01)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Anton Khirnov
d2eddcfc83 nuv: return meaningful error codes.
(cherry picked from commit 3344f5cb747bb1f54cc34878b66dc0536f194720)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
36fc320747 nuv: Pad the lzo outbuf
And properly update the buf_size with the correct size.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 075dbc185521f193c98b896cd63be3ec2613df5d)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
cda26ab21e nuv: Do not ignore lzo decompression failures
Update the fate reference since the last broken frame is not decoded
anymore.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aae159a7cc4df7d0521901022b778c9da251c24e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
5312fb8287 8bps: Bound-check the input buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd7b4da0f4627bb6c4a7c2575da83fe6b261a21c)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/8bps.c
2013-10-16 23:05:50 +02:00
Luca Barbato
2da49df19e lavc: set the default rc_initial_buffer_occupancy
rc_buffer_size is not set before.

Solve the initial the rate control underflow issue reported in
bug 222.

CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bff3607547fdbb6e32b3830a351e6a33280c1e0d)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-16 22:54:30 +02:00
Luca Barbato
c25bbb6fdb 4xm: Reject not a multiple of 16 dimension
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f034f255c49050e894ab9b88087c09ebe249f3f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:12 +02:00
Luca Barbato
12dc01bb1f 4xm: do not overread the prestream buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be373cb50d3c411366fec7eef2eb3681abe48f96)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:06 +02:00
Luca Barbato
cd9b0bb07a 4xm: validate the buffer size before parsing it
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit de2e5777e225e75813daf2373c95e223651fd89a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:03 +02:00
Luca Barbato
53c76b6803 indeo: Do not reference mismatched tiles
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f9e5261cab067be7278f73d515bc9b601eb56202)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:57 +02:00
Luca Barbato
7999ff8966 indeo: Sanitize ff_ivi_init_planes fail paths
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 28dda8a691f1c723a4a9365ab85f9625f1330096)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:30 +02:00
Luca Barbato
a0b8f85f29 indeo: Bound-check before applying motion compensation
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 25a6666f6c07c6ac8449a63d7fbce0dfd29c54cd)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:41:49 +02:00
Luca Barbato
c02b9e6e63 indeo: Bound-check before applying transform
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dc79685195a45c9b8b17d7b93d118e0aefa45462)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:41:12 +02:00
Luca Barbato
efe710f8a0 indeo: reject negative array indexes
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

(cherry picked from commit 6a10142faa1cca8ba2bfe51b970754f62d60f320)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:39:02 +02:00
Luca Barbato
aedde1a48d indeo: Cosmetic formatting
Trim some overly long lines.

(cherry picked from commit 6dfacd7ab126aea1392949d1aa10fdc3d3eeb911)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:38:28 +02:00
Luca Barbato
c5da487a38 indeo: Refactor ff_ivi_init_tiles and ivi_decode_blocks
Spin large and mostly self contained blocks into stand alone
functions.

(cherry picked from commit 62256010e9bc8879e2bf7f3b94af8ff85e239082)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:19:17 +02:00
Luca Barbato
f21dce6044 indeo: Refactor ff_ivi_dec_huff_desc
Spare an indentation level.

(cherry picked from commit f6f36ca8ca1b2526d3abff7d7c627322d3bce912)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:03:27 +02:00
Martin Storsjö
5473d23ece mpegvideo: Avoid 32-bit wrapping of linesize multiplications
This makes sure that linesize * start_y doesn't overflow, so that
emulated_edge_mc can get back the original value if needed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a711a2cb473dc95708f371a82c85c97fe789b5c2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:53:02 -04:00
Michael Niedermayer
7a9af1da39 mjpegb: Detect changing number of planes in interlaced video
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit af11fa5409cc72fc45ca7f3527400beca10967b9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:52:45 -04:00
Anton Khirnov
5773065a71 pictordec: break out of both decoding loops when y drops below 0
Otherwise picmemset can get called with negative y, resulting in an
invalid write.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5f7aecde02a95451e514c809f2794c1deba80695)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:46:49 -04:00
Luca Barbato
c225c620c6 ac3: Return proper error codes
(cherry picked from commit b1f9cdc37ff5d5b391d2cd9af737ab4e5a0fc1c0)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:33 -04:00
Luca Barbato
37e69e2dee ac3: Clean up the error paths
(cherry picked from commit 818d1f1a3e89d35213af0bd5dc4a772713951882)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:22 -04:00
Luca Barbato
3dff283de1 ac3: Do not clash with normal AVERROR
The parsing function return AVERROR and AAC_AC3_PARSE_ERROR values,
make sure they are not misunderstood.

(cherry picked from commit 6258d362b82934a2c27557e0984aed372d98091a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:05 -04:00
Luca Barbato
86c169c5b6 dxa: Make sure the reference frame exists
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5ef7c84a9374681c64722a96d91741f3b990af2b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dxa.c
2013-09-23 19:43:07 -04:00
Luca Barbato
91355bec88 h261: check the mtype index
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c59967fa7cc5bc2fa06b36c17d2c207240c06b3e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/h261dec.c
2013-09-23 19:42:57 -04:00
Luca Barbato
75b1b13aff vc1: check mb_height validity.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 43bacd5b7d3d265a77cd29d8abb131057796aecc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:09 -04:00
Luca Barbato
54e0386369 vc1: check the source buffer in vc1_mc functions
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 090cd0631140ac1a3a795d2adfac5dbf5e381aa2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vc1dec.c
2013-09-23 19:41:09 -04:00
Luca Barbato
9c779b5dd0 bink: Bound check the quantization matrix.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9991298f2c4d9022ad56057f15d037e18d454157)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:08 -04:00
Luca Barbato
8006716f21 xl: Make sure the width is valid
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:25:50 -04:00
Luca Barbato
246e0e2c99 alsdec: Fix the clipping range
mcc_weightings is only 32 elements.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 70ecc175c7b513a153ac87d1c5d219556ca55070)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:22:22 -04:00
Luca Barbato
0d24adbe8d dsicinav: Bound-check the source buffer when needed
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dd0bfc3a6a310e3e3674ce7742672d689a9a0e93)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:18:39 -04:00
Martin Storsjö
729143e2d2 ac3dec: Don't consume more data than the actual input packet size
This was handled properly in the normal return case at the end
of the function, but not in this special case.

Returning a value larger than the input packet size can cause
problems for certain library users.

Returning the actual input buffer size unconditionally, since
it is not guaranteed that frame_size is set to a sensible
value at this point.

Cc: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8f24c12be7a3b3ea105e67bba9a867fe210a2333)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:14:14 -04:00
Luca Barbato
36921fcdd3 indeo: Reject impossible FRAMETYPE_NULL
A frame marked FRAMETYPE_NULL cannot be scalable and requires a
previous frame successfully decoded.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5b2a29552ca09edd4646b6aa1828b32912b7ab36)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:11:37 -04:00
Luca Barbato
e2dcb8208e indeo5: return proper error codes
(cherry picked from commit b0eeb9d442e4b7e82f6797d74245434ea33110a5)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:09:10 -04:00
Luca Barbato
609345cd5e indeo4: Validate scantable dimension
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cd78e934c246d1b2510f8fba0abfe40bb75795f6)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 18:54:13 -04:00
Luca Barbato
06c52faef2 indeo4: Check the quantization matrix index
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6255ccf7d51c82ab79bf0cd47a921f572dda4489)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 18:53:01 -04:00
Luca Barbato
ba5dfc25ee indeo4: Do not access missing reference MV
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8435bca087c0e79385763c51de009fd89390b6a5)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:

	libavcodec/indeo4.c
2013-09-23 18:52:16 -04:00
Luca Barbato
68b1008719 adpcm: Unbreak ima-dk4
Was broken by commit b9dea1a085c4705e480bd17dfa8c8ce227fdce76
2013-09-21 15:33:11 +02:00
Justin Ruggles
763519536b ac3dec: validate channel output mode against channel count
Damaged frames can lead to a mismatch, which can cause a segfault
due to using an incorrect channel mapping.

CC:libav-stable@libav.org
(cherry picked from commit d7c450436fcb9d3ecf59884a574e7684183e753d)

Conflicts:

	libavcodec/ac3dec.c
2013-09-21 15:21:54 +02:00
Luca Barbato
521cbcb7d3 dca: Respect the current limits in the downmixing capabilities
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3802833bc1f79775a1547c5e427fed6e92b77e53)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-21 15:19:06 +02:00