17947 Commits

Author SHA1 Message Date
Michael Niedermayer
558c1f35fa avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a89761964bdd0a023eff6d37a1131fb7e1d7a0)

Conflicts:
	libavcodec/h264_refs.c
2013-10-26 01:27:53 +02:00
Michael Niedermayer
8096691a68 avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f17b64723884c9fa72cfcbd369a1ba2)

Conflicts:
	libavcodec/h264_refs.c
2013-10-24 11:49:04 +02:00
Luca Barbato
abb41f19cc nuv: Reset the frame on resize
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
36fc320747 nuv: Pad the lzo outbuf
And properly update the buf_size with the correct size.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 075dbc185521f193c98b896cd63be3ec2613df5d)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
4a11d773f9 nuv: check rtjpeg_decode_frame_yuv420 return value
CC: libav-stable@libav.org
(cherry picked from commit 85ac12587bfef970d0e0e4abc292df346daf8478)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
c1ebdef01b nuv: Use av_fast_realloc
The decompressed buffer can be used after codec_reinit, so it must be
preserved.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2df0776c2293efb0ac12c003843ce19332342e01)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
cda26ab21e nuv: Do not ignore lzo decompression failures
Update the fate reference since the last broken frame is not decoded
anymore.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aae159a7cc4df7d0521901022b778c9da251c24e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Anton Khirnov
d2eddcfc83 nuv: return meaningful error codes.
(cherry picked from commit 3344f5cb747bb1f54cc34878b66dc0536f194720)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/nuv.c
2013-10-16 23:05:51 +02:00
Luca Barbato
5312fb8287 8bps: Bound-check the input buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd7b4da0f4627bb6c4a7c2575da83fe6b261a21c)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/8bps.c
2013-10-16 23:05:50 +02:00
Luca Barbato
2da49df19e lavc: set the default rc_initial_buffer_occupancy
rc_buffer_size is not set before.

Solve the initial the rate control underflow issue reported in
bug 222.

CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit bff3607547fdbb6e32b3830a351e6a33280c1e0d)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-10-16 22:54:30 +02:00
Luca Barbato
c25bbb6fdb 4xm: Reject not a multiple of 16 dimension
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f034f255c49050e894ab9b88087c09ebe249f3f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:12 +02:00
Luca Barbato
12dc01bb1f 4xm: do not overread the prestream buffer
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be373cb50d3c411366fec7eef2eb3681abe48f96)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:06 +02:00
Luca Barbato
cd9b0bb07a 4xm: validate the buffer size before parsing it
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit de2e5777e225e75813daf2373c95e223651fd89a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 22:37:03 +02:00
Luca Barbato
53c76b6803 indeo: Do not reference mismatched tiles
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f9e5261cab067be7278f73d515bc9b601eb56202)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:57 +02:00
Luca Barbato
7999ff8966 indeo: Sanitize ff_ivi_init_planes fail paths
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 28dda8a691f1c723a4a9365ab85f9625f1330096)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:48:30 +02:00
Luca Barbato
a0b8f85f29 indeo: Bound-check before applying motion compensation
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 25a6666f6c07c6ac8449a63d7fbce0dfd29c54cd)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:41:49 +02:00
Luca Barbato
c02b9e6e63 indeo: Bound-check before applying transform
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dc79685195a45c9b8b17d7b93d118e0aefa45462)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:41:12 +02:00
Luca Barbato
efe710f8a0 indeo: reject negative array indexes
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

(cherry picked from commit 6a10142faa1cca8ba2bfe51b970754f62d60f320)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:39:02 +02:00
Luca Barbato
aedde1a48d indeo: Cosmetic formatting
Trim some overly long lines.

(cherry picked from commit 6dfacd7ab126aea1392949d1aa10fdc3d3eeb911)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/ivi_common.c
2013-09-29 21:38:28 +02:00
Luca Barbato
c5da487a38 indeo: Refactor ff_ivi_init_tiles and ivi_decode_blocks
Spin large and mostly self contained blocks into stand alone
functions.

(cherry picked from commit 62256010e9bc8879e2bf7f3b94af8ff85e239082)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:19:17 +02:00
Luca Barbato
f21dce6044 indeo: Refactor ff_ivi_dec_huff_desc
Spare an indentation level.

(cherry picked from commit f6f36ca8ca1b2526d3abff7d7c627322d3bce912)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-29 21:03:27 +02:00
Mashiat Sarker Shakkhar
89d56f3be1 vc1dec: Do not use random pred_flag if motion vector data is skipped
This fixes SA10143.vc1 from test-suite. Also partially fixes MC-VC1.ts
from videolan streams archive.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 082829520e2191625d3c41ed6ad0522e8d27ebe1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 03:03:40 +02:00
Michael Niedermayer
f66ecdb1b4 avcodec: add emuedge_linesize_type
Currently all uses of the emu edge code as well as the code itself
assume int linesize
changing some but not changing all would introduce a security issue
once all use this typedef a simple search and replace can be
done to switch them all to ptrdiff_t

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2ffead98ddd384f61cdf6b1cb3f36592f54cd34a)

Conflicts:

	libavcodec/mpegvideo_common.h
	libavcodec/videodsp.h
	libavcodec/videodsp_template.c
	libavcodec/x86/videodsp_init.c
2013-09-27 02:52:04 +02:00
Michael Niedermayer
f6057c5a62 avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2013-09-27 02:52:04 +02:00
Michael Niedermayer
f3dc3bef4b avcodec/truemotion2: Fix av_freep arguments
Fixes null pointer dereference
Fixes Ticket2944

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c54aa2fb0f869ec025933944cbd1634fffe95d09)

Conflicts:

	libavcodec/truemotion2.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 02:52:04 +02:00
Michael Niedermayer
4a5bb426e2 avcodec/mjpegdec: Add some sanity checks to ljpeg_decode_rgb_scan()
These prevent the rgb ljpeg code from being run on parameters that it doesnt
support. No testcase available but it seems possible to trigger these.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61c68000eda643dfce96dc46b488d39fd5c4e309)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 02:52:04 +02:00
Michael Niedermayer
a99aff4e4b avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 02:52:04 +02:00
Michael Niedermayer
9300b1f64e avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6a527d8bc807d09b93f971c0fe0bb07)

Conflicts:

	libavcodec/pngdsp.c
2013-09-27 02:52:04 +02:00
Michael Niedermayer
1a311ad99a jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 00:35:00 +02:00
Michael Niedermayer
ef8145270f avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 00:18:12 +02:00
Michael Niedermayer
e288124394 avcodec/flashsv: check diff_start/height
Fixes out of array accesses
Fixes Ticket2844

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880c73cd76109697447fbfbaa8e5ee5683309446)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-27 00:18:12 +02:00
Michael Niedermayer
20854f9bff avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9abfb2ae75bb32bc44e2c34194c8dc40a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-26 23:24:52 +02:00
Michael Niedermayer
c8ca385676 Merge commit 'b0ca5fef09d1b1268ea0c8f89bf53cd38aaa85e7' into release/0.10
* commit 'b0ca5fef09d1b1268ea0c8f89bf53cd38aaa85e7':
  dv: Add a guard to not overread the ppcm array
  mpegvideo: Avoid 32-bit wrapping of linesize multiplications
  mjpegb: Detect changing number of planes in interlaced video
  matroskadec: Check that .lang was allocated and set before reading it
  ape demuxer: check for EOF in potentially long loops
  lavf: avoid integer overflow when estimating bitrate
  pictordec: break out of both decoding loops when y drops below 0
  ac3: Return proper error codes

Conflicts:
	libavcodec/pictordec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-26 00:18:12 +02:00
Michael Niedermayer
c6a4397410 Merge commit '37e69e2dee7c5167083bb42d669f73f038111a79' into release/0.10
* commit '37e69e2dee7c5167083bb42d669f73f038111a79':
  ac3: Clean up the error paths
  ac3: Do not clash with normal AVERROR
  dxa: Make sure the reference frame exists
  h261: check the mtype index
  segafilm: Error out on impossible packet size
  ogg: Always alloc the private context in vorbis_header
  vc1: check mb_height validity.

Conflicts:
	libavcodec/h261dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 23:35:30 +02:00
Michael Niedermayer
210a437e10 Merge commit '54e03863691dcae73260f70108b3731b70773e7c' into release/0.10
* commit '54e03863691dcae73260f70108b3731b70773e7c':
  vc1: check the source buffer in vc1_mc functions
  bink: Bound check the quantization matrix.
  xl: Make sure the width is valid
  alsdec: Fix the clipping range
  dsicinav: Bound-check the source buffer when needed
  mov: Do not allow updating the time scale after it has been set
  ac3dec: Don't consume more data than the actual input packet size
  indeo: Reject impossible FRAMETYPE_NULL

Conflicts:
	libavcodec/alsdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 23:03:08 +02:00
Michael Niedermayer
3308b443f9 Merge commit 'e2dcb8208e8f6cffef58a85127765047f5ef8868' into release/0.10
* commit 'e2dcb8208e8f6cffef58a85127765047f5ef8868':
  indeo5: return proper error codes
  indeo4: Validate scantable dimension

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 22:55:03 +02:00
Michael Niedermayer
91ad27e8f5 Merge commit '06c52faef27e5bded4ceda7e6d1541f9fb20e84c' into release/0.10
* commit '06c52faef27e5bded4ceda7e6d1541f9fb20e84c':
  indeo4: Check the quantization matrix index
  indeo4: Do not access missing reference MV

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-25 22:54:14 +02:00
Martin Storsjö
5473d23ece mpegvideo: Avoid 32-bit wrapping of linesize multiplications
This makes sure that linesize * start_y doesn't overflow, so that
emulated_edge_mc can get back the original value if needed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a711a2cb473dc95708f371a82c85c97fe789b5c2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:53:02 -04:00
Michael Niedermayer
7a9af1da39 mjpegb: Detect changing number of planes in interlaced video
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit af11fa5409cc72fc45ca7f3527400beca10967b9)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:52:45 -04:00
Anton Khirnov
5773065a71 pictordec: break out of both decoding loops when y drops below 0
Otherwise picmemset can get called with negative y, resulting in an
invalid write.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5f7aecde02a95451e514c809f2794c1deba80695)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:46:49 -04:00
Luca Barbato
c225c620c6 ac3: Return proper error codes
(cherry picked from commit b1f9cdc37ff5d5b391d2cd9af737ab4e5a0fc1c0)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:33 -04:00
Luca Barbato
37e69e2dee ac3: Clean up the error paths
(cherry picked from commit 818d1f1a3e89d35213af0bd5dc4a772713951882)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:22 -04:00
Luca Barbato
3dff283de1 ac3: Do not clash with normal AVERROR
The parsing function return AVERROR and AAC_AC3_PARSE_ERROR values,
make sure they are not misunderstood.

(cherry picked from commit 6258d362b82934a2c27557e0984aed372d98091a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:45:05 -04:00
Luca Barbato
86c169c5b6 dxa: Make sure the reference frame exists
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5ef7c84a9374681c64722a96d91741f3b990af2b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/dxa.c
2013-09-23 19:43:07 -04:00
Luca Barbato
91355bec88 h261: check the mtype index
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c59967fa7cc5bc2fa06b36c17d2c207240c06b3e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/h261dec.c
2013-09-23 19:42:57 -04:00
Luca Barbato
54e0386369 vc1: check the source buffer in vc1_mc functions
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 090cd0631140ac1a3a795d2adfac5dbf5e381aa2)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vc1dec.c
2013-09-23 19:41:09 -04:00
Luca Barbato
75b1b13aff vc1: check mb_height validity.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 43bacd5b7d3d265a77cd29d8abb131057796aecc)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:09 -04:00
Luca Barbato
9c779b5dd0 bink: Bound check the quantization matrix.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9991298f2c4d9022ad56057f15d037e18d454157)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:41:08 -04:00
Luca Barbato
8006716f21 xl: Make sure the width is valid
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:25:50 -04:00
Luca Barbato
246e0e2c99 alsdec: Fix the clipping range
mcc_weightings is only 32 elements.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 70ecc175c7b513a153ac87d1c5d219556ca55070)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-09-23 19:22:22 -04:00