15513 Commits

Author SHA1 Message Date
Janne Grunau
6f4404b24b h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2014-05-31 20:07:52 -04:00
Martin Storsjö
110680c5a2 alac: Limit max_samples_per_frame
Otherwise buffer size calculations in allocate_buffers could
overflow later, making the code think a large enough buffer
actually was allocated.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
2014-05-31 20:07:52 -04:00
Rafaël Carré
65c3593792 apedec: do not buffer decoded samples over AVPackets
Only consume an AVPacket when all the samples have been read.

When the rate of samples output is limited (by the default value
of max_samples), consuming the first packet immediately will cause
timing problems:

- The first packet with PTS 0 will output 4608 samples and be
consumed entirely
- The second packet with PTS 64 will output the remaining samples
(typically, a lot, that's why max_samples exist) until the decoded
samples of the first packet have been exhausted, at which point the
samples of the second packet will be decoded and output when
av_decode_frame is called with the next packet).

That means there's a PTS jump since the first packet is 'decoded'
immediately, which can be seen with avplay or mplayer: the timing
jumps immediately to 6.2s (which is the size of a packet).

Sample: http://streams.videolan.org/issues/6348/Goldwave-MAClib.ape

Bug-Debian: http://bugs.debian.org/744901
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 91d4cfb8127f1de6c4ad173a30fffe584700046d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-05-31 20:07:52 -04:00
Anton Khirnov
079758e49a h264: reset next_output_pic earlier in start_frame()
In case start_frame() fails, this potentially invalid frame can still be
output to the caller.

Bug-Id: 672
Bug-Id: debian/741240
Bug-Id: ubuntu/1288206
2014-05-31 20:07:51 -04:00
Justin Ruggles
a0a90b1a11 tiffdec: use bytestream2 to simplify overread/overwrite protection
Based on a patch by Paul B Mahol <onemda@gmail.com>

CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Justin Ruggles
fa60904ebd bytestream: add bytestream2_copy_buffer() functions
This is basically an overread/overwrite-safe memcpy between a
GetByteContext and a PutByteContext.

CC:libav-stable@libav.org
(cherry picked from commit 5748faf291fec297ef25d81962b52b3438f54278)
2014-05-31 20:05:19 -04:00
Paul B Mahol
b473fdcde3 bytestream: add functions for accessing size of buffer
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

CC:libav-stable@libav.org
(cherry picked from commit de9d2705f61ef569487ec5f8974a9c7ce34ec783)
2014-05-31 20:05:19 -04:00
Janne Grunau
9938e450c8 mpeg12: check scantable indices in all decode_block functions
Add checks to the fast functions used with CODEC_FLAGS2_FAST and move
the check for all other functions to before the invalid memory is
accessed. Fixes https://trac.videolan.org/vlc/ticket/9713 with
CODEC_FLAGS2_FAST.

CC: libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
71b8c8430c sgidec: fix buffer size check in expand_rle_row()
Right now it will spuriously fail if the linesize is exactly equal to
the data width.

CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
d0ecfe3249 adx: check that the offset is not negative
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 5569146d48f06564e8fa393424782cceed510916)
2014-05-31 20:05:19 -04:00
Anton Khirnov
07558d0b9f mpegvideo: set reference/pict_type on generated reference frames
Otherwise the generic code will unref them, which can then result in
last_picture_ptr == current_picture_ptr, which causes deadlocks at least
in rv40.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
27ac9585c9 h264: reset data partitioning at the beginning of each decode call
Prevents using GetBitContexts with data from previous calls.

Fixes access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
35ba079fbf h264: reset ref count if decoding the slice header fails
Otherwise the ER code might try to use some already freed references.

Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
a7cce9ebf3 h264: reset first_field if frame_start() fails for missing refs
In this case we may not have a current frame, while first_field being
set implies we do.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
51ae8e26af h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
c4033cd4eb h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:19 -04:00
Anton Khirnov
7f33a24e82 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:18 -04:00
Anton Khirnov
0f71a5df4b h264: do not use 422 functions for monochrome
Fixes invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:18 -04:00
Anton Khirnov
3ee26080d6 h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:18 -04:00
Anton Khirnov
e0d8a17402 h264_refs: make sure not to write over the bounds of the default ref list
Fixes invalid writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:18 -04:00
Anton Khirnov
2cbc8dfedd h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-05-31 20:05:18 -04:00
Keiji Costantini
16f0f97eec ituh263: reject b-frame with pp_time = 0
Avoid a division by 0 in ff_mpeg4_set_one_direct_mv.

Sample-Id: 00000168-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit 9514440337875e0c63b409abcd616b68c518283f)
(cherry picked from commit 5df52b0131d3d4d804ad6e221bc9a2cd8b201ef2)
(cherry picked from commit aa2a3ca27a3269e2b975686652204607fad8bc49)
2014-05-31 20:03:35 -04:00
Janne Grunau
0120e480bf arm: hpeldsp: fix put_pixels8_y2_{,no_rnd_}armv6
The overread avoidance fix in cbddee1cca0ebd01e8c5aa694d31228eb4de4b41
broke the computation for the last row since it prevented the safe
reading from the height+1-th row.
2014-03-09 00:31:31 +01:00
Janne Grunau
fd2fc130b2 arm: hpeldsp: prevent overreads in armv6 asm
Based on a patch by Russel King <rmk+libav@arm.linux.org.uk>

Bug-Id: 646
CC: libav-stable@libav.org
2014-03-06 09:06:39 +01:00
Anton Khirnov
3da4fdd5ac lagarith: reallocate rgb_planes when needed
Fixes invalid writes on pixel format changes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 4c3e1956ee35fdcc5ffdb28782050164b4623c0b)
(cherry picked from commit bd57e783437f990c3ac4747eeebe20332e103980)
2014-02-28 23:07:41 -05:00
Anton Khirnov
2fb0a52e70 lagarith: avoid infinite loop in lag_rac_refill()
range == 0 happens with corrupted files

CC:libav-stable@libav.org
(cherry picked from commit de6dfa2bb82df916a67e5036b0ef96a944781ed3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 8bce2c60b8ebc31899d576dde3bbe6205faae97d)
2014-02-28 23:07:40 -05:00
Ronald S. Bultje
2c1d84499b lagarith: pad RGB buffer by 1 byte.
For left HFYU prediction, we predict from the buffer buf+1 using 8- or
16-byte reads. This means that aligning the buffer by 16 bytes is in
itself not sufficient, because if the width itself is 16- or 8-byte
aligned, the buffer will not be padded, and thus a read of size 16 at
buf+1 will overflow boundaries at the right edge. Padding the buffer by
1 byte is sufficient to not overflow its boundaries.

Fixes bug 342.

(cherry picked from commit 98d0d19208959766a58f13dd6a678d1f765a26ac)
2014-02-28 23:07:40 -05:00
Anton Khirnov
de0e442e9d truemotion1: check the header size
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 2240e2078d53d3cfce8ff1dda64e58fa72038602)
(cherry picked from commit 76b40a9bf93e387d98aa7dc02ec7a8d13f51722f)
2014-02-28 23:07:40 -05:00
Anton Khirnov
43aa7eb38e shorten: pad the internal bitstream buffer
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 1713eec29add37b654ec6bf262b843d139c1ffc6)
(cherry picked from commit 5881ec0ea58a95403bd375b63f22d49905cdd8e5)
2014-02-28 23:07:40 -05:00
Luca Barbato
4279e0e8d0 h264: Fix a typo from the previous commit
f777504f640260337974848c7d5d7a3f064bbb45 changed a - in +

CC: libav-stable@libav.org
(cherry picked from commit d922c5a5fbaf0b6c73bd8c81ae059bc6e406961c)
(cherry picked from commit 3ce77e04c2ca4b9e7fa6b94b51e8d7c5f188da86)
(cherry picked from commit 8cba6f58c8acaa0ca6749110a2746bbe60ff2dab)
2014-02-28 23:07:40 -05:00
Vittorio Giovara
a6003760bd h264: Lower bound check for slice offsets
And use the value from the specification.

Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit f777504f640260337974848c7d5d7a3f064bbb45)
(cherry picked from commit 5bd083d0216d9ee649039c84999fb61386536ac1)

Conflicts:
	libavcodec/h264.c

(cherry picked from commit 41380e017afcca3119acb560c08a60a97d416c3c)

Conflicts:
	libavcodec/h264.c
2014-02-28 23:07:40 -05:00
Anton Khirnov
cf676c159b rpza: limit the number of blocks to the total remaining blocks in the frame
Fixes invalid writes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 77bb0004bbe18f1498cfecdc68db5f10808b6599)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-02-14 11:43:59 +01:00
Michael Niedermayer
30c8a5e4f6 vc1: Always reset numref when parsing a new frame header.
Fixes an issue where the B-frame coding mode switches from interlaced
fields to interlaced frames, causing incorrect decisions in the motion
compensation code and resulting in visual artifacts.

CC: libav-stable@libav.org
Signed-off-by: Tim Walker <tdskywalker@gmail.com>
(cherry picked from commit dd2d0039b6405dc724e4fef0d5b8f49530eea3aa)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 3cc8d9bc1ffc6c0888960fb009f12fa3047bb663)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-02 12:48:34 -05:00
Anton Khirnov
716ee73c99 h264: reset num_reorder_frames if it is invalid
An invalid VUI is not considered a fatal error, so the SPS containing it
may still be used. Leaving an invalid value of num_reorder_frames there
can result in writing over the bounds of H264Context.delayed_pic.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9ecabd7892ff073ae60ded3fc0a1290f5914ed5c)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/h264_ps.c

(cherry picked from commit 299c5dcfb0cd3debdf07943edfb46f4aeb02ca91)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-02 12:48:16 -05:00
Anton Khirnov
979f77b0dc h264: check that an IDR NAL only contains I slices
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 8b2e5e42bb9d6a59ede5af2e6df4aaf7750d1195)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 62ed6da016b789eee00e0fff517df4a254e12e5d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/h264.c
2014-02-02 12:48:04 -05:00
Anton Khirnov
23144c5f06 h264_cavlc: check the size of the intra PCM data.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit b5275ca1a805436ca12540c34dd5ed1671877434)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-02 12:41:02 -05:00
Luca Barbato
e964207e6c cavs: Check for negative cbp
Sample-Id: 00000647-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c85e5f13f6ac9c4c90125e7671d89009e57f9df9)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/cavsdec.c
2014-02-02 12:36:15 -05:00
Michael Niedermayer
cb5d0ea0be flashsv: Check diff_start diff_height values
Fix out of array accesses.

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Adresses: CVE-2013-7015
(cherry picked from commit 57070b1468edc6ac8cb3696c817f3c943975d4c1)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 10d48fe6d3963842319b1d8d738a318020836e72)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 14:06:24 -05:00
Michael Niedermayer
ef6c90e102 dsputil/pngdsp: fix signed/unsigned type in end comparison
Fixes out of array accesses and integer overflows.

(cherry picked from commit d1916d13e28b87f4b1b214231149e12e1d536b4b)
Adresses: CVE-2013-7010, CVE-2013-7014

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit af9799790d7a6342027e0261b5dd87657abb7a0b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/pngdsp.c
2014-02-01 14:05:47 -05:00
Michael Niedermayer
d04194db45 vqavideo: check chunk sizes before reading chunks
Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bfa1e20127a16392a0b85a4aa4840889)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767b922661132a3c1f4b5ba2c7338b660)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f7d18deb73d1dd1b27b2c7062c9a10d168a6c62a)

Addresses: CVE-2013-0865

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit ab434bf0d051008a329d49d0256faa5d64e2bf4d)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-02-01 14:03:28 -05:00
Luca Barbato
a89acaa0b0 get_bits: change the failure condition in init_get_bits
Too much code relies in having init_get_bits fed with a valid
buffer and set its dimension to 0.

Check for NULL buffer instead.

(cherry picked from commit 4603ec85ed620e585fc6e2e072c99858ed421855)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-25 01:01:25 +01:00
Luca Barbato
8b24e17d09 twinvq: Cope with gcc-4.8.2 miscompilation
Apparently gcc-4.8.2 miscompiles enums resulting in a lucky fpe soon
after it.

Passing the enum value as integer makes the ftype == FT_PPC condition
evaluates correctly.
2014-01-07 14:21:53 +01:00
Ben Jackson
1123870879 pthread: Avoid spurious wakeups
pthread_wait_cond can wake up unexpectedly (Wikipedia: Spurious_wakeup).

The FF_THREAD_SLICE thread mechanism could spontaneously execute
jobs or allow the caller of avctx->execute to return before all
jobs were complete.

Test both cases to ensure the wakeup is real.

Signed-off-by: Ben Jackson <ben@ben.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 311583e7798237be5cc531d672a9e37f8c729d83)
2014-01-07 09:43:58 +01:00
Derek Buitenhuis
48d57650f1 pthread: Fix deadlock during thread initialization
Sometimes, if pthread_create() failed, then pthread_cond_wait() could
accidentally be called in the worker threads after the uninit function
had already called pthread_cond_broadcast(), leading to a deadlock.

Don't call pthread_cond_wait() if c->done is set.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
(cherry picked from commit 1a5a6ac01b0ad2cf3d2128372ea41f3c1cfc2d3f)
2014-01-07 09:43:58 +01:00
Martin Storsjö
371659d1ad mpegvideo: Initialize chroma_*_shift and codec_tag even if the size is 0
This fixes breakage in a few fate tests on certain setups
(that for some reason didn't break on OS X) after the previous
commit (8812a8057). Currently, some video streams are initialized
in ff_MPV_common_init with width/height set at 0 and only changed
to a proper video size with ff_MPV_common_frame_size_change later.

The breakage was diagnosed by Anton Khirnov.

Signed-off-by: Martin Storsjö <martin@martin.st>
2014-01-07 09:43:58 +01:00
Michael Niedermayer
c00e491aeb vc1dec: Don't decode slices when the latest slice header failed to decode
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Conflicts:
	libavcodec/vc1dec.c
2014-01-07 09:43:58 +01:00
Martin Storsjö
9925f7df0a vc1dec: Make sure last_picture is initialized in vc1_decode_skip_blocks
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5e25fdbfe01635cfc650ac4adc27d434b2df0d64)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/vc1dec.c
(cherry picked from commit 494f2d4f9e834db1eaf1a7d0160d497f9802013d)
2014-01-07 09:43:58 +01:00
Martin Storsjö
fbc52044f3 fraps: Make the input buffer size checks more strict
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Conflicts:
	libavcodec/fraps.c
2014-01-07 09:43:58 +01:00
Martin Storsjö
49c1defee5 svq3: Avoid a division by zero
If the height is zero, the decompression will probably end up
failing due to not fitting into the allocated buffer later
anyway, so this doesn't need any more elaborate check.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 601c2015bc16f0b281160292a6a760cbbbb0eacb)
2014-01-07 09:43:58 +01:00
Martin Storsjö
591d5281f5 twinvqdec: Check the ibps parameter separately
This is required, since invalid parameters actually could
pass the switch check below.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit c77d409bf95954aceb762dd800d1ee2868c4b0d4)
(cherry picked from commit 9b9aee27f4e43b4a6b0884f8a6f49eb0289d7c09)
2014-01-07 09:43:58 +01:00