If there is an error during frame parsing, but AVCodecContext.channels was
changed and AC3DecodeContext.out_channels was set previously, the two may not
match.
Fixes CVE-2012-2802
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.
This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.
With this change, the remaining bytes are skipped so the whole packet gets
consumed.
CC:libav-stable@libav.org
Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The decode function assumes that the video will have those dimensions.
Fixes CVE-2012-2801
CC:libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Fixes a part of CVE-2012-2795
CC:libav-stable@libav.org
Based on a patch by Michael Niedermayer <michaelni@gmx.at>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Fixes a part of CVE-2012-2795
CC:libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Fixes a part of CVE-2012-2795
CC:libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.
Based on a patch by Michael Niedermayer <michaelni@gmx.at>
Fixes CVE-2012-2800
CC:libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.
Fixes CVE-2012-2794
CC:libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This prevents decoding happening on a half initialized context.
Fixes CVE-2012-2779
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Our decoder does not support changing w/h.
Fixes CVE-2012-2777 and CVE-2012-2784.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The value should be always 3, as it follows from the specification.
Fix a stack buffer overflow in exponents_from_scale_factors as reported
by asan. Thanks to Dale Curtis for the sample vector.
This way we can directly remap channels from Opus' channel order to
libav's internal channel order, instead of mapping channels from
Opus' order to Vorbis' order then to libav's order.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Do not use rlelen field for buffer size in init_get_bits, it is
only the size of the data for the first field.
Since it is not reliable, just use the size of the whole buffer.
Additional comments add removal of unused rlelen variable by
Reimar Döffinger.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This avoids using the global_quality field and QSCALE flag for
passing the VBR modes, since the value range of the global_quality
field doesn't really map cleanly to this codec's VBR modes.
Signed-off-by: Martin Storsjö <martin@martin.st>
These modes were not originally exposed by the library at all.
In practice, only a few of them work for each sample rate/profile
combination, and they don't work at all for the more uncommon
sample rates.
Signed-off-by: Martin Storsjö <martin@martin.st>
Not all applications (e.g. MPlayer) set block_align, and
when using a different demuxer it might not even be
easily available.
So fall back to selecting mode based on bit rate as before
if block_align has not useful value.
It can't be worse than failing to decode completely.
(cherry picked from commit 1d0d63052b)
CC: libav-stable@libav.org
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Inline functions declared without extern do not provide an external
definition in standard C99. This code only works because most
compilers do not implement the inline semantics correctly. With a
stricter compiler, linking fails with unresolved references to these
functions.
Declaring the functions extern inline works correctly with some
compilers while some others still fail to create external definitions.
For maximum portability, create a static inline version with an
externally visible wrapper for ff_get_mb_score. ff_epzs_motion_search
is so large that no sane compiler inlines it anyway, so there the
inline keyword can simply be dropped with no effect.
Signed-off-by: Mans Rullgard <mans@mansr.com>
Adds a flag context_reinit to MpegEncContext to relieable keep track
of frame parameter changes which require a context reinitialization.
This is required for broken inputs which change the frame size but
error out before the context can be reinitialized.
This is mainly required for frame parameter changes during frame based
multithreading but single threaded usage profits too from avoiding
ff_MPV_common_end()/ff_MPV_common_init() cycles.
ALS spec:
11.6.3.1.1 Quantization and encoding of parcor coefficients
...
In all cases the resulting quantized values ak are restricted to the range [-64,63].
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
Fixes out of array write in quant_cof.
Also make sure no invalid opt_order stays in the context.
Fixes CVE-2012-2775
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
Values that fail this check will cause failure of decode_rice()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
These arguments are either constants or copies of MpegEncContext
fields just as easily accessed within the function.
Signed-off-by: Mans Rullgard <mans@mansr.com>
These functions do not benefit from being inlined. They are large,
and there are no opportunities for constant propagation.
Signed-off-by: Mans Rullgard <mans@mansr.com>
In both usages of FASTDIV the denominator might be 1.
Using a branch could make the function slower than using a normal
division.
Both denominator and numerator can be multiplied by 2 safely and
using shifts is faster than using a branch.
For some reason add_hfyu_median_prediction_cmov is only selected
on 3Dnow-capable CPUs, even though it uses no 3Dnow instructions.
This patch allows it to be selected on any cpu with cmov with the
possibility of being overridden by the mmxext version.
Signed-off-by: Mans Rullgard <mans@mansr.com>
It calculates the sum of power of two series, which can be done in one step.
Suggested by Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
That division can be replaced with a comparison:
((c->value - c->low) << 1) + 1 >= range
By expanding 'range' definition and simplifying this inequation we obtain
the final expression.
Suggested by Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
Prevents dangling pointers and makes access after free more obvious.
Setting AVFrame.qscale_table to NULL is required for successfully
allocating a previously freed Picture with ff_alloc_picture().
It is possible in various error paths as well as gap handling
that this has already been allocated. It is not clear why that
would be a problem with the current code, thus disable the
assert to avoid a common assert failure when asserts are enabled.
Signed-off-by: Martin Storsjö <martin@martin.st>
The data in coded_frame isn't allocated using get_buffer, but
is copied from the input frame to the encoder, so we should
not try to free it ourselves.
This fixes an assert failure when running in debug mode.
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously, the put_bits call writing the value wrote a value
larger than the number of bits specified, failing asserts
in debug mode. There was no actual bitstream writer corruption,
since the overwritten bit already always was set to 1.
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously, the value given to put_bits was 10 bits long for positive
predictors, even though 9 bits were to be written. The extra bit could
in some cases overwrite existing bits in the bitstream writer cache.
This fixes a failed assert in put_bits.h, when running a version
built with -DDEBUG.
The fate test result gets slightly improved, thanks to getting rid
of the overwritten bits in the bitstream writer cache.
Signed-off-by: Martin Storsjö <martin@martin.st>
This way it won't interfere with WMV9 initialisation inside MSS2 decoder and
avplay will play it fine.
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
Also, align the mangled RGB planes, which is required for the
SIMD versions of dsputils' median predict.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The optimized dct_quantize template functions reference optimized
fdct symbols, so these functions must only be enabled if the relevant
optimizations have been enabled by configure.
Using the malloc variant avoids pointless memcpy on size
increase and simplifies handling allocation failure.
Also change code to ensure that allocation, bswap and bitstream
reader all use the same size, even when the packet size is odd
for example.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
This fixes building with DEBUG defined after the function was made
static and the prototype removed in d7f9786cbc.
Signed-off-by: Martin Storsjö <martin@martin.st>
This is a preparatory step for the MSS2 decoder which needs to use
the WMV9 decoder to decode some kinds of frames.
From the patch by Alberto Delmás <adelmas@gmail.com>
This reverts commit 484a337cd7.
These functions were used in f8bed30 "VC1: merge idct8x8, coeff
adjustments and put_pixels" which was reverted in 18b6a69.
Signed-off-by: Mans Rullgard <mans@mansr.com>
Instead, use it on the first member, since by definition, if
any member is aligned, the whole struct must be, in order to
maintain that alignment.
Fixes compilation with some finicky compilers.
Idea for fix from Måns Rullgård.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
The bitstream buffer must be padded, or the bitstream reader might
read over the end.
Fixes the following valgrind warning:
Use of uninitialised value of size 8 at 0x591BAE: cllc_decode_frame (cllc.c:166)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Put the zero length check in place of code that was never used
during decoding, as zero-length slices were generally refused
in decode_frame().
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
This table is used only by mpegaudiodsp and mpegaudioenc. Separating
it allows dropping some dependencies from mpc[78] and qdm2.
Signed-off-by: Mans Rullgard <mans@mansr.com>
Pass pointer to sample buffer instead of channel number to various
functions called from decode_subframe(). Also simplify a few
expressions within this function.
