This was handled properly in the normal return case at the end
of the function, but not in this special case.
Returning a value larger than the input packet size can cause
problems for certain library users.
Returning the actual input buffer size unconditionally, since
it is not guaranteed that frame_size is set to a sensible
value at this point.
Cc: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8f24c12be7)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
If the channel mapping map multiple output channels to one
input channel, we should only increment the actual pointer once.
Cc: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 68e57cde68)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Damaged frames can lead to a mismatch, which can cause a segfault
due to using an incorrect channel mapping.
CC:libav-stable@libav.org
(cherry picked from commit d7c450436f)
Conflicts:
libavcodec/ac3dec.c
If there is an error during frame parsing, but AVCodecContext.channels was
changed and AC3DecodeContext.out_channels was set previously, the two may not
match.
Fixes CVE-2012-2802
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
That way all mix levels as exported by avpriv_ac3_parse_header()
will have the same meaning.
Previously the 3-bit center mix level for E-AC-3 was used to index in a
4-entry table, leading to out-of-array reads.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
This allows the AC-3 decoder to be used directly with RealMedia
decoders that unlike the libavformat one do not byte-swap automatically.
Since the new code is only used in case we would fail directly otherwise
there should be no risk for regressions.
Depending on error_recognition is not correct, low values do
certainly not mean it is ok to crash.
Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
This function is not tightly coupled to mdct, and it's in the way
of making a fixed-point mdct implementation.
Signed-off-by: Mans Rullgard <mans@mansr.com>
This removes the rather pointless wrappers (one not even inline)
for calling the fft_calc and related function pointers.
Signed-off-by: Mans Rullgard <mans@mansr.com>
This will be beneficial for use with the audio conversion API without
requiring it to depend on all of dsputil.
Signed-off-by: Mans Rullgard <mans@mansr.com>
