See https://trac.videolan.org/vlc/ticket/14022#comment:6
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4851db80a4f80ddade1d50d2ec741375c763f001)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 732c3ebffaff5005367d7f947fa903f3b6e92f68)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b4e1922197a8955669f8ab3589c8cef)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Change register constraint on the v variable from = to +. This was causing GCC
to think that the v variable was never read and therefore not initialize it.
This fixes about 20 fate failures on mips64el.
Signed-off-by: James Cowgill <james410@cowgill.org.uk>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9de1303a6414174ab2f3bccefa801bfabcf0f88)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
log2 values which imply numeric overflow are not supported
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d677330e023aac2f4bd3f624039256b9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-oob_4d4eb0_3994_cov_3169972261_zmbv_15bit.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f5c7781e63d6519192ada59c1e36bcecc92791d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes integer overflow and out of array read.
Fixes: signal_sigsegv_24169e6_3445_cov_3778346427_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f4cbf940212079a34753c7f4d6c6b5a43586d30)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 29bbc1be488ea4fc591d3e0ef12f0fc7c8812afb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This also fixes a memleak
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4da351ff0cff460db2110cf22f2e3eded8733a58)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d654546a99f8ddb045ca4fa5d33778a617e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes the use of uninitialized values in avpriv_do_elbg.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab759f8f4a3f7178361e32ab719e6bc49d8afecb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes invalid reads of the packet buffer in av_dup_packet
Based on patch by Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d96142e9af92ded84f2580620c571ab96c4bb657)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes freeing the frame buffer twice on cleanup leading to a crash.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39e4ed7c1d8d840be47f6d604704d47a59a9ae5d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl
fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.
See also https://bugs.gentoo.org/show_bug.cgi?id=471756
$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints
4551 -> 4509 dezicycles
Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a558153a3c464edec1378d637a755fe)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295c51d100101e0ca0434e199ad6de6b5)
Conflicts:
libavcodec/h264.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 383fddeec65f4cebcb197eae702dfefdc6192eb0)
Found-by: jamrial
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Tested-by: Andreas Haupt
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cab6302534962331753fb69c674df86a458b098d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi
Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2fd9ce92af43e6dcbc8ed7c26c00b052de48ccad)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This might fix a hypothetical race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed61103f9fa8fdda41473a23da016bdaa)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This reverts commit 3b4ffba3af968ae702e3a44f6b5f53445efc7363.
Unbreaks the SSSE3 code on mingw32
Conflicts:
libavcodec/x86/lossless_audiodsp.asm
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6c2c8fe3f076d14c0170b4d1965a0dea0393b8d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This is simpler and more robust, and fixes mismatching XMM save restore
mismatches
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1214763af1abf5d7f49b98f88c06e13b98932a6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This is needed as the mmx code is used as fallback from the ssse3 code
Suggested-by: jamrial
Tested-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b4ffba3af968ae702e3a44f6b5f53445efc7363)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes integer overflow and out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fd52d2d3d1ee41822a9801dffd41c0e1a2db32a8)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095660982cc0bc63242c459561fa37037)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
also add GET_TMP2() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc35df893175cedd65c14a4f7ebab6f33aeae4a6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4531e2c489d279bfc90d54ca26ed898c5b265a7f)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The mb address fits in int
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 592ba6ec106206f97133c9345313010c76361e12)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This is probably unneeded and normal int would be fine, but its
safer to use LL and this isnt speed relevant
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4ad2853c50d055e9ba8c29f2e1c83b292f29d7a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The vec_ste calls were mistakenly changed to vec_vsx_st in c5ca76a, which
caused stack smashing.
Changing them back fixes crashes on ppc64el, when configured with
--toolchain=hardened.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 840c3c05316a59c70a7470ed27aaa9c2f3ba410a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Check memory earlier, check one more allocation and clean up on error.
CC: libav-stable@libav.org
Bug-Id: CID 1257773
(cherry picked from commit 014b6b416fec89777cb9cff61bcf7896eaf7cf39)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes CID1260704
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e172f5e53ae4dbbcdcf81c9a3b962dc9f5a8a98d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If refdata was NULL, the memcpy() ended up copying the same memory
block onto itself, which is not only pointless, but also undefined
behavior.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 921706691a87c3ea5f5b92afd9b423e5f8c6e9d9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The parser must always set the out_size and out_data pointers. The API
seems to require it, and the common code in parser.c also relies on it.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b88e80589bd11ef935a5e9dab53d4edb00de16e4)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
dvdsub_decode() can call append_to_cached_buf() 2 times, the second time
with ctx->buf as argument. If the second append_to_cached_buf() reallocs
ctx->buf, the argument will be a pointer to the previous, freed block.
This can cause invalid reads at least with some fuzzed files - and
possibly with valid files.
Since packets can apparently not be larger than 64K (even if packets are
combined), just use a fixed size buffer. It will be allocated as part of
the DVDSubContext, and although some memory is "wasted", it's relatively
minimal by modern standards and should be acceptable.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 816577716bc6170bccfea3b9e865618b69a4b426)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Attemtping to decode them could lead to invalid writes with some fuzzed
samples.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcaa9099b3648b47060e1724a97dc98b63c83702)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The code blindly trusted buffer offsets read from the file in the RLE
decoder. Explicitly check the offset. Also error out on other RLE
decoding errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9151de7c42553bb145be608df8513c1287f1f24)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This simplifies identifying from which revision a binary of a lib came from
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 649c158e8c94ac0cff7f03e97d6ea8bbf71b7f02)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
